Documenti di Didattica
Documenti di Professioni
Documenti di Cultura
Based Approach
Prof Megat Johari Megat Mohd Noor
UTM QRIM@KL & MJIIT
Lahore/Faisalabad/Jamshoro, Pakistan
September 2016
Introduction
We are joining the Mutual Recognition
Train (MRT) !
3
EDUCATION ACCORDS PRACTICE AGREEMENTS
IPEA
International Professional Engineers Agreement
WASHINGTON (ENGINEERS MOBILITY FORUM)
ACCORD 4 YEARS
APEC ENGINEER
SYDNEY IETA
ACCORD 3 YEARS International Engineering Technologist Agreement
(ENGINEERING TECHNOLOGISTS MOBILITY FORUM)
DUBLIN AIET
ACCORD 2 YEARS
Agreement of International Engineering Technician
Development Development
New Accords
WA signed by 6 of formal of graduate
and
organisations peer review attribute
Agreements
processes exemplars
1990s 2001
28 Sep 1989 1997-2015
onwards onwards
7
Importance of Accreditation to Institutions of Higher
Learning
Nominator
• Nominator (EME, GIKI) Prof Abang (MAL)
Prof Megat (MAL)
Prof Lock (SIN)
Mentor
• Mentor (Islamabad, Topi, Risalpur,
Prof Megat (MAL)
Faisalabad, Peshawar, Karachi, Ir Azlan (MAL)
Lahore) Prof Lock (SIN)
Reviewer?
• 1a Reviewer (Universities?) Kim (Korea)
Collins (UK)
• 1b Reviewer (ADM) Basil (NZ)
9
Constituents in Accreditation
Representatives of
Universities DECIDE
ENDORSE
10
EAB Manual
• From input based to outcome based
20.. 2014 11
Students
(…..)
15
Engineering & Technology Domain
Engineers
Supervision &
Research & Design Work
Maintenance
Technologists
Education
Strong in Appropriate
Mathematics, Mathematics,
Engineering Engineering
Sciences, Sciences,
Professional Professional
courses courses
(Theoretical) (Practical)
Engineering Technology
Breadth & Depth Breadth & Depth
of Curricula of Curricula
16
Educational Process & Stakeholders Internal Stakeholders
Teachers
Students
Pull Programme EO / O Development/ Review University
Specification
factor
External Stakeholders
Course O / Content Potential Employers / Industry
Alumni
Development / Review Regulatory Body
1, 2, 3 ……
Formative / Summative
Summative - direct
Exit Survey - indirect
Industry Survey - indirect
Alumni Survey - indirect External Stakeholders
External – direct Potential Employers / Industry
Accreditation - direct Alumni
Regulatory Body
External Assessor 17
Academic IQA Practices in Perspective
and administration
MISSION
STAKEHOLDER’S SATISFACTION
learning outcomes
Review
19
Characteristics of OBE curricula
24
Strategy of OBE
• Top down curricula design
• Appropriate Teaching & Learning Methods
• Appropriate Assessment & Evaluation
Methods
25
Different Levels of Outcomes
26
Accreditation
Preparation for Accreditation
• Comprehend the EAB Manual
• Prepare the SAR
• Address previous accreditation report
• Arrange the evidence
• Complete the Checklist
• Assign key persons according to accreditation
schedule
28
SAR & Evidence
29
Azlan
or
a man sitting
down in a
garden and a
lady passing
by?
30
Issues from
• Approval Report
• First Year Visit Report
• External Examiner’s Report
CQI
• Stakeholders’ Recommendations
• Recent Accreditation Visit
Documents & Records
EVIDENCE BASED Interviews
Observations
Exercise
Self Assessment Report
• Prepare a checklist of questions based on the following
criteria in preparing a SAR:
– Programme Objectives
– Programme Outcomes
– Curriculum and Learning Process
– Students
– Faculty
– Support staff
– Industry stakeholders
– Alumni
– Facilities and Infrastructure
– Institutional and Financial Support
– Continual Quality Improvement
– Industry Linkage
Exercise
Industrial Linkage
Using the Clause 9 of the PEC Manual, evaluate
the relevant part of the submitted SAR.
3.2.9 Criterion 9: Industrial Linkages
Students are expected to undertake assignments
from industry to provide solutions to complex
engineering problems. Students and faculty should
be encouraged to establish collaboration for R&D
and product development related projects, with
due regard to environmental and societal impact.
Feedback from the industry and employers is
crucial and an essential part of curriculum review
process used to evaluate attainment of the
program objectives.
4.1.9 Industrial Linkages
4.1.9.1
Discuss the involvement of
industry in discussions and forums,
professional practice exposure, and
collaborative projects / research
for the solutions to engineering
problems.
Exercise
36
Exercise
37
PROGRAMME EVALUATOR’S APPROACHES
• Sensible questioning
• Check records
• Observing processes
• Analyse inputs and outputs
• Organised using tables, matrices,
flowcharts and checklists
38
Quiz
39
Quiz
40
Cause for concerns at
Accreditation Decision Meeting
• Phases of OBE
– Planning
– Implementation
– Effectiveness
• CQI
• List of concerns
• Breadth & depth (taxonomy & complex problem)
• Staffing
• Industrial Training
• Commitment to change
• System failure
• Stagnant (no improvement)
• Repeat offender
• Safety 41
Rubrics for New Programme, New
Cycle & Continuing
42
What WA will be observing?
• Adherence to EAB document
• EAB PEV’s aplomb and decorum
• Probing questions (not interrogative)
• Discussion level
• Clarity of reports
• Graduate outcomes
• Health & safety at HEIs
• Equivalency of practice
43
Exercise
Opening Meeting
You are the Dean/HoD with three programmes
to be evaluated; Mechanical Engineering, Civil
Engineering and Electrical Engineering, for the
third cycle. Prepare a list of talk points to
address the Programme Evaluators (PEVs) at the
Opening Meeting.
44
OPENING MEETING
• Welcome Evaluation Team
• Introduce team members
• Corrective & Preventive
Actions from previous
accreditation
• Short presentation on 10 minutes
Faculty/Dept/Prog strengths
• Fill up with the latest (within
a specified timeframe) if any
45
PEC Manual 2014
Programme Educational Objectives
46
Exercise
List down potential stakeholders
• Major
• Minor
Programme Educational Objectives
48
Programme Educational
PROGRAM EDUCATIONAL Objectives
OBJECTIVE (PEO)
50
Exercise
51
PEC Manual 2012
Programme Outcomes
52
Programme Outcomes (PO)
Programme Outcomes
• Discuss on HEI’s possible approaches or
methods to demonstrate implementation of
the 12 programme outcomes
• Discuss on the possible models to show
attainment of the 12 programme outcomes
54
PEO
WHAT YOU WANT YOUR GRADUATES TO BE IN 3 - 5 YEARS
WA3 WA9
DESIGN IND & TEAM
WA 1
UNIVERSITY EXPERIENCE
WA10
WA5
EXTRA-CURRICULAR
ENGINEERING COMMUNICAT-
KNOWLEDGE MODERN TOOLS
ION
WA 2
PROBLEM
4 YEARS WA11
WA6 ENGR & SOC
ANALYSIS PROJ MGMT &
WA7 ENV & SUST
FINANCE
WA8 ETHICS
WA4 WA12
INVESTIGATION LIFE LONG
Problem Organised Project Work
or POPBL (Project Oriented Problem Based
Learning)
Literature Lectures Group Studies
57
POPBL Requirements
• High degree of supervision
• Office space
• Lectures to be constantly changing or
renewed
• Flexibility in the distribution of resources
58
Instructors/Supervisors
• Pedagogical skills
• Scientific skills
• Time management
• Project based on staff research
59
Graduates’ Strength
60
Typical questions on PEO/PO
• What are the PEOs/POs?
• Who were involved in the development of the
PEOs/POs?
• How were they developed/improved?
• To what extent the stakeholders were
involved?
• How their attainment were determined?
• What were the improvements introduced?
61
Exercise
Curriculum
• Discuss on the possible relationship between
taxonomy levels and the different knowledge
profile with consideration to the 12 programme
outcomes
• What are typical probing questions in
ascertaining that student’s POs have been
attained?
• What are typical probing questions in
ascertaining that student’s COs have been
attained?
62
ASSESSMENT:
Processes that identify, collect, use and
prepare data for evaluation of achievement
of programme outcomes or educational
objectives.
EVALUATION:
Processes for interpretation of data and
evidence from assessment practices that
determine the programme outcomes are
achieved or result in actions to improve
programme. 63
Outcome-Based Assessment
Implementation Assessment Data
Strategy Strategy Sources/Assessment
instruments
Industrial project Exams, interview, Reports, interview
Improve student survey, observe, schedule, survey,
competence in assess skill level, observation records,
communication, monitor grades of exams and
teamwork, and project development of projects, exit skill
management skills checklist
Design course Assessment by List of assessment
Address industry industry, and criteria, observation,
needs lecturers reports, interview,
students evaluation,
exams, exit skill
checklist 64
Assessment
66
What Assessment?
• Assessing Student/Cohort (Course Outcome)
• Assessing Student/Cohort & Faculty
(Programme Outcome)
67
Assessment Process
68
MEASURE & EVALUATE
• Assessment Plan
• Who is doing what and when
• Stakeholder participation
• CQI in place
71
Programme Outcome Assessment
Matrix
Outcome indicators PO 1 PO 2
& core courses
Project Report A B
Course 1 B B
Course 2 C B
Table 2 PO 1 PO9
Q1 CO1 + CO2 +
Q2 CO2 + CO3 +
Q3 CO3 + CO4 +
Q4 CO4 +
78
Quiz
Delivery Assessment
Lecture
Laboratory
PBL
Case Method
Project Based
Programme or
Student
Improvement ?
Selective
Culminating
Hybrid
S&A
Yr. 4 S&A 30%
30%
Yr. 2
S&A
30%
Yr. 1 S&A
30%
A B C D 82
Evaluation of Outcomes at Programme Level
ECV3092 Civil Engineering Design
MyOBE
(Capstone Design Course) Software
83
Evaluation of Outcomes at Programme Level
ECV3092 Civil Engineering Design
MyOBE Process
(Capstone Design Course) Module
Programme Evaluation
Programme Objectives
Programme Outcomes Evaluation
Summative
Stakeholders’ Survey
Trend Analysis
84
Evaluation of Outcomes at Programme Level
ECV3092 Civil Engineering Design
MyOBE
(Capstone Design Course) Snapshots
Lecturers’ Module:
Enter all course
assessment marks
85
Evaluation of Outcomes at Programme Level
ECV3092 Civil Engineering Design
MyOBE
(Capstone Design Course) Snapshots
86
PO Attainment
Final Year
Final Year Project Final Year Courses
Design Project
87
What constitutes strength?
Exceeds the minimum standard set by the EAC
Engineering Accreditation Manual.
Extensive benchmarking (not only via the external
examiners path) with more established
programmes/institutions.
The curriculum is built on strong fundamentals
(engineering sciences) and appropriate engineering
knowledge according to the discipline, which transcend
national boundaries.
Generic attributes (professional and/or interpersonal
skills) should also be evident to prepare graduates for the
advanced part of their career.
88
What constitutes strength? Cont…
• A curriculum with clear (measurable) objective(s)
and outcomes (that satisfies the (12) EAC stipulated
outcomes)
• Involved stakeholders, both internal and external,
extensively
• An appropriate working load for students
determined through extensive consultation with the
academics (Usually a 15 – 16 credit per semester
loading)
• Blend of delivery methods
89
What constitutes strength? Cont…
Programme challenges students to achieve greater
heights than just satisfying the minimum standard
Attain competency in the open-ended project based
and problem oriented courses
Majority of the staff has PhD qualification and the
number available indicates a low staff-student ratio
(that enables greater contact with students)
The academic staffs also conduct research that
permeates/contributes to teaching and learning.
90
What constitutes strength? Cont…
• Over and above Industrial Training (extensive &
distributed professional exposure) that does not
compromise on the cognitive domain
• Ergonomics is taken seriously by the institution to
reduce occupational hazard
• Safety culture
• Show that they have the plan and the completion of
the quality cycles is widespread
• Monitoring of the QMS also indicates strength.
91
What constitutes strength? Cont…
• Students’ ability to give opinion and articulate
with substance
• Students are clear of their goals upon graduation
and highly motivated during their course of
study (“constructive criticisms”)
• Widespread involvement of students in co-
curricular activities (not forced as part of
curriculum nor limited to small group of
students).
92
What constitutes strength? Cont…
• Academic staff with Professional Engineer
status
• Academic staff are actively participating in
professional activities (not merely members)
• Design courses are taught by experienced
academics (with consultancy experience or
professional engineers).
93
What constitutes strength? Cont…
• Up to date facilities are made available and
they exceed the recommended student-
equipment ratio appropriate to the relevant
discipline.
• Extensive electronics publications for life long
learning, project based courses and the final
year project
94
Taxonomy & Course Outcome
Bloom’s Taxonomy
• Knowledge (list)
• Comprehension (explain)
• Application (calculate, solve, determine)
• Analysis (classify, predict, model,derived)
• Synthesis (design, improve)
• Evaluation (judge, select, critique)
96
97
lower order Intermediate Higher order 98
lower order Intermediate Higher order 99
Why are course outcomes important?
Poor
• Students should be able to design research.
Better
• Students should be able to independently design
and carry out experimental and correlational
research.
Best
• Students should be able to independently design
and carry out experimental and correlational
research that yields valid results.
Source: Bergen, R. 2000. A Program Guideline for Outcomes Assessment at Geneva College
102
WA Knowledge Profile (WK)
103
WK1 WA Knowledge Profile WK5
natural sciences engineering
(WK) design
WK2
mathematics,
numerical
WK6
analysis,
engineering
statistics,
practice
computer and
information
science
4 YEARS WK7
WK3 engineering in
engineering society
fundamentals
WK4 WK8
engineering research
specialist literature
knowledge 104
WA Knowledge Profile (Curriculum)
Theory-based natural sciences WK1
Conceptually-based mathematics, numerical WK2
analysis, statistics and formal aspects of
computer and information science to
support analysis and modelling
Theory-based engineering fundamentals WK3
Engineering specialist knowledge that WK4
provides theoretical frameworks and bodies
of knowledge for the practice areas; much is
105
forefront
WA Knowledge Profile
Knowledge that supports Engineering design in WK5
the practice areas
Knowledge of Engineering practice WK6
(technology) in the practice areas
Comprehension of the role of Engineering in WK7
society and identified issues in engineering
practice: ethics and professional responsibility
of an engineer to public safety; the impact of
engineering activity: economic, social,
cultural, environmental and sustainability
Engagement with selected knowledge in the WK8
Research literature 106
WA Programme Outcome or
Graduate Attributes (WA)
107
Washington Accord Graduate Attributes
PROGRAMME OUTCOMES
WA1 Engineering Knowledge Breadth & depth of knowledge
WA2 Problem Analysis Complexity of analysis
WA3 Design/Development of Breadth & uniqueness of engineering problems i.e. the extent to
Solutions which problems are original and to which solutions have
previously been identified and coded
WA4 Investigation Breadth & depth of investigation and experimentation
WA5 Modern Tool Usage Level of understanding of the appropriateness of the tool
WA6 The Engineer and Society Level of knowledge and responsibility
WA7 Environment and Type of solutions
Sustainability
WA8 Ethics Understanding and level of practice
WA9 Individual and Team Work Role in and diversity of team
WA = Programme Outcome
WK = Knowledge Profile = Curriculum 109
PROGRAMME OUTCOME
Problem Analysis - Complexity of analysis
(WA2) Identify, formulate, research literature
and analyse complex engineering problems
reaching substantiated conclusions using first
principles of mathematics, natural sciences and
engineering sciences (WK1 – WK4)
110
PROGRAMME OUTCOME
Design/Development of Solutions – Breadth and
uniqueness of engineering problems i.e. the extent
to which problems are original and to which
solutions have previously been identified or codified
(WA3) Design solutions for complex engineering
problems and design systems, components or
processes that meet specified needs with appropriate
consideration for public health and safety, cultural,
societal, and environmental considerations (WK5)
111
PROGRAMME OUTCOME
Investigation - Breadth & Depth of Investigation
& Experimentation
(WA4) Conduct investigation of complex problems
using research based knowledge (WK8) and
research methods including design of
experiments, analysis and interpretation of data,
and synthesis of information to provide valid
conclusions 112
PROGRAMME OUTCOME
Modern Tool Usage - Level of understanding of the
appropriateness of the tool
(WA5) Create, select and apply appropriate
techniques, resources, and modern engineering
and IT tools, including prediction and modelling, to
complex engineering problems, with an
understanding of the limitations. (WK6)
113
PROGRAMME OUTCOME
The Engineer and Society - Level of knowledge
and responsibility
(WA6) Apply reasoning informed by contextual
knowledge to assess societal, health, safety, legal
and cultural issues and the consequent
responsibilities relevant to professional
engineering practice and solutions to complex
engineering problems. (WK7) 114
PROGRAMME OUTCOME
Environment and Sustainability - Type of solutions
(WA7) Understand and evaluate the sustainabilty
and impact of professional engineering work in the
solutions of complex engineering problems in
societal and environmental contexts (demonstrate
knowledge of and need for sustainable
development) (WK7)
115
PROGRAMME OUTCOME
Ethics - Understanding and level of practice
(WA8) Apply ethical principles and commit to
professional ethics and responsibilities and norms
of engineering practice. (WK7)
116
PROGRAMME OUTCOME
Individual and Team Work – Role in and diversity
of team
(WA9) Function effectively as an individual, and as
a member or leader in diverse teams and in multi-
disciplinary settings
117
PROGRAMME OUTCOME
Communication – Level of communication
according to type of activities performed
(WA10) Communicate effectively on complex
engineering activities with the engineering
community and with society at large, such as being
able to comprehend and write effective reports
and design documentation, make effective
presentations, and give and receive clear
instructions 118
PROGRAMME OUTCOME
Project Management and Finance – Level of
management required for differing types of
activity
(WA11) Demonstrate knowledge and
understanding of engineering and management
principles and economic decision-making and
apply these to one’s own work, as a member and
leader in a team, to manage projects and in
multidisciplinary environments 119
PROGRAMME OUTCOME
Life-long Learning – Preparation for and depth of
continuing learning
(WA12) Recognise the need for, and have the
preparation and ability to engage in independent
and life-long learning in the broadest context of
technological change
120
WK1 WA9 WK5 WA3
natural sciences
IND & TEAM
engineering DESIGN
design
WK2
mathematics,
WA1 numerical WA10
WK6 WA5
ENGINEERING analysis,
engineering
COMMUNICAT-ION MODERN TOOLS
KNOWLEDGE statistics,
practice
computer and
information
WA2
PROBLEM
4 YEARS
science
WA6 ENGR & SOC
ANALYSIS WA11 WK7 WA7 ENV & SUST
WK3MGMT &engineering
PROJ FINANCE in
engineering society WA8 ETHICS
fundamentals
122
Complex Problem
Difficult Decision
Uncertain Strategy
Confusing Idea
Contentious Product
Intractable Change
123
Difficulty & Uncertainty
• Complexity – the problem contains a large
number of diverse, dynamic and
interdependent elements
• Measurement – it is difficult or practically
unfeasible to get good qualitative data
• Novelty – there is a new solution evolving
or an innovative design is needed
124
Scientific/Technical
Problems A
can combine to Complex Problem
form
125
Limited
Explanation,
Explanation,
Prediction,
Prediction,
Control
Control
Isolatable
Unbounded Results in an
Systems, Results in a
Systems, No educated
Controlled Covering Law
Experiment guest
Experiment
Complex
A limited
All the Salient
number of
Difficult to features are
Measurable features are
measure captured by
captured by
the Model
the Model
Operating with Operating with
scare adequate
resources resources 126
Characteristics
Technical Problems Complex Problems
• Isolatable boundable problem • No definitive problem boundary
• Universally similar type • Relatively unique or unprecedented
• Stable and/or predictable • Unstable and/or unpredictable
problem parameters problem parameters
• Multiple low-risk experiments are
possible • Multiple experiments are not
possible
• Limited set of alternative
solutions • No bounded set of alternative
• Involve few or homogeneous solutions
stakeholders • Multiple stakeholders with different
• Single optimal and testable views or interest
solutions • No single optimal and/or objectively
• Single optimal solution can be testable solution
clearly recognised • No clear stopping point
127
Complex Problems (Need High Taxonomy Level)
Complex Engineering Problems have characteristic WP1 and some or all of WP2 to WP7, EP1 and EP2, that
can be resolved with in-depth forefront knowledge
129
Complex Engineering Activities (Project based)
Preamble Complex activities means (engineering) activities or
projects that have some or all of the following
characteristics listed below
Range of Diverse resources (people, money, equipment,
resources materials, information and technologies).
Level of Require resolution of significant problems arising
interaction from interactions between wide ranging or
conflicting technical, engineering or other issues.
Innovation Involve creative use of engineering principles and
research-based knowledge in novel ways
Consequences to Have significant consequences in a range of
society and contexts, characterised by difficulty of prediction
the environment and mitigation.
Familiarity Can extend beyond previous experiences by
130
applying principles-based approaches.
WA – WK – WP Relationships
WA1 – Engineering Knowledge WK1 - natural sciences (WA1) (know what)
(Science, Mathematics & Engineering)
(WK1, WK2, WK3, WK4) WK2 - mathematics, numerical analysis,
statistics, computer and information science
(WA1)
to solve WK3 - engineering fundamentals (WA1)
Complex Engineering Problems
WK4 - engineering specialist knowledge
(WA1)
138
Assessment
• Report – style and content (flow)
• Display – attractive ?
• Viva / Articulation
• Teamwork
• Management – scheduling
139
Example 2: Complex Problem Solving
River
Spring
Fissured Rocks
Sandy soil
Clayey soil
Igneous rock
140
How does complexity relates to
curriculum?
• General Subjects
• Industrial Placement
• Core & Specialist (Engineering) Subjects –
Complex Problem Solving
• Elective Subjects – Complex Problem Solving
• Design Project – Complex Engineering
Activities
• Final Year Project – Complex Problem Solving 141
Closing Remarks
142
Thank You
Appendix
144
Complex Problem Solving (CPS)
• Dynamic, because early actions determine the
environment in which subsequent decision must
be made, and features of the task environment
may change independently of the solver’s actions;
• Time- dependent, because decisions must be
made at the correct moment in relation to
environmental demands;
• Complex, in the sense that most variables are not
related to each other in a one-to-one manner
145
Microworld CPS Model
• The problem requires not one decision, but a
long series, in which early decisions condition
later ones.
• For a task that is changing continuously, the
same action can be definitive at moment t1
and useless at moment t2.
• Include novel solutions to an old dilemma in
general science (external validity vs.
experimental control)
146
Expert-novice CPS Model
• Expert-novice approach most of the time
produces conclusions that are crystal-clear.
• It almost guarantees statistically significant
results, because the groups compared (expert
and novices) are very different and tend to
perform very differently when confronted with
similar experimental situations (Sternberg
1995).
147
Naturalistic decision making (NDM)
• Naturalistic decision making (NDM) (e.g.,
Zsambok and Klein 1997, Salas and Klein
2001)
• ‘real-world’ task
• Example interviewing firefighters after
putting out a fire or a surgeon after she has
decided in real time what to do with a
patient.
148
Dynamic decision making DDM
• Dynamic decision making (DDM) (Brehmer
1992, Sterman 1994).
• Discrete dynamic decision tasks that change
only when the participant introduces a new
set of inputs.
• Variables like time pressure have been
successfully integrated in models like
Busemeyer and Townsend’s (1993) decision
field theory
149
Implicit learning in system control
• This tradition has used tasks like the sugar
factory (Berry and Broadbent 1984) or the
transportation task (Broadbent et al. 1986), that
are governed by comparatively simple
equations.
• The theorization and computational modeling in
this branch of CPS are extremely rich. Models
are based on exemplar learning, rule learning,
and both (e.g., Dienes and Fahey 1995, Gibson
et al. 1997, Lebiere et al. 1998).
150
European complex problem solving (CPS)
152
Variable related
• Number and type (discrete/continuous) of
variables
• Number and pattern of relationships
between variables
• Non-Linear - Linear
153
System behaviour related
• Opaque - transparent.
• Stochastic - deterministic
• Delayed feedback - immediate feedback.
154
Delivery
• Knowledge-lean vs. knowledge-intensive
• Skill based vs planning based (reactive vs
predictive
• Learning vs. no learning during problem
solving
• Understanding-based vs. search-based
problems
• Ill-defined vs. well-defined
155
Conclusion
• Problem solving has been traditionally a
task-centered field. VanLehn (1989) think
that ‘task’ and ‘problem’ are virtually
synonymous.
156
157
The author would like to thank the contributors of the clip arts
used in this presentation
158