Outcome Based Teaching and Learning

Edited by: Aman Shah and Andrea Hope Published by: Hong Kong Shue Yan University 10 Wai
Tsui Crescent Braemar Hill, North Point Hong Kong Please direct queries and suggestions to: Ms. Edith Leung Email: pkleung@hksyu.edu First Published in October 2012
2012 by Hong Kong Shue Yan University
Foreword by Editors
There is a large number of resources on the web relating to Outcomes Based Teaching and Learning (OBTL). This Handbook is intended as a general guide for academic staff to support and strengthen the implementation of OBTL in Shue Yan University. It is hoped that this short guide will facilitate and disseminate good practice relating to OBTL within the University. It supplements the information contained on the University website and in other University resources. Its purpose is to provide a convenient source of useful information for academic staff members in the design and implementation of OBTL approaches to courses of study within their area of responsibility.
Aman Shah, Associate Vice President (Quality Assurance) Andrea Hope, Associate Academic Vice President
Acknowledgements
We are grateful for the contributions of Linda Gross, Education Consultant, for chapters 2-4. We are also grateful to Dr. Alex Li and Dr. L.Y. Lo, Department of Counselling and Psychology for agreeing to the use of their course material in chapter 2, Dr. Karen Lee, Department of Law and Business and Dr. Mark Greene, Department of Counselling and Psychology for agreeing to the use of their material for chapter 4 drawn from the presentations given by them in the first OBTL Experience Sharing Session in Hong Kong Shue Yan University, in September 2012. We thank them for their contributions. We thank Edith Leung, Registry, for formatting the text and helping us with the production of this Handbook. We also acknowledge the work of those who have generously shared online resources through access to their websites. Every attempt has been made to credit sources of information referred to and used in the Handbook. Should a source have been missed, please contact the editors.
October 2012
Message from the Academic Vice President
Over the past four decades, from being a post-secondary college to becoming Hong Kongs first private university, Hong Kong Shue Yan University has played an active role in extending the range of higher education opportunities for senior secondary school graduates in Hong Kong. The University has vigorously ensured the quality of the degree programmes it offers, by taking seriously the recommendations of the Hong Kong Council for the Accreditation of Academic and Vocational Qualifications and by rigorously adhering to the Universitys academic quality assurance protocols.
In this respect, the introduction of Outcomes Based Teaching and Learning since 2009 is an important development for the University, which reflects the fact that a student-centred approach to learning is at the heart of our mission. Its full implementation has my wholehearted support, and I believe this Handbook will engage colleagues in the reflective practice of teaching and facilitate continuous quality enhancement of the courses and programmes of study we offer.
Professor Y.S. Hu Academic Vice President
October 2012
ii
CONTENTS
Page i i ii 1
Foreword Acknowledgements Message from the Academic Vice President 1. Implementing Outcomes Based Teaching and Learning in SYU 1.1 Purpose/Intended Outcomes 1.2 What is OBTL? 1.3 The Context 1.4 The Starting Point 2. Intended Learning Outcomes 2.1 What are Intended Learning Outcomes (ILOs)? 2.2 Writing Intended Learning Outcomes 2.3 Programme and Course Intended Learning Outcomes 2.4 Tips to Address Common Pitfalls 2.5 Mapping Intended Learning Outcomes 3. Teaching and Learning Issues 3.1 Tips on Making OBTL Work in the Classroom 3.2 Variety of Teaching and Learning Strategies 3.3 Choosing Learning Activities That Support ILOs 3.4 Importance of Feedback 4. Assessment Issues 4.1 How Does Assessment Work in an OBTL Environment? 4.2 Characteristics of Sound Assessment 4.3 Common Weaknesses in Assessment 4.4 Developing an Assessment Plan 4.5 Assessment Tasks 4.6 Marking and Grading Issues 4.7 Developing Rubrics 4.8 How to Determine Whether your Assessment Plan is Working Glossary Appendices References
18
22
31 32 65
HKSYU OBTL Handbook
Chapter 1 Implementing Outcomes Based Teaching and Learning (OBTL)

Purpose/Intended Outcomes
The main purpose of this Handbook is to help colleagues in SYU in the design and delivery of courses and programmes in the OBTL mode and to provide an opportunity for them to reflect on the courses and programmes they have already designed and delivered. By using the Handbook, colleagues should be able to: 1. design courses and programmes in the OBTL mode; 2. reflect on, and pursue changes as necessary in, existing courses and programmes for delivery in the OBTL mode; and 3. critically review the effectiveness of their approach to OBTL on student learning.
What is OBTL?
Although other acronyms have been used in the literature on the subject to describe various approaches to teaching, learning, assessment and course design in the learning outcomes mode, for the sake of simplicity and consistency, only one expression is used in this Handbook: OBTL, meaning Outcomes Based Teaching and Learning. The basic philosophy of OBTL is described by Biggs and Tang, p.177 1 as follows: The logic is stunningly obvious: Say what you want students to be able to do, teach them to do it and then see if they can, in fact, do it. OBTL is a student-centred approach to teaching and learning. The origins of OBTL can be traced back to Ralph Tyler (1949, p.63) 2 who describes learning as taking place through the action of the student: "It is what he does that he learns, not what the teacher does". The basic framework of OBTL consists of five main elements: (a) the course content or the syllabus, which is the core; (b) the intended learning outcomes
1
Biggs. J & Tang. C, Teaching for Quality Learning at University, 3 ed, p.177. Open University Press, 2007 2 Tyler, R.W. (1949) Basic Principles of Curriculum and Instruction . Chicago: University of Chicago
rd
HKSYU OBTL Handbook
(ILOs) derived from the course content; (c) the teaching and learning activities (TLAs) designed to facilitate the achievement of those outcomes; (d) the design of assessment tasks (ATs) aimed at assessing the learning outcomes; and (e) assessment rubrics to help assess the achievement of the learning outcomes and to determine the level at which the outcomes have been achieved. The alignment of TLAs with ILOs is as important as the alignment of ATs with ILOs to successfully implement OBTL. In a recent study in Hong Kong on the impact of OBTL on approaches to learning, Pang, To and Man (2007)3 found that OBTL helps students align their learning motives and strategies in forming their personal study contracts. With a focus on the learner, the ILOs also help in promoting deep learning and lifelong learning skills. Other immediate benefits of OBTL are that it promotes reflective practice among faculty, thereby facilitating continuous improvement of courses and programmes, and helps the institution in respect of accountability by providing a means to address institutional outcomes.
The Context
The requirements of accrediting and professional bodies, the expectations of employers and the community in general and the growing sense of consumerism in the higher education sector have all contributed to the need for transparency in the delivery of academic courses and programmes. Teaching is no longer a private affair. It is very much a collaborative affair between and among academic staff and between teachers and students. The Academic Development Plans of the University for the periods 2006-2010 and 2010-2015 clearly identify, and put an emphasis on, the Universitys commitment to continue to develop interactive teaching and learning strategies that facilitate the achievement of learning outcomes in terms of both specialist knowledge and generic skills. OBTL is now the modus operandi in the delivery of academic programmes in a significant number of academic institutions across the world. Professional and accrediting bodies are increasingly assessing academic programmes in the context of OBTL. Quality assurance and accountability are also facilitated through the implementation of OBTL. It is useful to note that spurred by federal regulations, institutional accreditation requirements in the U.S. now
Pang, To and Man, The Impact of Outcome-Based Teaching and Learning: A Case Study In Hong Kong (China), 2007 Oxford Business and Economics Conference, Oxford University,ISBN : 978-0-9742114-7-3
HKSYU OBTL Handbook
demand public display (on websites or publications) of intended learning outcomes for all academic offerings . (Ewell, 2006)4 In the changing educational scenario in Hong Kong in recent times, the introduction of academic audits by the Quality Assurance Council of the UGCfunded institutions related partly to the adoption of an outcome-based approach to student learning by the institutions .. (QAC Audit Manual, p.5)5 The Hong Kong Council for the Accreditation of Academic and Vocational Qualifications (HKCAAVQ) also requires institutions under its scope of work to submit information relating to programme proposals for accreditation under a number of headings, including the following, which point clearly to the expectation that institutions have implemented OBTL: Programme Aims/Objectives Programme Intended Learning Outcomes (PILOs) Programme Design Consolidated Summary of the Contribution of Courses to PILOs Mapping of Courses against the Generic Level Descriptors (GLDs) of the Hong Kong Qualifications Framework Delivery Methods (A general description of the teaching and learning strategies and approaches to meet the PILOs) Assessment Plan
The Qualifications Framework launched by the HK Government in 2008 is underpinned by a philosophy that promotes outcomes based qualifications rather than a traditional focus on educational inputs. 6 SYUs formal effort to implement OBTL began in 2008 when the University invited a consultant, Mr. Aman Shah, who was then the coordinator of the OBTL project in the City University of Hong Kong, to conduct workshops for its academic staff. A series of workshops customised to the needs of individual departments were conducted during 2008-2009. A lot of work has gone into this exercise since then, and by the end of 2009, all course and programme documents have been written in the OBTL format. In the light of the experience of implementing OBTL in 2008-2009, it is necessary to re-visit programme and course documents to ensure the alignment between course intended learning
4 5
P. Ewell, a report to the UGC, 2006 Quality Assurance Council, Audit Manual, Para1.2, p.5, http://www.ugc.edu.hk/eng/qac 6 http://www.hkcaavq.edu.hk/en/news_events_qa.asp
HKSYU OBTL Handbook
outcomes (CILOs) and programme intended learning outcomes (PILOs), between PILOs and the Generic Level Descriptors of the Qualifications Framework, between TLAs and CILOs and between ATs and CILOs, and to review the appropriateness of assessment rubrics. Concurrently, it will be necessary to confirm that the actual delivery of courses in the classroom is in the OBTL mode to ensure the quality of student learning. To be able to move forward to the next level, it is also necessary for colleagues in the University to find opportunities for continuing campus conversations on OBTL. The feedback from the first OBTL Experience Sharing Session, held on 03 September 2012, clearly suggests the usefulness and value of such conversations where colleagues can learn from each others experience and share good practice. The need to continue such conversations in some form or the other is evident from the feedback received. The Experience Sharing Session has identified both strengths and gaps and has generated a level of transparency in the efforts of individual departments, which will undoubtedly help in the process of continuous improvement.
The Starting Point

The starting point for designing programmes and courses in the OBTL mode is institutional goals, followed by the statement of graduate attributes. Other helpful sources of information in this exercise relate to the requirements of the Qualifications Framework and the professional bodies for professional programmes, and feedback from HKCAAVQ (if any on specific programmes), employers, external examiners and advisers, alumni, and professional colleagues in the relevant field. The hierarchical structure of outcomes cascades down from institutional goals to the level of programme outcomes and then course outcomes. A copy of SYUs mission statement and graduate attributes statement are in Appendix A and B respectively. Designing courses and programmes in the OBTL mode is an iterative process. It promotes reflective practice and provides immense opportunity for quality enhancement. Colleagues in SYU have already implemented OBTL. What is now required is to move it to the next level, by ensuring that it is effective in enhancing and enriching student learning experience. This can be done by revisiting the course and programme documents and reflecting carefully on OBTLs implementation and effectiveness in the classroom. Feedback from students and from other stakeholders constitutes an important source of confirming effectiveness.
HKSYU OBTL Handbook
Chapter 2 Intended Learning Outcomes (ILOs)

What are Intended Learning Outcomes?
Intended Learning Outcomes (ILOs) are single-sentence statements that clearly articulate the key elements of what students are expected to know, understand, and/or be able to do on successful completion of their studies for a defined period of learning, at a specified level (e.g. course, programme, project, unit). ILOs do not describe course content or what happens to students during the course. They are not a list of topics from the syllabus! Depending upon the level at which the ILOs are aimed, they will be expressed at different levels of generality and specificity to accommodate both breadth and depth of learning. At the course level, ILOs should explicitly: Identify and describe the range of observable and measurable cognitive and practical abilities that are deemed to be essential to successfully complete the course, project, or unit of a course; Relate directly to the aims of the course, project, or unit. Relate to the Programme Intended Learning Outcomes (PILOs) which should, in turn, relate to Graduate Attributes approved by the University and institutional mission, the requirements of professional bodies, if any, and the Generic Level Descriptors (GLDs) at level 5 for undergraduate programmes and level 6 for postgraduate programmes of the Hong Kong Qualifications Framework (HKQF). A copy of the GLDs at level 5 and level 6 is in Appendix C.
It should be noted that students often learn worthwhile, unpredetermined things while participating in course activities (unintended learning outcomes). And, in some courses, students may be encouraged to set their own learning outcomes (negotiated learning outcomes) that are more appropriate to their personal inquiry as they advance through the programme of study. When writing ILOs, the instructors might consider ways to provide recognition for unintended learning outcomes and negotiated learning outcomes, if deemed appropriate for students in senior classes.
HKSYU OBTL Handbook
Writing Intended Learning Outcomes Why Bother?

Busy academics may discover that the time they spend writing effective learning outcomes results in several benefits7. Learning outcomes are extremely useful in several ways. They: Help students learn more effectively because the learning outcomes, together with clear information on assessment tasks, provide a road map to students on what they are supposed to learn; Make it clear what students can hope to gain from following a particular course of study; Motivate students by providing a sense of direction, thereby enabling them to take responsibility for their own learning; Help instructors to design their teaching materials more effectively by acting as a template for them; Help instructors select the appropriate teaching strategy; for example, lecture, seminar, student self-paced, or laboratory class (it obviously makes sense to match the intended outcome to the teaching strategy); Help instructors tell their colleagues more precisely what a particular activity is designed to achieve; Help to ensure that appropriate assessment strategies and tasks are employed; Provide a useful guide to inform potential candidates and employers about the general knowledge and understanding of a subject that a graduate will possess; Facilitate the compiling of information on students progress in a more coherent manner; and Enable opportunities for self-reflection: for instructors on the design and delivery of the course; and for the students on their own progress in the course, which thereby facilitates quality enhancement of the course and programme.
How to Write Intended Learning Outcomes

When writing learning outcomes statements, it is helpful to remember that what the student does is actually more important in determining what is learned than what the teacher does (Shuell, 1986: p.429). The statements should clearly describe what the student needs to do to provide evidence of learning.
Writing Intended Learning Outcomes A guide: writing intended learning outcomes at module level: NSHU, the Swedish Agency for Networks and Cooperation in Higher Education: http://kursutveckling.se/dok/Larandemal_eng_061011.pdf
HKSYU OBTL Handbook
The ILOs should be written with these characteristics and benefits in mind: Student-Centred Learning ILOs should focus on what students need to do, not what instructors do and should therefore contain action verbs; ILOs should provide direction on what students must achieve by the end of the course; ILOs should be written in a manner that facilitates the design of meaningful teaching activities aimed at supporting student learning and achievement of the ILOs.
Clarity and Transparency ILOs should be worded in user-friendly terms, to let students know what is expected of them (e.g. the knowledge, professional and generic skills, attitudes, attributes that are valued, promoted, and developed in various courses within a programme); ILOs should be written in a manner that responds to a common interest amongst all stakeholders (i.e., students, parents, instructors, potential employers, professional bodies, etc.) in terms of how the courses are linked together within a programme of studies.
Alignment Programme learning outcomes and course learning outcomes should be clearly aligned; Course aims, course learning outcomes and assessment should clearly connect instructors and students should be able to identify the lookfors.
When developing ILOs for the first time, it may be helpful to remember that learning outcomes are best formulated in consultation with others (e.g. academic and professional colleagues, and perhaps by drawing upon student feedback).
HKSYU OBTL Handbook
Programme and Course Intended Learning Outcomes Programme Intended Learning Outcomes (PILOs)
Programme outcomes are in the nature of a charter for the programme providing information which is of immense value to students, potential students, accrediting and professional bodies and to the instructors. HKCAAVQ for example, would expect universities to show that they have thought through the learning outcomes they expect their students to achieve and that they are organising educational experiences to enable students to achieve these outcomes. Universities must demonstrate that undergraduate degree programme outcomes are aligned with level 5 of the Generic Level Descriptors and Graduate Attributes and that postgraduate programmes are at QF level 6. The following features of a well-designed and well-delivered higher education programme put learners and learning centre-stage: Outcomes based Benchmarked to graduate skills and attributes Aligned to HKQF generic level descriptors
It is necessary when writing PILOs and CILOs to distinguish between aims and outcomes. In this Handbook, Aims are: General, teacher-centered statements of what the course/programme is about; and Statements that provide succinct information on the overall agenda for the course/programme
Example: Subject Cognitive Psychology

This course aims to introduce to students the information processing approach in the study of human cognition and its central concepts. By learning the methods and skills in investigating cognitive process throughout the course, students will also gain firsthand experience by conducting their own experimental research. Students will also learn how to present their own studies in formal academic format in the form of presentations and reports.
Outcomes are: Specific, student-centred statements indicating what the student is expected to do to demonstrate evidence of learning; Statements that facilitate observation and measurement of students performance; Statements that establish the means by which the agenda is achieved.
HKSYU OBTL Handbook Example: Subject Cognitive Psychology After successful completion of this course, students should be able to:
1. Explain the human mind from a cognitive approach; 2. Identify the strengths and limitations of attention theories and models in explaining cognitive functioning; 3. Explain the basis of artificial intelligence; 4. Contrast the differences between conscious and unconscious processing; 5. Conduct at least one type of cognitive psychology study; Etc. etc.
PILOs should align with institutional goals, Graduate Attributes, and the relevant Generic Level Descriptors of the Qualifications Framework. Compiling the programme outcomes should be a collaborative effort of the programme team. A common understanding of what the programme is trying to achieve is important for the development of appropriate teaching and assessment strategies. Once you have devised your PILOs, you need to make sure that their attainment is clearly achievable through the outcomes of the different courses in the programme. You may find it useful to break down course outcomes of different courses in the programme over the different levels (e.g. 1st year, 2nd year, etc.) so that you can verify that students are progressively working towards the programme outcomes year by year or level by level. If you have any longer-term outcomes on a programme and feel a student may only be able to demonstrate them after completion of the entire programme, it may be advisable to state them as programme or course aims rather than as course or programme outcomes.
Course Intended Learning Outcomes (CILOs)

Course level learning outcomes should8: Cascade down from PILOs; Narrow down the essential knowledge content and the abilities and skills practised; Describe what the student should be able to demonstrate after completing the course; Be formulated as requirements for passing the course and should, therefore, be assessable; Be observable, relevant, and achievable; and Provide a foundation for evaluation, assessment, and grading criteria.
Writing Intended Learning Outcomes A guide: writing intended learning outcomes at module level: NSHU, the Swedish Agency for Networks and Cooperation in Higher Education: http://kursutveckling.se/dok/Larandemal_eng_061011.pdf
8
HKSYU OBTL Handbook A clear learning outcome has: Action verbs that express what the student is expected to be able to do (observable actions) at the end of the course Words or phrases that describe the material, the area, subject, etc. that the student works with or is the result of the educational experience If desired, words or sentences that describe how the knowledge should be used (individually, in summary, in detail, with the help of, orally, written, etc.)
With reference to the HKQF (www.hkqf.gov.hk) Generic Level Descriptors (GLDs), start to draft your ILOs by first answering these questions: What are the aims and learning outcomes of my course, drafted in terms of what students need to do and know/understand, rather than in terms of content to be covered? (e.g. do what exactly? know what exactly, and how should they know it?) How do I want my students to be different by the time they have completed my course? (e.g. As a result of successfully completing this course, what sort of perspective and skills should students have relating to the discipline, the world, themselves and what should they appreciate/value?) Which of the Universitys graduate attributes do I think my course might best contribute towards? What would these graduate attributes look like in the context of my course content?
Writing your ILOs9:

Begin with the statement, On successful completion of this course, students should be able to . Begin with an action verb and describe something substantive relating to the course content (knowledge, skill, attitude, attribute) that is observable and measurable. Preferably, use one action verb for each learning outcome. Focus on what you expect students to be able to demonstrate upon completion of the course. Learning outcomes should be written with assessment in mind ILOs must be concrete, observable and measurable and focus on demonstration / performance of the stated intended learning outcomes. Use clear, short sentences. Once you are satisfied with your ILO statements, it is often helpful to ask a colleague to review them. Two sets of verbs that can be used in defining learning outcomes are given in the Biggs SOLO taxonomy and Blooms taxonomy, together with some useful URLs, in Appendix D.
http://www.dcu.ie/afi/docs/FINAL_GUIDE_LOs-1%20May%2019th.pdf
10
HKSYU OBTL Handbook
Tips to Address Common Pitfalls10

Pitfalls
Jargon and Slang
Explanations/Examples
http://www.buzzle.com/articles/jar gon-examples.html
Tips to Address Pitfalls

Use simple, user-friendly language and terms that can be understood by students, colleagues and external bodies. Write broad learning outcomes for an entire course or a programme. Use prescriptive ILOs to describe actions of a student that may be achievable at the end of a specific learning activity.
Vague or Overly Specific Language
ILOs are too vague or too specific for the course level: e.g. Too broad: Students will be able to identify and demonstrate the dynamic nature of the environment in which marketing decisions are taken Too specific: Students will be able to outline the functions of marketing within a financial institution Terms like know, understand, be familiar with, be exposed to, be aware of, appreciate, learn, be acquainted with, excellent, good, satisfactory, etc. may be interpreted differently by students,instructors, employers: e.g. How might different stakeholders demonstrate or measure an outcome such as, Students will be able to understand the function, structure and components of the musculoskeletal system?
Ambiguous Words and Phrases
Focus on what the student can actually demonstrate: e.g. Students will be able to explain the function, structure and components of the musculoskeletal system. Carefully choose verbs that will rate the ILOs11.
Too Many ILOs
A large number of ILOs in a course can cause both students and instructors to feel undue pressure and confuses students about those key ILOs on which to focus. The presence of too many outcomes indicates that the outcomes may be too detailed or redundant.
You may wish to consider whether some ILOs might be combined. You might help students prioritise the ILOs by naming those specific to various learning activities. If you observe that a particular ILO is more specific to a single lecture rather than the whole course, you may wish to remove it. Use your assessment strategies and tasks to prompt you as to which ILOs are most important.
10 11
http://www.dcu.ie/afi/docs/FINAL_GUIDE_LOs-1%20May%2019th.pdf Page 11, http://www.dcu.ie/afi/docs/FINAL_GUIDE_LOs-1%20May%2019th.pdf
11
HKSYU OBTL Handbook Pitfalls

Too Many Verbs in One ILO
Too many verbs in one ILO may confuse students as to which ILO is most important for them to demonstrate for assessment: e.g. Students will work in small groups and consider how basic principles are applied to industrial processes. Where two verbs are codependent and relevant to one ILO, the required actions may be clear: e.g. Students will be able identify and solve problems relating to the basic problems of anorexia. It can be tempting to use the same verb for finding alternatives to terms such as understand, know, be familiar with, etc. In some subject disciplines, however, there may be a need to repeat terms where there is no alternative required or possible: e.g. in maths solve, or calculate.

Write concise ILO statements. Use simple sentence structure, rather than complex, for ILO statements. Use required assessment tasks to guide you.
Overuse of the Same Verb
Avoid a cut and paste remedy. Use a verbs list mapped to learning domains such as the SOLO Taxonomy or Blooms Taxonomy (Appendix D) to choose appropriate verbs that will differentiate achievement levels for various demonstrations of learning and assessment tasks: e.g., verbs describe the degree of complexity and depth of intellectual/practical processes. The verb to understand is not a good choice because it is vague and not observable. As an example, the verb to understand can be replaced with a verb that describes student performance describe in your own words, exemplify, illustrate, classify, etc. Avoid references to progression when writing ILOs: e.g. Students will demonstrate proficient presentation skills. In this example, a rubric that lists look-fors and describes proficiency levels for the ILO would assist both students and instructors. Instructors should set realistic expectations and keep in mind the assessment tasks and workload for both themselves and their students when setting ILOs. Consider how to adjust various ILOs and/or learning activities to optimise resources.
Use of ILOs that refer to improvement or Progression imply progression may be difficult in ILOs to measure without pretests/post-tests at different stages of learning: e.g. Students will improve their presentation skills.
Impractical ILOs
Constraints such as time and resources may undermine the credibility and practicality of the ILO.
12
HKSYU OBTL Handbook Pitfalls

Inability or Failure to Assess ILOs
Some ILOs are not, or cannot be, assessed.

Check that each ILO is/can be assessed. Check that the assessment tasks cover all ILOs. Avoid ILOs that are more teacher intention oriented than student performance oriented. Examine assessment tasks and grading instruments to ensure coverage and alignment of subject content and process (generic skills) outcomes as stated in ILOs. Write clear and observable ILOs.
13
HKSYU OBTL Handbook Some good and bad examples: After completing the course, the student should be able to Ex 1. Write a report describing the relationship between chemical structures and material properties (the focus is on the structure of knowledge, principles, and connections). Ex 2. With the help of reference material, select an appropriate construction material and justify the choice, taking into account its function, economics, and environment (the focus is on the practical application of knowledge). Ex 3: Understand how laws and guidelines for social planning are applied. Read academic texts about the scholarship of teaching (learning outcomes are not observable or measurable). Ex 4: Explain the relationship between applied legislation and the process of urban planning. Review relevant scientific texts about the scholarship of teaching (rewritten learning outcomes that are observable and measurable). Ex. 5: Have visited at least three community organisations and have observed one counselling session with a pre-defined aim in mind. (Question: Why? The ILO needs to make a distinction between learning activities and results. The learning outcomes should not describe how the outcome is achieved, but detail the results of the various learning activities that occur during the course). Ex. 6: Categorise and analyse observed counselling activities to achieve a pre-determined aim, while at the same time drawing conclusions about her or his own personal reactions as a counselor (describes the result of the learning activity). Ex. 7: Carry out a company audit (vague). Ex. 8: Carry out a company audit from the perspective of an economic theory, justifying and explaining her or his choice of method (ILO specifies content and how knowledge will be used).
14
HKSYU OBTL Handbook
FAQs Review
Q: What is the difference between course intended learning outcomes (CILOs) and programme intended learning outcomes (PILOs)?12 A: PILOs are broad statements and cover an entire range of courses comprising the programme; CILOs are more specific in describing what the student will be able to do; CILOs determine the content, delivery and assessment of each course, which, when combined with all other courses, meet the PILOs.
Q: What is the difference between course intended learning outcomes (CILOs) and course aims?13 A: CILOs are statements of what the student will be able to do or demonstrate as a result of her/his learning (e.g. Students will be able to apply the principles of xxx in a literature review.). Course aims are statements of what the instructor intends for the students, and are written from the teachers perspective (e.g. Students will be taught the basic principles of xxx.).
Want more information?
http://www.learningandteaching.info/teaching/objectives.htm http://www.ces-vol.org.uk/index.cfm?pg=123
12 13
http://www.dcu.ie/afi/docs/FINAL_GUIDE_LOs-1%20May%2019th.pdf http://www.dcu.ie/afi/docs/FINAL_GUIDE_LOs-1%20May%2019th.pdf
15
HKSYU OBTL Handbook
Mapping Intended Learning Outcomes (ILOs)

Curriculum mapping is a system that thematically aligns assessment, curriculum and instruction. Mapping ILOs within the curriculum is done to ensure coherence in courses, aligning: PILOs with Graduate Attributes, University Mission, and Generic Level Descriptors Course Intended Learning Outcomes with Programme Intended Learning Outcomes Teaching and learning activities with CILOs to show that the activities do in fact facilitate the achievement of the outcomes Assessment tasks with CILOs to ensure that there is no disconnect between outcomes and assessment
Curriculum mapping helps staff to identify gaps, repetition and overlaps of ILOs within courses and across programmes, ensures progressive alignment and variety of assessment tasks, and enables tracking of content and methodology across course or programme levels. Eventually, curriculum review, design and implementation should improve as faculty communicate and collaborate to map and monitor ILOs within their courses. The template suggested by HKCAAVQ for mapping of CILOs to PILOs and CILOs to GLDs of the Qualifications Framework as a part of the documentation for programme validation is given in Appendix E. For other examples of curriculum mapping, please refer to the Teachers Resource Bank at: http://www.hksyu.edu/resourcebank/p1.htm
The importance of alignment is apparent from Figure 1 below, which also indicates clearly the levels at which learning outcomes may be pitched. Note that the author has used the words Curriculum Objectives to mean curriculum outcomes.
16
HKSYU OBTL Handbook Figure 1
Curriculum Objectives Expressed as verbs; students must enact A The very best understanding that could be reasonably expected. Verbs include hypothesize, apply to other domains, generate, relate to principle, etc. B Highly satisfactory understanding. Verbs include explain, solve, understand main ideas, analyse, compare, etc.
Teaching and Learning Activities Designed to elicit desired verbs
Assessment Tasks Evaluate how well the target verbs are deployed in context.
The activities may be: teacher controlled, peer controlled, or self controlled, as best suits context.
The highest level verb that is clearly manifested indicates the highest grade, A, B, C, etc.
C Quite satisfactory learning, with understanding at a declarative level. Verbs include elaborate, classify, cover topics, etc.
D Understanding at a passing level. Low-level verbs and also inadequate by salvageable attempts at meeting the higherlevel verbs.
Source: John Biggs (1999) Assessment: An integral part of the teaching system. AAHE Bulletin 51, 9 (May), pp.10-12.
17
HKSYU OBTL Handbook
Chapter 3 Teaching and Learning Issues

Tips on Making Outcomes Based Teaching and Learning (OBTL) Work in the Classroom
1. First, determine what you want your students to learn. 2. Learning outcomes must be clearly established and articulated to all participants. Students must be able to see the roadmap clearly and what is expected at the end of it. 3. Explain why an OBTL approach will be used and how it will work in the class. 4. Choose learning activities that are appropriate for the ILOs. Ask yourself What practicable learning and teaching activities can I use to get students to do what the intended learning outcomes nominate?" 5. Choose relevant and meaningful teaching and learning activities that require students to apply, invent, generate new ideas, diagnose and solve problems, and whatever other things they are expected to be able to do after they graduate. Provide a context that requires the same action by the student that is already contained in the ILO verb(s). 6. One size does not fit all. Think about which ones will work for your class each class is different. 7. Create a comfortable learning environment where learning is participative and enjoyable. 8. Have an assessment strategy in place right at the start. Explain what students have to do to demonstrate what they have learned and how well they have learned it. 9. Provide the necessary support to facilitate student learning and let students know where they can get help. 10. Clearly outline your role and the students role in achieving the ILOs. Remember, what the students do is more important than what you do! 11. Anticipate potential obstacles and make a plan for how you will overcome them. 12. Get timely feedback from your students on their learning experience and use it to inform your teaching practice.
Variety of Teaching and Learning Strategies

The effective teacher understands that making the students do the work is both educationally sound (i.e. it encourages deep learning) and realistic (i.e. it lets students know that the teacher should not be used as the constant source of information). Teaching and learning activities (TLAs) may be teacher-managed, group/peer managed, or self-directed. Each has its place, serving different ILOs14.
Choosing Learning Activities That Support ILOs

If students are to demonstrate their learning, it makes sense that they be given a level playing field in which to acquire, explore, document, and explain what they know and can do and test how well they can do it. It is important to choose TLAs that provide ample opportunities for students to achieve the ILOs.
14
Outcomes-Based Teaching and Learning (OBTL) What is it, Why is it, How do we make it work? (Preworkshop Reading), John Biggs & Catherine Tang
18
HKSYU OBTL Handbook This table15 provides some examples of ways of thinking about how TLAs can support ILOs: Learning Aims
Knowledge Elaborating, Analysing & Applying Knowledge Case studies Practicals Laboratories Demonstrations Experiments Simulations Discussion Debate Seminars and tutorials Supervision Computer mediated discussion Projects Problem-based learning Essays Work based learning Group work Student presentations Student led seminars Literature reviews Peer assessment Student mentoring Peer assisted learning Self-assessment Generating Ideas & Evidence Workshops Brainstorming Laboratories Practicals Fieldwork Discussion Experiments Supervision Personal Development Feedback Learning contracts Role play Mentoring Coaching Experiential learning Planning and Managing Own Learning
TeacherDirected Activities
Lectures Guest lectures Concept mapping Modelling through thinking aloud Electronic voting systems (evs)
StudentDirected Activities
Information searching tasks Directed private study
Enquiry-based learning Research projects Dissertations Group work Mind-mapping Problem solving Inter-disciplinary projects Service learning Student-organised events Critical Incident Reports
Reflective journals Self-assessment Peer assessment
Learning logs Projects Independent study Dissertations Work placements Portfolio development
In addition to the online resources on the HKSYU Teachers Resource Bank (e.g. active learning ideas, et. al. at http://www.hksyu.edu/resourcebank/p1.htm), some useful URLs for helping instructors choose and design a variety of appropriate TLAs that support ILOs are given in Appendix F.
15
Teaching and learning methods (adapted from Bourner, 1997 and Biggs, 2003)
19
HKSYU OBTL Handbook
Mapping ILOs to Teaching and Learning Activities (TLAs)

Your teaching needs to be geared to the kinds of learning that you want to engender in students. If, for example, you want your students to develop critical thinking skills, you need to model a questioning approach and provide students with explicit opportunities to practice these skills. If you want students to be able to write using the language of the discipline, your teaching approach should incorporate occasions when students can practise doing this. If application to a scenario is a goal, this approach needs to be consistently integrated into your teaching16. One way of ensuring that your teaching and learning activities reinforce ILOs is to map TLAs to ILOs. The ILOs provide the scaffolding for your course delivery, and a simple table mapping the TLAs to the ILOs and Assessment tasks (ATs) to ILOs can help to identify how the ILOs are addressed as you progress with the course content. Figure 1 on page 17 also indicates the importance of the connectivity between these various components of OBTL to deliver a student-centred study programme. Example Aligning TLAs and ATs with ILOs ILO Be able to make valid comparisons between European, Asian and North American media practices across different historical periods on the basis of sound historical evidence. ATs
A debate paper in which two student teams must: take a position and argue their case to their seminar colleagues conduct a vote, and submit their argument in writing, noting the outcome of the vote submit a reflective paper on what they learned from the debate
TLAs
A series of lectures covering the topic/period A discussion seminar using primary sources and a general secondary text to provide an overview Recommended readings Group work
Sharing your plan with the students is an effective way of creating transparency. They can see what they are meant to be achieving, monitor their own progress towards achieving the ILOs, and be proactive in their learning activities. They can begin to appreciate the why behind the TLAs and ATs, helping them to connect the dots to get a sense of the underlying coherence of a course and how the different elements contribute to the whole. It is also a good strategy for reminding students of certain core questions, themes and concepts that the course is addressing17.
16
Introduction to Course Design, Teaching Development / Wahanga Whakapakari Ako, Dorothy Spiller, February 2011 17 Guidelines for Programme/Course Design and Review (media_178827_en.pdf), Learning and Teaching Centre, University of Glasgow, Revised November 2010
20
HKSYU OBTL Handbook
Importance of Feedback
Feedback is the most important source of information for multiple purposes, as noted below: Feedback on students work: End-of-semester examinations (summative assessment) do not provide any opportunity for instructors to give feedback, which would be of help to students to improve on their performance in a course they have already completed. The period leading up to the examinations in a course that includes such an examination, however, is a significant opportunity for instructors to develop formative assessment tasks and provide timely and meaningful feedback to students on their work. This practice will help students to reflect on their learning and enhance their self-confidence for future tasks. With increased weight for coursework or continuous assessment as opposed to end-of-semester examinations, it is necessary to provide feedback to students to reinforce and enhance learning and to help them develop themselves personally and professionally. Providing feedback to students in a large class is an onerous task. It would be useful to address this difficulty through carefully planned assessment methods, including self and peer assessment. Feedback from students: Although there are many ways to examine the effects of your teaching and learning approaches in helping students achieve the ILOs, student feedback offers a unique perspective on their learning experience and expectations. Rather than wait for the end-of-course teaching evaluation survey results, it is important for instructors to consider collecting feedback at regular intervals so as to adjust teaching and learning activities and make them work for your classes. That is, student feedback helps in enhancing the quality of teaching and learning through changes in course delivery on the basis of comments received. As a follow up action, it is good practice to let students know, in broad terms, what action has been taken on the feedback they provided. Students are generally unhappy if information on follow up action is not given and are less motivated in providing meaningful feedback on subsequent occasions. Other sources of feedback: Other important sources of feedback are peers and colleagues in the department, alumni, external examiners, academic advisory board of the department, employers, and professional bodies for professional programmes. Each of these sources provides a useful perspective, which helps in the process of continuous improvement of the quality of the courses and programmes, thereby enhancing the quality of student learning. Collecting and using information from these sources is both important and useful.
For some ideas and resources on getting and using feedback, you may wish to refer to Dr. Cecilia Chans presentation at the OBTL Experience Sharing Session held in the University in September 2012 at http://www.hksyu.edu/qa/ and the online HKSYU Teachers Resource Bank at http://www.hksyu.edu/resourcebank/p1.htm. And what all good instructors should be doing giving formative feedback to students: http://www.brookes.ac.uk/services/ocsld/firstwords/fw21.html
21
HKSYU OBTL Handbook
Chapter 4 Assessment Issues

How Does Assessment Work in an OBTL Environment?
The Universitys Academic Development Plans identify three primary aims of assessment of students learning: first, to ascertain the extent to which the students have achieved the learning outcomes of each course as detailed in the course outline; second, to safeguard the standard of the final award; and third, and most important, in the culture of OBTL, to promote student learning. In this perspective, assessment tasks provide continuous opportunities throughout each course for students to practice the skills and apply the knowledge they have gained and measure their achievement of the intended learning outcomes. Assessment results also provide valuable feedback to instructors concerning the effectiveness of the teaching strategies they have employed to deliver the intended learning outcomes. It is important to recognise that assessment performs two functions support for learning and accreditation of learning. Assessment for learning uses assessment tasks as a means of providing students with formative feedback and engaging students in activities that are in themselves learning activities. Assessment of learning uses assessment tasks to judge and award credit for the level of achievement a student attains through a course or programme18. From the students point of view, assessment drives learning. This is why it is so important to get the assessment strategy right and to make sure it is aligned with the CILOs19. Continuous assessment relating to coursework can be a powerful tool to motivate and enhance learning. It alleviates examination anxiety and tension for students and, if carefully planned, helps them to see, and take control of, their own progress from one level to another. In an OBTL Experience Sharing Session held in SYU in September 2012, Dr. Karen Lee of the Department of Law and Business suggested that one issue deserves attention: that is whether or not the content of our course assessments lives up to the promise of OBTL, that what students learn in the course has a bearing on the practical problems they will face and need to tackle in the real world? Dr. Lee opined that examples of current cases in the law courts in Hong Kong will give meaning and relevance to classroom teaching and arouse student interest. It makes sense that an answer to this question may be found in authentic assessment tasks. According to Fraser, W.J. (1999:16), authentic, performance-based assessment:
18
Uses assessment tasks that resemble skills, activities and functions in the real world and in class. Aims at determining competences in contexts that closely resemble situations in which
Guidelines for Programme/Course Design and Review (media_178827_en.pdf), Learning and Teaching Centre, University of Glasgow, Revised November 2010 19 Guidelines for Programme/Course Design and Review (media_178827_en.pdf), Learning and Teaching Centre, University of Glasgow, Revised November 2010
22
HKSYU OBTL Handbook these competences are required. Provides a learning experience in which learners are prepared to apply their knowledge, skills and values in an integrated manner.
Characteristics of Sound Assessment20 Clarity of purpose Enables the student to review progress and plan further learning strategies Clarity on what is being assessed and how judgments are reached Assessment will essentially assess what it claims to assess Credible to teachers, students and the institution Cost-efficient Provides a clear record of attainment, which is useful to third parties The assessment system itself will be subject to quality assurance procedures
Characteristics of an Outcomes Based Approach to Assessment Takes a holistic approach in describing the competence of a learner in terms of knowledge, skills and values Assesses competence by using a variety of assessment tasks Calls for performance-based and authentic assessment strategies Highlights continuous and criterion-referenced assessment Use of rubrics
Common Weaknesses in Assessment21 The tasks do not match the stated outcomes The criteria do not match the tasks or outcomes The criteria are not known to students Students do not understand the criteria Overuse of one mode of assessment: such as, written examinations, essays or closed problems Overload of students or staff Insufficient time for students to do the assignments Too many assignments with the same deadline (creating a bottleneck) Insufficient time for staff to mark the assignments or examinations Absence of well-defined criteria, so consistency is difficult to achieve Unduly specific criteria, which creates a straitjacket for students and makes marking burdensome for lecturers Inadequate or superficial feedback provided to students Wide variations in marking between modules and assessors and within assessors (selfconsistency) Variations in assessment demands of different modules
By implication, more effective assessment systems are relatively free of these weaknesses. Are yours?
20 21
Based on the UK Employment Department paper (1992) cited in Brown and Knight, op cit, p22) Brown, G, Assessment: A Guide for Lecturers, LTSN Generic Centre Assessment Series 2001, p.7
23
HKSYU OBTL Handbook
Developing an Assessment Plan

It is important to design an assessment plan and assessment tasks that elicit the intended learning outcomes. Herman, Aschbacher, and Winters (1992) suggest ten steps as part of the assessment design process: http://www.ncrel.org/sdrs/areas/issues/methods/assment/as7sele2.htm When designing an assessment plan, it is important to: Scrutinise the timing and number of assessment tasks in a course so as to ensure that the workload for students and staff is realistic. Integrate assessment across the programme/course rather than look at assessment as an add-on to be undertaken at the end. Build in progressive complexity and demands, beginning with low-stake small-scale tasks that provide timely and pertinent feedback, across the course of study, in 3 assessment phases: diagnostic, formative and summative. Plan to use a variety of assessment methods in order to minimise the disadvantages of each, and in order to provide individual students (who will have different strengths and learning styles) with a range of opportunities to demonstrate their achievement. Develop an assessment strategy and criteria to assess skills (professional and generic) and attributes as well as knowledge and understanding and higher order learning (such as analysis, synthesis and evaluation) rather than recall, taking into account both content and the process of achieving ILOs. It is good practice to provide and discuss the assessment criteria with students before an assessment task is assigned. Include frequent opportunities for feedback. Build in opportunities for self and peer assessment.
Assessment Tasks
A university education goes beyond mastering factual knowledge. It focuses on higher order thinking skills and real world competencies. If we want to measure students ability to think critically and creatively and solve problems, we need to use assessment methods that engage our students in thinking and doing things beyond the classroom walls. Assessment methods that focus on lower cognitive skills like memorisation / recall will not do the job. Relying on just one or two measures is bound to produce an incomplete picture of what students are accomplishing. Departments should consider various options and then select the strategies that will capture students achievements and distinctiveness. The Assessment Tool Checklist in Appendix G provides useful guidance in considering the design of assessment tasks. In an OBTL environment, it is important to remember that as you write and refine the learning outcomes, you need to be thinking simultaneously about the implications for
24
HKSYU OBTL Handbook assessment tasks and for your teaching content and strategies. To align outcomes and assessment, the following general principles need to be kept in mind: Design assessment tasks that promote learning and good academic practice. Be sure that the assessment task genuinely corresponds to the learning outcome (e.g. grade for learning/performance, not hours). Make sure that certain groups of students are not disadvantaged through the design or content of an assessment task. Provide for student choice of assessment tasks. Make sure that feedback is timely, sensitive and constructive and indicates what was good and why, what was wrong/inappropriate and why, and how the work could be improved. Do not rely on grades for feedback. Motivate students by recognising and rewarding progress and attainment rather than focusing on failure, giving students chances to benchmark and practise the learning required (e.g. shifts in attitudes). Use teaching approaches that encourage development of those professional and generic skills required for assessment (e.g. assessment tasks that elicit critical thinking and application of learning in new contexts). For example, assessment tasks for a given ILO are aligned to that ILO by presenting the student with a task that requires them to use the operative ILO verb. In order to perform the assessment task, the students have to enact that verb or a closely related one. The best form of alignment is where the teaching/learning activity (TLA) is itself the assessment, as in problem based learning (PBL). The students, or usually groups of students, are presented with a problem to solve, and with hints and resources to work out how to do it (the TLA); how well they do it is the assessment task. The summative assessment task may be the same kind of problem used in the TLA, or in final years, a different kind of problem, to test the students ability to generalise. Not all ILOs need to be assessed where they are enabling higher order ILOs or are subsumed by them. For example, if a student has been able to meet an ILO containing explain, there is no need to assess describe or identify, unless there is another good reason to do so.22 One assessment task can incorporate a number of learning outcomes. Use learning outcomes as a baseline guide for developing your assessment criteria. Minimise the possibility of plagiarism through assessment task design. Theory or practice? Subject knowledge or application? Skills? Cognitive learning (intellectual skills) Affective learning (behavioural skills that indicate attitudes, generic skills) Psychomotor learning (physical skills) What we have always assessed? What is easy to assess?
Some questions that require consideration when choosing what we assess:

22
Outcomes-Based Teaching and Learning (OBTL) What is it, Why is it, How do we make it work? (Preworkshop Reading), John Biggs & Catherine Tang
25
HKSYU OBTL Handbook It is important to select appropriate assessment task formats with a purpose (http://www.ncrel.org/sdrs/areas/issues/methods/assment/as7purp.htm) to assess the desired behaviour targeted in the ILOs ones that23: Assess specific instructional elements for observation or performance in proportion to their emphasis in instruction or training Represent the content knowledge and skills expected to be attained by students Enable students to demonstrate their proficiencies and capabilities Allow assessment of multiple goals Reflect an authentic, real-world context (i.e. the best assessment tasks reflect real life by being authentic to the profession or discipline, but non-invigilated or off-campus assessment may raise plagiarism fears. An answer to that is that the more the task requires personal involvement by the student e.g., a case study during placement, a reflective journal the less likely the student is to get it off the Net or from someone else.)24 Allow an interdisciplinary approach
In Appendix, H, you will find the Top 10 Tips on Assessment for Learning. You may wish to refer to additional resources for ideas on designing assessment tasks to achieve learning outcomes in Appendix I. After you have reviewed several examples of assessment strategies and tools, you should adapt the ones that fit your own situation. Every class is different and a cut and paste job doesnt always work. Its also important to remember that one size doesnt fit all, and you may discover that what works with one of your classes may not work with another. With some feedback from your students, and a bit of trial and error, you will find those strategies that work best for your classes. Whatever your assessment plan, make sure that you prepare your students for assessment tasks that means no surprises! Communicate often (orally and in writing), clearly (make sure your students are familiar with the verbs used in the ILOs), and strategically (do your students know the rules of the game?). Some samples of assessment tasks that may be used are in Appendix J.
Marking and Grading Issues

The primary purpose of grading students work is to give them feedback on their learning progress and the quality of their work. In an outcomes based approach, this means grading should help students to understand where they are individually in terms of the assessment criteria (criterion referenced) rather than where they stand in comparison to other students (norm-referenced). An assessment rubric can be a powerful tool to motivate students and to enable them to take responsibility for their own learning. In the criterion-referenced environment, students
23 24
Based on Joan Herman, Pam Aschbacker, and Lynn Winters (1992) Outcomes-Based Teaching and Learning (OBTL) What is it, Why is it, How do we make it work? (Preworkshop Reading), John Biggs & Catherine Tang
26
HKSYU OBTL Handbook are not competing with each other to achieve a higher grade, as is the case in normreferenced assessment. Clearly communicating what standard and quality of performance a student needs to demonstrate relating to a learning outcome can motivate students, compel them to take responsibility for their own learning, and help promote collaborative learning. While your ILOs have established the requirement to pass a course and can be linked to the minimum standard to fulfil a particular ILO assessment criterion, grading criteria explain what a student must do to earn a higher grade. How do you make decisions in the assessment plan about the percentage weightings to be assigned to assessment tasks and establish grades? Weightings should be considered on the basis of whether they are appropriate for fairly recognising/rewarding and ranking the importance of the ILOs. Judging How Well the Assessment Task Has Been Performed Marking Versus Grading The University regulations prescribe a scheme for marking and grading of students performance. Instructors need to develop assessment criteria that are compatible with the regulations. Some instructors prefer to assign marks on a percentage scale and convert them to a grade while others may determine a grade first on a holistic basis and then assign marks within the range of marks related to the grade. Instructors need to decide the approach they wish to adopt in assessing student work with the help of assessment criteria that have been communicated to the students previously.
Developing Rubrics
Rubrics show assessment criteria to help determine the quality and level of expected student performance in achieving the ILOs. Rubrics are no longer an option. They are an essential component of an assessment process. A rubric contains: A list of the things you are looking for in a students work. Guidelines for evaluating those things. A simple list, chart, or guide that lists the criteria and describes the standards that you are looking for to score or grade students performance in achieving the intended learning outcomes (ILOs)
27
HKSYU OBTL Handbook
In a recent survey on the impact of assessment rubrics in SYU of academic staff in the Department of Counselling and Psychology, Dr. Mark Greene reported his findings at the OBTL Experience Sharing Session held in September 2012, under three headings, as follows: Q.1 How has the implementation of Assessment Rubrics in your courses benefited you, the instructor? Rubrics improve student-instructor interaction. Save time rubrics serve as point of reference for grading standards and dispute resolution. Are clear, easy to understand, and standardised. Provide clear and concrete criteria to assess students performance. Help maintain a fair grading system. Q.2 How has the implementation of Assessment Rubrics in your courses benefited/impacted your students? Rubrics reduce student anxiety towards assessment processes. Increase students self-confidence and motivation to get better results. Provide clear indication of strengths and weaknesses on marked work. Accurately anticipate many performance-related questions that students will have. Provide transparency by democratising communication since often only certain students actively seek out nuanced instructions. Streamline the communication process. Provide unified language for discussing performance and expectations. Motivate students to study better with clearer sense of the requirements. Q.3 How do you feel the implementation of Assessment Rubrics in all COUN and PSY courses is an example of best practices? Rubrics create consistency with 1 course, various instructors. Articulating and specifying objectives improves achievement. Rubrics form a record of standards useful in developing new courses. Enhance instructor accountability. Facilitate communication and resource sharing among colleagues. Most western universities, including the Ivy League, use assessment rubrics.
A well-designed rubric will provide a set of statements to help assessors (the instructor, students, and community professionals) differentiate the students performance level fairly. The idea behind this is that, rather than focusing on the threshold level stated in the ILOs, students can see the criteria for different levels of performance and what it is that they need to do to earn Grade A, B, C, First, or a Distinction, and will shift their focus to the highest performance level. For example: ILO: On successful completion of the course, students should be able to use evidence appropriately in support of an argument. Q: What more is needed for assessing this ILO?
28
HKSYU OBTL Handbook
Designing Rubrics Huba & Freed, 2000

Question What criteria or essential elements must be present in the students work to ensure high quality? How many levels of achievement do I want to illustrate? For each criterion, what is a clear description of performance at each achievement level? What are the consequences of performing at each level? What rating scheme will I use? Action Include these as rows in rubric.
Include these as columns and label them.
Include descriptions in the appropriate cells of the rubric. Add description of consequences to commentaries in the rubric. Add this to the rubric in a way that fits with your grading philosophy.
When I use the rubric, what aspects work well and what aspects need improvement? Is process as important as outcome?
Revise rubric accordingly. Include and describe criteria to reflect aspects of the process.
Suggested Rating Scales Relating to Rubrics:

Rubrics with numbers or letters (recommend 4-point scale) Verbal descriptions distributed along a straight line Checklists Select one that most closely matches? Verbally anchored scales, beginning with highest rating/most positive statement: Strongly Agree, Agree, Uncertain, Disagree, Strongly Disagree Very Often, Frequently, Sometimes, Rarely Can do this well enough to teach others, Can do this well, Can do some of this, Cant do this yet
Some Exemplars
Exemplars are offered as ideas for rubric creation. Rubrics should always be designed with specific ILOs in mind that are suited specifically to your own courses. They should also be collectively developed within the Department so that all academic staff members are familiar with their content and intended use. Please refer to the Appendix K (Rubric Packet) for a collection of rubric examples used in a variety of subject disciplines in Shue Yan University and other universities. Student-Created Rubrics: http://pblmm.k12.ca.us/PBLGuide/Activities/DesignRubric.html
29
HKSYU OBTL Handbook
How to Determine if Your Assessment Plan is Working

Ultimately, your students achievements provide evidence of the success of your programme. Once you have your students assessment and feedback results, analyse this data in a meaningful way to find out whether the outcomes are being achieved. Prepare a clear report of the findings to share with all of the members of your course team and Department. Decide together what the findings mean. If all aspects of the learning outcomes were satisfactorily achieved, then arrange to reassess the outcomes at a later time to monitor whether they are still being satisfactorily achieved. If all aspects of the learning outcomes were not satisfactorily achieved, then propose to make changes designed to improve student learning in your programme (e.g., to the curriculum, pedagogy, assignments). Then reassess the outcomes at a later time to determine if improvement in student learning occurred 25 . It would be important to discuss how you will use the information to improve your programme.
If you are uncertain whether your assessment plan is working, you might try using the Creating an Assessment Plan Flowchart as a guide, which is available from: http://www.lmu.edu/about/services/academicplanning/assessment/Assessment_Resources/ Creating_an_Assessment_Plan.htm
25
Creating an Assessment Plan, Loyola Marymount University, http://www.lmu.edu/about/services/academicplanning/assessment/Assessment_Resources/Creating_an_Ass essment_Plan.htm
30
HKSYU OBTL Handbook
Glossary
ATs CILOs GAs GLDs Assessment Tasks Course Intended learning Outcomes Graduate Attributes Generic Level Descriptors of the Qualifications Framework of the HKCAAVQ describe the requirements of each level of a programme in four aspects - "Knowledge and Intellectual Skills", "Processes", "Application, Autonomy and Accountability" and "Communications, IT and Numeracy". Two levels of descriptors are of immediate relevance, Level 5 for an undergraduate degree and Level 6 for a postgraduate degree. Hong Kong Council for the Accreditation of Academic and Vocational Qualifications Intended Learning Outcomes Outcomes Based Teaching and Learning Programme Intended Learning Outcomes Problem Based Learning The Quality Assurance Committee of HKSYU is a committee appointed by the Academic Board. It works on its behalf to ensure the academic quality and rigour of programmes and that there are policies and procedures to support this effort. It advises and makes recommendations to the Board as appropriate. http://www.hksyu.edu/qa/QAC.htm Qualifications Framework as described by the HKCAAVQ Assessment Criteria The Teachers Resource Bank contains a number of resources to facilitate teaching staff to implement outcomes based teaching and learning: www.hksyu.edu/resourcebank/p1.htm Teaching and Learning Activities The University Grants Committee of Hong Kong is a nonstatutory advisory committee responsible for advising the Government of the Special Administrative Region (SAR) of the People's Republic of China on the development and funding needs of higher education institutions in the HKSAR.
HKCAAVQ ILOs OBTL PILOs PBL QAC
QF Rubrics Teachers Resource Bank TLAs UGC
31
HKSYU OBTL Handbook
Appendices
Page 33 34 35 37 42 43 44 46 49 50 51
Appendix A Appendix B Appendix C Appendix D Appendix E Appendix F Appendix G Appendix H Appendix I Appendix J Appendix K
SYUs Mission Statement Graduate Attributes GLDs at Level 5 and 6 SOLO and Blooms Taxonomies HKCAAVQ Template for Mapping ILOs Designing TLAs Assessment Tool Checklist Top 10 Tips on Assessment for Learning Designing ATs Some Examples of Assessment Tasks Examples of Rubrics
32
Appendix A
University Mission When Shue Yan College was founded in 1971, the first Chairman of the Board of Governors, the late Hon. Wilfred S B Wong, expressed its aims in terms of Confucian philosophy, defining the ideal education that it would provide as the cultivation of virtue ( ), which helps every student to build character and develop a harmonious way of living with others. The Confucian way as a human way is predicated on the faith that the human condition is improvable through self effort, and its core values of humanity, sympathy, reciprocity, responsibility, public-spiritedness and communality (Tu Weiming, 2003) continue to underpin the Universitys mission as a teaching-led, research informed liberal arts university in 2010 to: Preserve and disseminate traditional Chinese culture; Enable students to cultivate a whole, balanced personality including moral character and to realise their full potential as individuals, as scholars and as good citizens; Provide rigorous intellectual training to enable students to become independent critical thinkers; and Combine academic theory with the acquisition of professional and practical skills to produce graduates who can immediately be of service to meet the needs of the community.
33
Appendix B
Statement of Graduate Attributes of HKSYU Upon successful completion of four years of study, we aim to produce graduates who are: 1. Articulate, open-minded critical thinkers with a passion for lifelong learning and self-improvement; 2. Committed to appropriate ethical behaviour, based on a strong sense of social responsibility; 3. Well prepared to apply their specialist knowledge, skills and creativity in their chosen field of employment; and 4. Ready to apply their global outlook and understanding of Chinese cultural values to support the development of Hong Kong and China in the 21st century.
34
Appendix C
Generic Level Descriptors at Level 5 and Level 6
35
Appendix C
36
Appendix D
SOLO And Blooms Taxonomies of Verbs for Writing ILOs

The SOLO Taxonomy with sample verbs indicating levels of understanding
Create Formulate Generate Hypothesize Reflect Theorize
Competence
Identify Name Follow simple procedure Fail Incompetent Misses point Incompetence Prestructural one relevant aspect Unistructural
Combine Describe Enumerate Perform serial skills List
Analyze Apply Argue Compare/ contrast Criticize Explain causes Relate Justify
....
several relevant integrated into independent aspects a structure Multistructural Relational
generalized to new domain Extended Abstract
Biggs (1999, 2003) Teaching for Quality Learning at University, Open University Press
Biggs taxonomy suggests that there are 5 levels of understanding. The lowest 3 levels have been described as the quantitative and the 2 upper levels focusing on qualitative aspect of learning. The verbs given as examples at each level can be used in defining learning outcomes and they also immediately indicate their usefulness in designing assessment tasks.
37
Appendix D
Bloom's Taxonomy Revised Key Words, Model Questions, & Instructional Strategies
http://www.center.iupui.edu/ctl/idd/docs/Bloom_revised021.doc, February 8, 2006
Blooms Taxonomy (1956) has stood the test of time. Recently Anderson & Krathwohl (2001) have proposed some minor changes to include the renaming and reordering of the taxonomy. This reference reflects those recommended changes. I. REMEMBER (KNOWLEDGE)
(shallow processing: drawing out factual answers, testing recall and recognition) Verbs for Objectives choose describe define identify label list locate match memorize name omit recite recognize select state Model Questions Who? Where? Which One? What? How? What is the best one? Why? How much? When? What does It mean? Instructional Strategies Highlighting Rehearsal Memorizing Mnemonics
38
Appendix D
II. UNDERSTAND (COMPREHENSION)

(translating, interpreting and extrapolating) Verbs for Objectives defend demonstrate distinguish explain express extend give example illustrate indicate interrelate interpret infer judge match paraphrase represent restate rewrite select show summarize tell translate Model Questions State in your own words. Which are facts? What does this mean? Is this the same as ...? Give an example. Select the best definition. Condense this paragraph. What would happen if ...? State in one word ... Explain what is happening. What part doesn't fit? Explain what is meant. What expectations are there? Read the graph (table). What are they saying? This represents ... What seems to be ...? Is it valid that ...? What seems likely? Show in a graph, table. Which statements support ...? What restrictions would you add? Instructional Strategies Key examples Emphasize connections Elaborate concepts Summarize Paraphrase STUDENTS explain STUDENTS state the rule Why does this example ...? create visual representations (concept maps, outlines, flow charts organizers, analogies, pro/con grids) PRO| CON
NOTE: The faculty member can show them, but they have to do it.
Metaphors, rubrics, heuristics
III. APPLY
(Knowing when to apply; why to apply; and recognizing patterns of transfer to situations that are new, unfamiliar or have a new slant for students) Verbs for Objectives choose dramatize explain generalize judge organize paint prepare produce select show sketch solve use Model Questions Predict what would happen if Choose the best statements that apply Judge the effects What would result Tell what would happen Tell how, when, where, why Tell how much change there would be Identify the results of Instructional Strategies Modelling Cognitive apprenticeships Mindful practice NOT just a routine practice Part and whole sequencing Authentic situations Coached practice Case studies Simulations Algorithms
39
Appendix D
IV. ANALYZE (breaking down into parts, forms)

Verbs for Objectives analyze categorize classify compare differentiate distinguish identify infer point out select subdivide survey Model Questions What is the function of ...? What's fact? Opinion? What assumptions ...? What statement is relevant? What motive is there? Related to, extraneous to, not applicable. What conclusions? What does the author believe? What does the author assume? Make a distinction. State the point of view of ... What is the premise? State the point of view of ... What ideas apply? What ideas justify the conclusion? What's the relationship between? The least essential statements are What's the main idea? Theme? What inconsistencies, fallacies? What literary form is used? What persuasive technique? Implicit in the statement is ... Instructional Strategies Models of thinking Challenging assumptions Retrospective analysis Reflection through journaling Debates Discussions and other collaborating learning activities Decision-making situations
V. EVALUATE (according to some set of criteria, and state why)

Verbs for Objectives appraise judge criticize defend compare Model Questions What fallacies, consistencies, inconsistencies appear? Which is more important, moral, better, logical, valid, appropriate? Find the errors. Instructional Strategies Challenging assumptions Journaling Debates Discussions and other collaborating learning activities Decision-making situations
40
Appendix D
VI. CREATE (SYNTHESIS)

(combining elements into a pattern not clearly there before) Verbs for Objectives choose combine compose construct create design develop do formulate hypothesize invent make make up originate organize plan produce role play tell Web References: Model Questions How would you test ...? Propose an alternative. Solve the following How else would you ...? State a rule. Instructional Strategies Modeling Challenging assumptions Reflection through journaling Debates Discussions and other collaborating learning activities Design Decision-making situations
u/appm/courses/7400/1996Spr/bloom.html
s/bloom.html References: Anderson, L. W. & Krathwohl, D. R. (2001). A Taxonomy for learning, teaching, and assessing. Bloom, B. S. (Ed.). (1956). Taxonomy of educational objectives: The classification of educational goals, by a committee of college and university examiners. New York: Longmans. John Maynard, University of Texas, Austin Marilla Svinicki, University of Texas, Austin
Compiled by the IUPUI Center for Teaching and Learning, Revised December 2002
Some Useful URLs for Writing ILOs

A step-by-step guide to programme and course design and review is available at: http://www.gla.ac.uk/services/learningteaching/goodpracticeresources/designingprogra mmesandcourses/ Tips for designing ILOs for effective assessment: http://tldu.waikato.ac.nz/resources/LearningOutcomestoAssessment.pdf
41
Appendix E
HKCAAVQ Template for mapping ILOs

Consolidated summary of the contribution of courses to PILOs PILO Course
Course Code Course Code Course Code
1 2 3
Programme Intended Learning Outcomes (PILOs)

4 5 6 7 8 9 10 11 ---------
(Note: A in the above table indicates that the course contributes to that particular PILO.)
Mapping of courses against the Generic Level Descriptors (GLDs) of the QF GLDs (QF Level ___) Domain GLDs (QF Level ___)
Course
Course Code Course Code Course Code (Note:
Knowledge Application, Knowledge Application, Communication, Communication, and Autonomy and Autonomy Processes IT and Processes IT and Intellectual and Intellectual and Numeracy Numeracy Skills Accountability Skills Accountability K1 K2 ... P1 P2 A1 A2 C1 C2 K1 K2 ... P1 P2 A1 A2 C1 C2
Please give courses that benchmark at the exit QF level of the qualification and also at one QF level below the exit level. Please use separate tables if there are more than one exit qualification. A in the above table indicates that the course fulfils the requirement of that particular GLD domain.)
42
Appendix F
Resources for TLAs
General and online: http://www.merlot.org/merlot/index.htm TLAs for active learning: http://www1.umn.edu/ohr/teachlearn/tutorials/active/ Good TLAs in different subjects and disciplines: http://www.heacademy.ac.uk/subjectcentres For group work: http://bokcenter.harvard.edu/fs/html/icb.topic58474/wigintro.html
Advantages and disadvantages of different methods: http://honolulu.hawaii.edu/intranet/committees/FacDevCom/guidebk/teachtip/comte ach.htm Good classroom teaching techniques: http://www.hellofriend.org/teaching/good_classroom.html#multisen Teaching methods (scroll down the page): http://www.crlt.umich.edu/tstrategies/teachings.php
43
Appendix G
Assessment Tool Checklist http://assessment.uconn.edu/docs/Choosing_the_Right_Assessment_Tool.pdf (14.03.2012) 1. Does the assessment adequately evaluate academic performance relevant to the desired outcome? (validity) 2. Does this assessment tool enable students with different learning styles or abilities to show you what they have learned and what they can do? 3. Does the content examined by the assessment align with the content from the course? (Content validity) 4. Does this assessment method adequately address the knowledge, skills, abilities, behavior, and values associated with the intended outcome? (Domain validity) 5. Will the assessment provide information at a level appropriate to the outcome? (Blooms) 6. Will the data accurately represent what the student can do in an authentic or real life situation? (Authentic assessment) 7. Is the grading scheme consistent; would a student receive the same grade for the same work on multiple evaluations? (Reliability) 8. Can multiple people use the scoring mechanism and come up with the same general score? (Reliability) 9. Does the assessment provide data that is specific enough for the desired outcomes? (alignment with outcome) 10. Is the assessment summative or formative - if formative does it generate diagnostic feedback to improve learning? 11. Is the assessment summative or formative - if summative, is the final evaluation built upon multiple sources of data? (AAHE Good practice) 12. If this is a summative assessment, have the students had ample opportunity for formative feedback and practice displaying what they know and can do? 13. Is the assessment unbiased or value-neutral, minimizing an attempt to give desirable responses and reducing any cultural misinterpretations? 14. Are the intended uses for the assessment clear? (Grading, program review, both) 15. Have other faculty members provided feedback?
44
Appendix G
16. Has the assessment been pilot-tested? 17. Has the evaluation instrument been normed? 18. Will the information derived from the assessment help to improve teaching and learning? (AAHE Good Practice) 19. Will you provide the students with a copy of the rubric or assignment grading criteria? 20. Will you provide the students with examples of model work? (carefully consider the pros and cons of doing this.)
45
Appendix H
Top 10 Tips on Assessment for Learning1 Assessment influences what a student sees as important in their modules. Good assessment design can enhance student learning or promote a superficial approach of churning out work for the production line of module requirements. INFLUENCE OF ASSESSMENT ON STUDY 1. Provide sufficient (spaced) assessed tasks for students to be encouraged to allocate sufficient time to study over a suitable time period and avoid cramming. Tips: Students required to display work publicly Group work which encourages support for the group More frequent tasks rather than one end of module assessment (or build in steps) 2. Design the assessment so that students tackle the task appropriately: i.e. they engage in the process of learning rather than simply producing a final product. Tips: Allocate some percentage of the overall mark to draft work or justifications of decisions made while completing the assignment Allow students to show their errors and explain their corrections Value the learning involved rather than the final assignment 3. Give students the opportunity to practice the skills they need for each assessment. Tips: Explain the assessment criteria Give feedback on formative work Discuss the assessment task with students
1 Taken from Gibbs, G & Simpson, C (2004) Conditions under which assessment supports students learning, Learning and Teaching in Higher Education, Issue 1., and Brown, S, Race, P & Smith B, (1996) 500 Tips on Assessment, Kogan Page: London.
46
Appendix H
USING FEEDBACK TO ENHANCE LEARNING 4. Sufficient feedback given in enough detail. Tips: Use feedback and self-assessment sheets Consider using audio or video recordings Avoid ticks and crosses Make your writing legible you would not accept unreadable student work! 5. Feedback focuses on student performance, learning or actions the student can control Tips: Identify errors clearly Outline options for action Avoid personal comments which can reduce a students sense of competence (linked to motivation) 6. Feedback must be timely: while it matters to the student and can be used to improve future performance Tips: Discuss a model answer at the point of students submitting the work, while the ideas are fresh in their minds Use peer feedback: immediate peer feedback is preferable to late tutor feedback Computer based practice tests (e.g. multiple choice with feedback) can provide immediate feedback for student selfpaced study 7. Feedback aligns with the purpose of the assignment and the assessment criteria Tips: Align your feedback with the aim of the assignment: are you trying to influence motivation with new students or encourage reflective learning or just correct errors? Use self and peer assessment to encourage the internalisation of assessment criteria and standards
47
Appendix H
8. Feedback is appropriate to the students level Research shows that students may have one of five views of learning 2; ranging from passive receipt of information and active memorization of information (or procedures) to understanding and a change in personal reality. Feedback aimed at understandings cannot guide a student focused on memorisation. Tips: Feedback needs to understandable to the student and provide ways to Progress Provide feedback sensitive to the students understanding of the discipline involved (i.e. separate generic study skills feedback from discipline specific comments) 9. Feedback needs to be read and noticed Tips: Get students to list points they need feedback on Give feedback only (no grade) Use self-assessment prior to any tutor marking Use two-part assignments: formative feedback at part 1; grade only at part 2 Use self-assessment, tutor feedback and then supply a grade 10. Feedback is acted on by the student Tips: Follow-up the feedback and be encouraging Provide feed-forward (applies to future work) Use feedback to promote self-directed learning
2
Slj, R (1982) Learning and Understanding, Goteborg: Acta Universitatis Gothoburgensis.
Further Reading: Gibbs, G & Simpson, C (2004) Conditions under which assessment supports students learning, Learning and Teaching in Higher Education, Issue 1 Brown, S, Race, P & Smith B (1996) 500 Tips on Assessment, London: Kogan Page
48
Appendix I
Resources on Assessment
http://www.bioscience.heacademy.ac.uk/ftp/Resources/gc/assess03Lecturers.pdf http://www.waikato.ac.nz/tdu/resources/booklets.shtml Principles of Assessment http://www.waikato.ac.nz/tdu/pdf/booklets/17_AssessmentPrinciples.pdf Assessment Tasks to Promote Student Learning http://www.waikato.ac.nz/tdu/pdf/booklets/10_AssessmentTasks.pdf Assessment: Setting and Marking Assessment Tasks http://www.waikato.ac.nz/tdu/pdf/booklets/16_AssessmentSettingandMarking.pdf Assessment: Feedback to Promote Student Learning http://www.waikato.ac.nz/tdu/pdf/booklets/7_AssessmentandFeedback.pdf Assessment Matters: Academic Integrity http://www.waikato.ac.nz/tdu/pdf/booklets/8_AcademicIntegrity.pdf Assessment Matters: Self and Peer Assessment http://www.waikato.ac.nz/tdu/pdf/booklets/9_SelfPeerAssessment.pdf Assessment Matters: Groupwork Assessment http://www.waikato.ac.nz/tdu/pdf/booklets/11_GroupworkAssessment.pdf HKSYU website Assessment Resources: http://www.hksyu.edu/resourcebank/p2_04.htm Twelve tips for good assessment practices: http://www.cshe.unimelb.edu.au/assessinglearning/07/index.html Good assessment practices by subject: http://www.cshe.unimelb.edu.au/assessinglearning/04/index.html Assessing group work: http://www.brookes.ac.uk/services/ocsd/firstwords/fw26.html http://www.cshe.unimelb.edu.au/assessinglearning/03/group.html Assessment ideas from HKUST: http://celt.ust.hk/ideas/afl/Mexam/index.html London South Bank University, Learning & Teaching Enhancement Unit http://www.lsbu.ac.uk/lteu/resources/pages/ob/ub7.shtml
49
Appendix J
Some Examples of Assessment Tasks

(http://teaching.unsw.edu.au/printpdf/588)
Examples of "doing" tasks laboratory practical work demonstration of procedure professional practicum performance creative performance clinical performance Objective Structured Clinical Exam (OSCE) role play class participation (face-to-face and online).
Examples of "making" tasks an artistic composition a scale model a working prototype a website a computer software application
Examples of "speaking" tasks oral presentation (individual or group) peer teaching elevator pitch (30-second speech) Pecha Kucha (presenting 20 slides, at 20 seconds per slide) interview by student of an expert debates and discussions critical incident interview viva voce examination mooting.
Examples of extended writing tasks case study analysis essay literature review report on experiment (lab, field, project) report on evaluation of something research paper reflective journal / diary.
Examples of shorter writing tasks summary or abstract critical review of one/a few articles annotated bibliography project plan / proposal work book / log book wiki blog short-answer exam questions
Other Assessment Tasks MCQs, Capstone Projects or Courses, Portfolios, end-of-semester examinations(invigilated, open book or take home, etc.
50
Appendix K
Examples of Rubrics: other Universities
51
Appendix K
52
Appendix K
53
Appendix K
Holistic Critical Thinking Scoring Rubric* 4 Consistently does all or almost all of the following: Accurately interprets evidence, statements, graphics, questions, etc. Identifies the salient arguments (reasons and claims) pro and con. Thoughtfully analyzes and evaluates major alternative points of view. Draws warranted judicious, non-fallacious conclusions. Justifies key results and procedures, explains assumptions and reasons. Fair-mindedly follows where evidence and reasons lead. Does most or many of the following: Accurately interprets evidence, statements, graphics, questions, etc. Identifies relevant arguments (reasons and claims) pro and con. Offers analyses and evaluations of obvious alternative points of view. Draws warranted non-fallacious conclusions. Justifies some results or procedures, explains reasons. Fair-mindedly follows where evidence and reasons lead. Does most or many of the following: Misinterprets evidence, statements, graphics, questions, etc. Fails to identify strong, relevant counter-arguments. Ignores or superficially evaluates obvious alternative points of view. Draws unwarranted or fallacious conclusions. Justifies few results or procedures, seldom explains reasons. Regardless of the evidence or reasons, maintains or defends views based on self-interest or preconceptions. Consistently does all or almost all of the following: Offers biased interpretations of evidence, statements, graphics, questions, information, or the points of view of others. Fails to identify or hastily dismisses strong, relevant counter-arguments. Ignores or superficially evaluates obvious alternative points of view. Argues using fallacious or irrelevant reasons, and unwarranted claims. Does not justify results or procedures, nor explain reasons. Regardless of the evidence or reasons, maintains or defends views based on self-interest or preconceptions. Exhibits close-mindedness or hostility to reason.
http://www.temple.edu/tlc/resources/handouts/grading/Holistic%20Critical%20T hinking%20Scoring%20Rubric.v2.pdf
54
Appendix K
Rubrics for class participation: CLASS PARTICIPATION RUBRIC AND GUIDE A B C/D Actively Makes a Limited supports, sincere effort interaction engages, and to with listens to interact with peers peers peers (ongoing) (ongoing) Arrives fully Arrives Preparation is prepared at mostly, if not inconsistent every class fully, prepared session (ongoing) Plays an Participates When active role in constructively prepared, discussions in discussions participates (ongoing) (ongoing) constructively in discussions
Peer Interaction
F Virtually no interaction with peers
Preparation
Rarely or never prepared
Participation
Contribution to Class
Comments advance level and depth of dialogue
Group Dynamics
Group dynamic and level of discussion are often better because of candidates presence
Comments vague if given; frequently demonstrates lack of interest Relevant When Demonstrates comments are prepared, a based on relevant noticeable assigned comments lack of material are based on interest on assignments occasion Group Group Group dynamic and dynamic and dynamic and level of level of level of discussion are discussion are discussion are occasionally sometimes often better, but not disrupted disrupted by worse, by candidates candidates because of presence presence candidates presence
http://daviss.people.cofc.edu/people.cofc.edu_~daviss/EDEE610_files/Class%2 0Participation%20Rubric%20and%20Guide.pdf (March 2012)
55
Appendix K
Examples of Rubrics : HKSYU - Department of Counselling and Psychology Grading rubric of written assignment follows: Descriptors Marks Adheres fully to APA format; paper written in good and edited English; 85 and over: A explores topic from multiple perspectives and shows ability to think independently and critically; appropriate citations which demonstrate a depth and breadth of knowledge. Shows good effort in adhering to APA format; understandable and fluent English writing; demonstrates considerable complexity of thinking in exploring the topic; appropriate citations show a good range of references 80-84: B+
Basically able to follow APA format; shows reasonable ability to 72-79: B/Bcommunicate in English; demonstrates relevance and layered thinking in exploring the topic; appropriate citation show an acceptable range of references Shows some but not gross deviation from APA format; writes understandable and factual English; demonstrates basic understanding of the topic; appropriate citations showing a common range of references Shows highly noticeable deviation from APA format; English writing marginally intelligible; demonstrates bare-bone understanding of the topic; citations are either inappropriate or fail to demonstrate reasonable range of references Paper shows gross deviation from APA format; poor English writing; demonstrates inadequate or irrelevant understanding of topic; inappropriate citations and poor range of references 60-71: C
50-59: D
Below 50: E/F
56
Appendix K
Scoring Rubric for Creating an Outline Criteria Exemplary 1 point Competent 0.75 pts Developing 0.5 pts
Parallelism
Each heading and subheading successfully preserve parallel structure
A great majority of headings and subheadings successfully preserve parallel structure
Few if any of the headings and subheadings successfully preserve parallel structure
Coordination
All the information contained in Heading 1 has the same significance as the information in Heading 2
Most information contained in Heading 1 has the same significance as the information in Heading 2
Some or little information contained in Heading 1 has the same significance as the information in Heading 2 The general/specific relationship between headings and subheadings is visible in a minority of cases
Subordination
Heading information is general while the information in the subheadings is more specific
Most heading information is general while most of the information in the subheadings is more specific
Division
Each heading is successfully divided into two or more parts
Most headings are successfully divided into two or more parts
Few if any headings are successfully divided into two or more parts
57
Appendix K
Group Presentation (14%): Students are required to work on one of the given data-sets, interpret the results, and make logical conclusion out of it. The length of presentation is around 20 minutes (including Q & A section) and PowerPoint presentation is expected. Grading rubric for Group Presentation: Impeccable analysis of the statistical output. The chosen mode of delivery ensures that the audience finds the concepts clearly articulated and easy to comprehend. Conduct essential statistical analysis out of the given data set, explain and interpret the statistical output. Able to cover all the core concepts with adequate elaboration. Marks 85 and over; A
72-84; B
Only manages to conduct basic statistical analysis, yet still 60-71; C find no big faults in the interpretation of results. Only manages to conduct basic statistical analysis, and major mistakes in interpretation are found. Little or no effort was apparently made to create a cohesive presentation, or evidence of plagiarism. 50-59: D Less than 50; E, F
58
Appendix K
Presentation (20%) Students in week 1 will form groups to create the 30-minute long Presentation. In this presentation, students are required to provide a biological account explaining one of the psychological theories that they have come across. This assignment tries to help students gaining a deep reflection on the relationship between hardcore biology and mental phenomena. Corresponding teaching faculties with related specialties will be invited to sit and assist in marking this presentation. Marks Performance should exhibit a good linkage between the psychology theory and the biological account. It is not necessary to have a complete match between the two, but an in-depth discussion on the mismatch, if any, is expected in a Grade A presentation. A good ability to explain to the audience is another measure in indicating a Grade A presentation. Presentation is indicated by an understanding of the association between the psychological theory and the biological account. Students show a willingness to generate a discussion between the discrepancy between the psychological and biological accounts. Also, presenters may not fully be able to understand questions raised by the audience so that the presenters may not be able to explain to the audience as well as grade A presenters. Performance in AT 2 is indicated by a general linkage between the psychological theory and the biological account. Not much effort is shown in explaining any discrepancy between the two approaches. Presenters also may not capture the essence of the audiences question raised in the Question and Answer session. Performance is indicated by a shaky linkage between the psychological theory and the biological account. Minimal effort is shown in explaining any discrepancy between the two approaches. In the Question and Answer session, no relevant answer can be provided to the questions raised by the audience. Performance is indicated by a poor linkage and understanding between the psychological theory and the biological account. No effort is shown in explaining any discrepancy between the two approaches. Also, presenters simply fail to respond to any questions in the Question and Answer session. A
E/F
59
Appendix K
Critique writing (15%) Students in the lecture may not agree with the linkage between the psychological theory and the biological account presented by his/her classmate. Hence each student is required to write a 2-page critique on one of the presented topics, excluding his/her own topic. The assignment aims to help student develop an independent and critical perspective in examining different academic theories and arguments. The critique writing is due in hard copy format seven days after the corresponding day of presentation. Marks Writing should include a clear logical flow of the arguments. Reviews and relevant support are convincingly presented. Recommendations and suggestions match correspondingly with the issue being discussed. Writing should include a fluent discussion of the topic. Adequate reviews and support are provided in the piece. Recommendations and suggestions are made, though they may not fully match the core issue. Literature reviews are provided but are insufficient to support the presented opinions. Suggestions may not be made explicitly in the piece. An average writing on the topic. Ideas are not well connected in the writing. Also there is no attempt in making any suggestions or recommendations in the piece. A poor writing on the topic. Reviews are severely inadequate. There is no sign of any suggestion or recommendation in the writing. A
E/F
60
Appendix K
Participation (10%) Participation means attendance and involvement. Two marks will be given to students for attending all lectures and tutorial classes, according to the attendance sheet. This is an all-or-nothing system although cases will be reviewed if medical reasons are supplied. The remaining 8% will be primarily based on students performance in the tutorial classes throughout the course. Whether the student can initiate, or participate in, relevant discussion in the lectures and tutorial classes (with the lecturer) or outside the class (with the lecturer) is a way in assessing participation. Quality of the discussion is also equally considered in this assessment. Marks A B C D E/F
Grade A performance indicates an active role in the discussion in different contexts with insightful inputs in the discussion. Grade B performance shows an active role in the discussion and a sense of willingness to share. Grade C performance shows a general interest in the discussion may not actively participate in it. Grade D performance indicate a bare willingness to attend the discussion. No input is made throughout the discussion. Grade E/F performance shows no interest at all to all kinds of discussion.
61
Appendix K
Class Participation (Maximum of 10 points): Regular class attendance is required, and students are expected to actively participate in class. As a professional, it is expected that you will attend and arrive on time for every lecture and tutorial meeting. Active participation refers to generating and asking questions or voluntarily participates in class or group discussion. You are expected to be prepared for class discussion and activities. Hence you are expected to have completed the assigned readings prior to each meeting. All percentage of class attendance, quantity, and quality of participation will contribute to your participation score. Grading of class participation is as follows: Descriptors Attend 85% or more of lectures and tutorials. Frequently contributes to ongoing discussion, and participates actively in tutorials. The students absence from class is acutely felt and substantially diminishes the interactivity of the class. Attend at least 80-84% of lectures and tutorials. Contribute well to ongoing discussion, and participate noticeably in tutorials. The students absence from class is clearly felt and diminishes very much the interactivity of the class. Attend at least 70-79% of lectures and tutorials. Contributes to ongoing discussion, and participates in tutorials. The students absence from class is easily felt and diminishes the interactivity of the class. Attend 60-69% of lectures and tutorials. Sometimes contribute to ongoing discussion, but participation in tutorials is still sporadic and passive. The students absence from class is noticed and has some effect on the interactivity of the class. Attend 50-69% of lectures and tutorials. Hardly contribute to ongoing discussion unless called upon, and participation in tutorials is rare. The students absence from class is hardly noticed and has very little effect on the interactivity of the class. Attend less than 50% of lectures and tutorials. Never contribute to discussion even called upon, and no participation in tutorials is observed. The students absence from class is not noticed and has no effect on the interactivity of the class. Marks A
B+
B, B-
E, F
62
Appendix K
Grading of group presentation is as follows: Marks A
Information is rich and accurate. The chosen mode of delivery ensures that audience finds the concepts clearly articulated and very easy to comprehend. Class participation is maximized. Able to cover all the core concepts with clear elaboration. There is a substantial amount of class participation, and audience finds the presentation comprehensible and clear. Able to cover many of the core concepts with adequate elaboration. There is a suitable amount of class participation, and audience finds the presentation comprehensible. Able to cover some of the core concepts with fair elaboration. There is some amount of class participation. Although audience still finds the presentation comprehensible but with difficulty sometimes. Only manages to cover several of the core concepts and without adequate elaboration. Class participation is marginal, and audiences are left with gaps in knowledge. Very little core concepts are covered, and there is a general lack of elaboration. Minimal class participation, and audiences are unable to grasp the central themes of the presentation.
B+
B, B-
E, F
63
Appendix K
Grading of individual papers is as follows: Marks A
Adheres fully to APA format; paper written in good and edited English; explore topic from multiple perspectives and shows ability to think independently and critically; demonstrate ability to apply theory to practice and give clear examples; appropriate citations which demonstrate a depth and breadth of knowledge. Shows good effort in adhering to APA format; understandable and fluent English writing; demonstrates considerable complexity of thinking in exploring the topic; able to show good linkage between theory and practice and give some relevant examples; appropriate citations showing a good range of references. Basically able to follow APA format; shows reasonable ability to communicate in English; demonstrate relevance and layered thinking Shows some but not gross deviation from APA format; writes understandable and factual English; demonstrates basic understanding of the topic; makes attempt to link theory with practice; appropriate citations showing an common range of references. Shows highly noticeable deviation from APA format; English writing marginally understandable; demonstrates bare-bone understanding of the topic; makes minimal attempt to link theory with practice; citations are either inappropriate or fail to demonstrate reasonable range of references. Paper shows gross deviation from APA format; poor English writing; demonstrates inadequate or irrelevant understanding of topic; no attempt to link theory with practice; inappropriate citations and poor range of references.
B+
B, B-
E, F
64
References
Chapter 1 1. Biggs. J & Tang. C, Teaching for Quality Learning at University, 3rd ed, p.177. Open University Press, 2007. 2. Tyler, R.W. (1949) Basic Principles of Curriculum and Instruction . Chicago: University of Chicago 3. Pang, To and Man, The Impact of Outcome-Based Teaching and Learning: A Case Study In Hong Kong (China), 2007 Oxford Business and Economics Conference, Oxford University,ISBN : 978-0-9742114-7-3 4. P. Ewell, a report to the UGC, 2006. 5. Para1.2, p.5, Quality Assurance Council, Audit Manual, http://www.ugc.edu.hk/eng/qac 6. http://www.hkcaavq.edu.hk/en/news_events_qa.asp Chapter 2 7. Writing Intended Learning Outcomes A guide: writing intended learning outcomes at course level: NSHU, the Swedish Agency for Networks and Cooperation in Higher Education. Source: http://kursutveckling.se/dok/Larandemal_eng_061011.pdf 8. Guide to Writing Course Learning Outcomes at DCU: http://www.dcu.ie/afi/docs/FINAL_GUIDE_LOs-1%20May%2019th.pdf 9. http://www.learningandteaching.info/teaching/objectives.htm 10. http://www.ces-vol.org.uk/index.cfm?pg=123 11. John Biggs (1999) Assessment: An integral part of the teaching system. AAHE Bulletin 51, 9 (May), pp.10-12. 12. http://www.hksyu.edu/resourcebank/p1.htm Chapter 3 13. Outcomes-Based Teaching and Learning (OBTL) What is it, Why is it, How do we make it work? (Pre-workshop Reading), John Biggs & Catherine Tang
65
14. Teaching and learning methods (adapted from Bourner, 1997 and Biggs, 2003) 15. Introduction to Course Design, Teaching Development / Wahanga Whakapakari Ako, Dorothy Spiller, February 2011 16. http://www.brookes.ac.uk/services/ocsld/firstwords/fw21.html 17. Guidelines for Programme/Course Design and Review (media_178827_en.pdf), Learning and Teaching Centre, University of Glasgow, Revised November 2010 Chapter 4 18. Guidelines for Programme/Course Design and Review (media_178827_en.pdf), Learning and Teaching Centre, University of Glasgow, Revised November 2010 19. UK Employment Department paper (1992) cited in Brown and Knight, op cit, p22) 20. Brown, G, Assessment: A Guide for Lecturers, LTSN Generic Centre Assessment Series 2001, p.7 21. http://www.ncrel.org/sdrs/areas/issues/methods/assment/as7sele2.htm 22. Outcomes-Based Teaching and Learning (OBTL) What is it, Why is it, How do we make it work? (Pre-workshop Reading), John Biggs & Catherine Tang 23. http://www.ncrel.org/sdrs/areas/issues/methods/assment/as7purp.htm 24. Joan Herman, Pam Aschbacker, and Lynn Winters (1992) 25. Student-Created Rubrics: http://pblmm.k12.ca.us/PBLGuide/Activities/DesignRubric.html 26. Creating an Assessment Plan, Loyola Marymount University, http://www.lmu.edu/about/services/academicplanning/assessment/Assessm ent_Resources/Creating_an_Assessment_Plan.htm 27. Davies, Allan, 2001: Writing Learning Outcomes and Assessment Criteria in Art and Design (October 30, 2001 QAA Art and Design Reviewers workshop) ADC LTSN (Learning And Teaching Fund Project: Effective Assessment in Art and Design). University of Brighton-Faculty of Arts & Architecture. Source: http://www.arts.ac.uk/docs/cltad_learningoutcomes.pdf
66
28. Seven Steps to Align Your Curriculum With Standards, Rev. 10/02, CIC Group, Boalsburg, PA (814) 466-7191 29. Top 10 Tips on Assessment for Learning, Adapted from the University of Kent, based on Gibbs, G & Simpson, C (2004) Conditions under which assessment supports students learning, Learning and Teaching in Higher Education, Issue 1., and Brown, S, Race, P & Smith B, (1996) 500 Tips on Assessment, Kogan Page: London.
67

Outcome Based Teaching and Learning

Caricato da

Informazioni sul documento

Copyright

Formati disponibili

Condividi questo documento

Condividi o incorpora il documento

Opzioni di condivisione

Hai trovato utile questo documento?

Questo contenuto è inappropriato?

Copyright:

Formati disponibili

Outcome Based Teaching and Learning

Caricato da

Copyright:

Formati disponibili

Edited by: Aman Shah and Andrea Hope Published by: Hong Kong Shue Yan University 10 Wai

2012 by Hong Kong Shue Yan University

Message from the Academic Vice President

Professor Y.S. Hu Academic Vice President

HKSYU OBTL Handbook

Chapter 1 Implementing Outcomes Based Teaching and Learning (OBTL)

HKSYU OBTL Handbook

HKSYU OBTL Handbook

HKSYU OBTL Handbook

The Starting Point

HKSYU OBTL Handbook

Chapter 2 Intended Learning Outcomes (ILOs)

HKSYU OBTL Handbook

Writing Intended Learning Outcomes Why Bother?

How to Write Intended Learning Outcomes

HKSYU OBTL Handbook

HKSYU OBTL Handbook

Example: Subject Cognitive Psychology

Course Intended Learning Outcomes (CILOs)

Writing your ILOs9:

HKSYU OBTL Handbook

Tips to Address Common Pitfalls10

Tips to Address Pitfalls

Vague or Overly Specific Language

Ambiguous Words and Phrases

Too Many ILOs

http://www.dcu.ie/afi/docs/FINAL_GUIDE_LOs-1%20May%2019th.pdf Page 11, http://www.dcu.ie/afi/docs/FINAL_GUIDE_LOs-1%20May%2019th.pdf

HKSYU OBTL Handbook Pitfalls

Tips to Address Pitfalls

Overuse of the Same Verb

HKSYU OBTL Handbook Pitfalls

Tips to Address Pitfalls

HKSYU OBTL Handbook

Want more information?

HKSYU OBTL Handbook

Mapping Intended Learning Outcomes (ILOs)

HKSYU OBTL Handbook Figure 1

Teaching and Learning Activities Designed to elicit desired verbs

HKSYU OBTL Handbook

Chapter 3 Teaching and Learning Issues

Variety of Teaching and Learning Strategies

Choosing Learning Activities That Support ILOs

Information searching tasks Directed private study

Reflective journals Self-assessment Peer assessment

HKSYU OBTL Handbook

Mapping ILOs to Teaching and Learning Activities (TLAs)

HKSYU OBTL Handbook

HKSYU OBTL Handbook

Chapter 4 Assessment Issues

HKSYU OBTL Handbook

Developing an Assessment Plan

Some questions that require consideration when choosing what we assess:

Marking and Grading Issues

HKSYU OBTL Handbook

HKSYU OBTL Handbook

Designing Rubrics Huba & Freed, 2000

Include these as columns and label them.

Suggested Rating Scales Relating to Rubrics:

HKSYU OBTL Handbook

How to Determine if Your Assessment Plan is Working

Creating an Assessment Plan, Loyola Marymount University, http://www.lmu.edu/about/services/academicplanning/assessment/Assessment_Resources/Creating_an_Ass essment_Plan.htm

HKSYU OBTL Handbook

HKCAAVQ ILOs OBTL PILOs PBL QAC

QF Rubrics Teachers Resource Bank TLAs UGC