An Instrument to Assess Secondary School Mathematics
Teachers Assessment Practice
Shafini Suhaimi Sultan Idris Education University, Tanjong Malim, Malaysia shafinisuhaimi@gmail.com
Mohd Faizal Nizam Lee Abdullah Sultan Idris Education University, Tanjong Malim, Malaysia faizalee@fsmt.upsi.edu.my
Mohd Sahandri Gani Hamzah Sultan Idris Education University, Tanjong Malim, Malaysia sahandri@fppm.upsi.edu.my
Che Nidzam Che Ahmad Sultan Idris Education University, Tanjong Malim, Malaysia nidzam@fsmt.upsi.edu.my
Mazlini Adnan Sultan Idris Education University, Tanjong Malim, Malaysia maszlini@fsmt.upsi.edu.my
Noraini Mohamed Noh Sultan Idris Education University, Tanjong Malim, Malaysia noraini.mn@fppm.upsi.edu.my
Abstract: The purpose of this study was to develop an instrument to assess secondary school mathematics teacher's assessment practices. The instrument employed a two-layer rubric scale. Based on literature, this instrument was developed consisting of two main components: (1) mathematics teacher's characteristics; and (2) mathematics teacher's assessment practices. For the first component, the mathematics teacher's characteristics encompassed of four constructs including teachers beliefs and values, teacher's knowledge, teachers skills and teacher's attitudes. The second component, teacher's practices comprised of five constructs including learning outcomes, subjects blueprint, expected target result, folio A and folio B. The instrument will inform us on mathematics teachers' assessment practices in the classroom. With this information, various types of assessment tools in teaching and learning mathematics can be developed.
Education process is divided into three main components namely teaching, learning and assessment that complement each other to achieve optimal results. NCTM (2000) emphasized that assessment as one of the six underlying principles of mathematics education stating that assessment should support the learning of important mathematics and furnish useful information to both teachers and students. Assessment plays an important role in examining students understanding of a concept or skill being taught (Noraini, 2005). To fulfill this role, assessment should be an ongoing process throughout teaching and learning. The three main purposes of assessment are: (1) assessment of learning; (2) assessment for learning; and (3) assessment as learning. Assessment of learning is intended to inform parents or the public about the ability of student related to curriculum while assessment for learning is to allow teachers to determine the next steps to be taken to enhance students learning (Azizi, 2010). Assessment as learning is for students to monitor and self assess their own learning. In order to achieve these purposes, various forms of assessment can be used by teachers such as portfolios, projects and written assessment. Assessment usually focuses on individual learning, learning environment, institutions involved and learning system (Azizi, 2010). Therefore, the assessment process should be integrated with classroom activities so that assessment carried out can improve the effectiveness of teaching and learning. Teachers assessment practice is crucial to help improve student learning. Through this study, a two- layer rubric scale instrument was developed to assess secondary school mathematics teachers assessment practices. This instrument is hoped to help teachers identify their own assessment practices.
Problem Statement
Assessment is an important phase in teaching and learning as it not only assesses what students have learned and what they need to learn more but also helps teachers to identify suitable and relevant instructions. Assessment at the beginning of the lesson helps teachers to identify students existing knowledge, strengths and weaknesses that could determine the appropriate method to employ in teachers instructional planning. During teaching and learning, assessment carried out could help teachers assess students understanding and their achievement in that particular subject. At the end of the lesson, assessment is used to determine the level of acceptance of what students learned and plan the improvement from the results obtained (McMillan, 2007). Classroom assessment includes a variety of students feedback as well as discussion to measure and evaluate the quality of learning process that took place (Azizi, 2010). The use of appropriate assessment can help teachers identify the problems faced by students while obtaining feedback on teaching activities. There are variety of assessment practices that can be adopted by teachers to improve the effectiveness of teaching and learning. Despite the method used by teachers, classroom assessment should be carried out continuously to help improve student learning in each teaching unit.
As such, teachers assessment practices in the classroom are vital to ensure the best teaching and learning practices can be implemented. Thus through this study, a two-layer rubric instrument was developed to assess secondary mathematics teachers assessment practices. This two-layer rubric instrument provides a more detailed input which describes not only score but also the scale according to their level of rubric scores. In addition, each item can also be explained using their mean scores and a clearer descriptive description of either weak, moderate or good is justified. Each item in this instrument will be accompanied by significant and relevant rubric. Scale of each item is determined by the scale of five scores and five rubrics for each item were evaluated using score 0, 1 or 2 (refer Table 2).
Objectives
The objective of this study is to develop a two-layer rubric instrument to assess secondary school mathematics teachers assessment practices.
Research Question
The following questions will be used to address the objective of this study: i. How the two-layer rubric instrument to assess secondary school mathematics teachers assessment practices was developed? ii. What is the content validity? iii. What is the satisfactory level of reliability?
Conceptual Framework
The conceptual framework in Figure 1 shows a brief overview on the interrelationship between the constructs that underlie the study employing the Stufflebeams (1960) evaluation model context-input-process-product (CIPP) which describe four possible actions. The input of this study is the demographic of secondary school mathematics teachers. The process includes mathematics teachers characteristic as teachers beliefs and values, teachers knowledge, teachers skills and teachers attitudes. Finally, the output consists of mathematics teachers assessment practices related to subjects blueprint, learning outcomes, expected target result, folio A and folio B. Folio A includes record keeping, data interpretation and reporting while folio B consists of students enrichment, restoration and recognition.
Figure 1: Conceptual Framework
Construction of Instrument
The secondary school mathematics teachers assessment practices is divided into two main components; (1) mathematics teachers characteristics; and (2) mathemat ics teachers assessment practices. The components are described as follows:
An Instrument to Assess Secondary School Mathematics Teachers Assessment Practices Input Process Output Teachers demographic 1. Teachers beliefs and values 2. Teachers knowledge 3. Teachers skills 4. Teachers attitudes 1. Subject blueprint 2. Learning outcomes 3. Expected target result 4. Folio A 5. Folio B Mathematics Teachers Characteristics There are several characteristic that should be possessed by teacher to form appropriate assessment practices. Such characteristics can be divided into four constructs and these constructs are mainly from the literature.
Teachers beliefs and values Beliefs refer to confidence or feel definite about something while value refers to degree of quality, status or standard (Kamus Dewan, 2005). Beliefs and values complement each other. Teachers beliefs includes believes on students involvement in classroom (Song & Kim, 2010), student self assessment (Mohamad Azhar & Shahrir, 2007), discussion on assessment criteria between teacher and students (Kanjee et al., 2010) and also believes on the important of teaching (Love & Kruger, 2005). Other than that most teachers felt that every student needs to be noted in the process of planning instruction as students feedback are taken into consideration to improve teaching. Mathematics education values refer to quality instilled through teaching and learning practices (Mohd Uzi, 2007). An example of Mathematics values is asking student to review the solution of the problems they engaged with. Mathematics education is essentially the value under pedagogy and socio-cultural consideration that take place in the classroom.
Teachers Knowledge Knowledge refers to anything that we know. In this study, knowledge refers to the knowledge of a mathematics teacher in developing their assessment practices. To be knowledgeable in the subject is necessary due to student sometimes asks questions beyond the syllabus and curriculum (Mohamad Johdi, 2007). Basic knowledge of education mostly focuses on pedagogical content knowledge (PCK) which closely related to the ability of teacher to transform knowledge into a form that suitable for teaching as well as compatible with the abilities and background of students. Teachers who have limited knowledge focus on specific concepts rather than establishing relationships between facts, concepts, structure and practice of mathematics. Some of other knowledge teacher should have is knowledge of mathematics, knowledge about mathematics, knowledge on variety of assessment techniques and knowledge about method of assessment.
Teachers Skills Skills refer to efficiency or know-how. Skills within teacher on subject being taught are very important to ensure teaching and learning helps students to increase their understanding. One of the skills that should be possessed by mathematics teacher is the skill to apply various teaching methods to suit students and their surroundings. Mohamad Johdi (2007) stated that the diversity of teaching methods which appropriate with content and skills is necessary to ensure smooth, engaging and effective learning. Oral questioning is important for assessment for learning as it provides the opportunities and encourages students to ask questions, to expand their learning, to confirm what they have learned, knowing what need to be given more attention and help them to make notes (Henry, 2002). Teaching diversity and diversity of assessment technique used also play a role in ensuring teaching and learning is at the optimal level.
Teachers Attitudes Attitude refers to as feelings, thoughts, beliefs, and appreciation shown through behaviour either positive or negative. In this study, attitude is referred to the willingness of the teachers to conduct assessment as well as their views on the use of specific assessment techniques. Level of preparation by teachers to assess student and confident level to carry out assessment is influenced by the way assessments done given the primary consideration is the suitability of the applied assessment with learning objectives (Suah et al., 2010). Watt (2005) in his study identify teachers attitude towards the use of alternative assessment and teachers employed traditional testing as their primary method of assessment.
Mathematics Teachers Assessment Practices Practice refers to something familiar done by an individual. Teachers assessment practices in this study comprises of five constructs including subject blueprint, learning outcomes, expected target result, folio A and folio B. These constructs are gathered through literature.
Subject Blueprint The subject blueprint includes headcount by teachers to identify students academic progress and their career potential. The rationale of implementing headcount in school is to project excellence practiced in reflecting the potential of students (Zulkefly, 2011). Several possible areas of concern of headcount include take-off value (TOV) process, setting expected target result (ETR) and operational targeted increment (OTI). TOV refers to the value of student work based on the existing potential and usually identified based on students year-end results. OTI represents an improvement of learning outcomes with respect to some evaluation before reaching ETR.
Learning Outcomes Good planning and effective learning is necessary in order to achieve high quality learning. Learning outcomes are set and to accomplish these outcomes students are to displayed their knowledge that they have learned to the tasks given to them (Noorlyda, 2012). Learning outcomes provide information on what need to be mastered by student and what effective method of learning teachers can employ. Characteristics of good learning outcomes are written explicitly, in detailed and are sequenced according to the level of difficulties based on the syllabus and are in the form of behavior that can be observed.
Expected Target Result (ETR) The expected target result are targets of learning outcomes. These targets are based on the students final results (Zulkefly, 2011). ETR is usually determined through results gathered from public examinations and achievement of the last five years will be taken into account. Teacher need to explain to students why the tasks need to be done, the objectives to be achieved, encourages them to participate and share the assessment criteria in order to achieve these targets. If the targets are not met, teachers should take appropriate action to overcome this.
Folio A Folio A includes record keeping by teachers and its interpretation for reporting purposes. Record is registered activities related to the development of systematic information, ability, progress and achievement of students (Nor Haiza, 2010). Upon completion of the data collection process, data interpretation was done to clarify student learning. Data are then translated according to the purpose collecting them. Reporting is the final stage in assessing student and usually done in the form of a written document that describes the progress of a student during school sessions for a particular subject (Azizi, 2010).
Folio B Folio B includes the enrichment, restoration and recognition process which are carried out after the completion of the assessment activities. Follow-up measures are essential steps to evaluate the effectiveness of students learning (Mohamad Johdi, 2007). It can be done through reinforcement, guidance or enrichment. Reinforcement is conducted based on the knowledge or skills that have been mastered by students while enrichment activities provide the opportunities for student to carry out more challenging activities. Recognition is given to outstanding students as it encourages them to excel (Utusan, 2012).
Based on the literature review obtained from previous study, an instrument specification table, Table 1 was developed as follow:
Table 1: Instrument Specification Table
Construct Variable Questioning Rubric Item Scale Knowledge Definition of educational assessment What do you know about the process of educational assessment? gather information about student learning, give meaning to information, integrate information, make inferences, make decision to improve learning I know that assessment in education is a process to Rubric scale 0 (none), 1 (partially), 2 (entirely) Purpose of assessment What do you know about the purpose of assessment carried out?
provide feedback on student progress, elicited students potential, identify student weaknesses and strength, decide on future action, evaluate the effectiveness of teaching method The purpose of assessment carried out is to Rubric scale 0 (none), 1 (partially), 2 (entirely) Assessment characteristic What do you know about characteristics of good assessment practices? objectivity, easy to administer, easy to interpret, comprehensive, practical I know that characteristic of good assessment includes Rubric scale 0 (none), 1 (partially), 2 (entirely)
Table 2 shows an example of the two-layer rubric instrument that has been developed.
Title : Secondary School Mathematics Teachers Assessment Practices For each statement provided, choose only one number that most accurately describe your assessment practice for each rubric by using either point: 0 - none which indicate that the statement does not explain what you do or believe 1 - partially which indicate that the statement only explain partially on what you do or believe 2 - entirely which indicate that the statement explain what you do and believe
Table 2: Example of two-layer rubric instrument
1. I know that assessment in education is a process to
Scale
Rubric gather information about students learning. give meaning to information. integrate information. make inferences. make decision to improve learning. 2. The purpose of assessment carried out is to
Scale
Rubric provide feedback on student progress. elicited students potential. identify students weaknesses and strength. decide on further action. evaluate the effectiveness of teaching methods. 3. I know that characteristic of good assessment includes
Scale
Rubric objectivity. easy to administer. easy to interpret. comprehensive. practical.
Instrument Validation
The instrument was validated by five experts to determine the appropriateness of the item listed representing each construct. The principle used to validate the instrument is by referring to the agreement of the five experts. The minimum level of acceptance for each item validation is not less than 80 percent. Based on the analysis of the agreement of the five experts, 44 items were maintained, 6 items were removed while 12 items were improved and modified. Improvements and modifications were made based on the recommendation and comments given by the experts in terms of sentence structure, replacement of idea or words used or rephrasing items and rubric.
Instrument Reliability
Reliability refers to the accuracy and precision of a measurement procedure. Pilot study was conducted to ensure the instrument developed has high level of reliability. High reliability is intended to defend the accuracy of instrument developed. In this study, a pilot study was conducted on secondary school mathematics teachers. Using the feedback obtained, reliability analysis was then conducted using Statistical Package for Social Sciences (SPSS) version 19. Results of the analysis are summarized in Table 3.
Table 3: Summary of Reliability Analysis
Construct Value of Cronbach Alpha Teachers beliefs and values 0.803 Teachers knowledge 0.788 Teachers skills 0.800 Teachers attitudes 0.658 Subject blueprint 0.801 Learning outcomes 0.720 Expected target result 0.824 Folio A 0.797 Folio B 0.698 Total 0.948
According to Lim (2007), reliability value which approximately 1.00 shows that the instrument has high reliability. Therefore, this instrument has high internal consistency. Based on the reliability value obtained, some items were modified or removed to enhance the reliability value.
Conclusion
An instrument should be able to measure what it suppose to measure. Development of instrument used in this study is done via few steps. First of all, constructs and variables for each constructs were obtained as a result from literature review on the previous study. Next the instrument specification table was developed to create the item and rubric of instrument by taking into account the constructs, variables and questioning. Validation by experts and pilot study was then conducted to determine the validity and reliability of the instrument.
References
Azizi Ahmad. (2010). Pentaksiran pendidikan. Kuala Lumpur, Malaysia: Dewan Bahasa dan Pustaka. Azizi Yahaya, Shahrin Hashim & Noordin Yahaya. (2005). Sejauhmanakah model Stufflebeam (KIPP) boleh membantu dalam penilaian program pembelajaran? In Yusuf Boon & Jamaluddin Ramli. Isu-isu dalam psikologi pendidikan sekolah (pp. 192-213). Skudai, Johor: UTM. Ball, D. L., & Bass, H. (2000). Interweaving content and pedagogy in teaching and learning to teach: Knowing and using mathematics. In J. Boaler (Eds.), Multiple perspectives on the teaching and learning of mathematics (pp. 83-104). Westport, CT: Ablex. Barksdale-Ladd, & Thomas. (2000). Whats at stake in high-stakes testing. Journal of Teacher Education, 51(5), 384-397. Bhasah Abu Bakar. (2003). Asas pengukuran bilik darjah. Tanjung Malim, Perak: Quantum Books. Black, P., & William, D. (1998). Inside the black box: Raising standards through classroom assessment. Phi Delta Kappan, 80(2). Brahier, D. J. (2009). Teaching secondary and middle school mathematics. (3rd ed.). Boston, MA: Pearson. Chua, Y. P. (2006). Asas statistik penyelidikan. Kuala Lumpur: McGraw-Hill. Creswell, J. W. (2009). Research design: Qualitative, quantitative, and mixed methods approaches. (3rd ed.). Thousand Oaks, CA: Sage. Fink, A., & Kosecoff, J. (1998). How to conduct surveys a step-by-step guide. (2nd ed.). Thousand Oaks, CA: Sage. Fraenkel, J. R., Wallen, N. E., & Hyun, H.H. (2012). How to design and evaluate research in education. (8th ed.). New York, NY: McGraw-Hill. Handal, B. (2003). Teachers mathematical beliefs: A review. The Mathematics Educator. 13(2), 47-57. Henry, J. (2002). Forget the marking, start talking. Retrieved from http://www.tes.co.uk/article.aspx?storycode=366141 Joshua, M. T. (2004, August). Secondary school: An assessment and evaluation. Paper presented at the National workshop on developing education: Issues of standards and sustainability in secondary schools in Nigeria. Kamus Dewan (4th ed.). (2005). Kuala Lumpur, Malaysia: Dewan Bahasa dan Pustaka. Kanjee, A., Molefe, M. R. M., Makgamatha, M. M., & Claassen, N. C. W. (2010). Teachers assessment practices in South African schools. (Unpublished report). Pretoria, South African: HSRC Press. Kendra, L. S. (2009). A brief summary of supervision models. Retrieved from http://www.gallaudet.edu/Documents/Academic/COU_SupervisionModels[1pdf Krejcie, R. V., & Morgan, D. W. (1970). Determining sample size for research activities. Educational and Psychological Measurement, 30(3), 607-610. Kuala Lumpur, Kementerian Pendidikan Malaysia. (2006). Sukatan pelajaran kurikulum bersepadu sekolah menengah (KBSM) matematik. Kuala Lumpur, Malaysia: Pusat Perkembangan Kurikulum. Lim, C. H. (2007). Penyelidikan pendidikan: Pendekatan kuantitatif dan kualitatif. Selangor, Malaysia: McGrawHill. Love, A., & Kruger, A. C. (2005). Teacher beliefs and student achievement in urban schools serving African American students. Journal of Educational Research, 99(1), 87-98. Lowery, N. V. (2003). Assessment insight from the classroom. The Mathematics Educator, 13(1), 15-21. McMillan, J. H. (2000). Fundamental assessment principles for teachers and school administrators. Practical Assessment, Research & Evaluation, 7(8). McMillan, J. H. (2007). Classroom assessment: Principles and practice for effective standards-based instruction. (4th ed.). Boston, MA: Allyn & Bacon. McMillan, J. H., & Schumacher, S. (2006). Research in education: Evidence-based inquiry. (6th ed.). Boston, MA: Allyn & Bacon. Mohamad Azhar Mat Ali, & Shahrir Jamaluddin. (2007). Amalan pentaksiran untuk pembelajaran di sekolah menengah. Jurnal Pendidikan, 27(1), 19-39. Mohamad Johdi Salleh. (2007). Guru efektif dan peranan guru dalam mencapai objektif persekolahan sekolah rendah: perspektif guru besar. Paper presented at Seminar Penyelidikan Pendidikan Institut Perguruan Batu Lintang. Mohd Fadhli Ahmad. (2010). Amalan pentaksiran pengajaran dan pembelajaran di kolej komuniti di negeri Johor. Paper presented at Seminar Transformasi Pendidikan Teknikal, Sungai Petani, Malaysia. Mohd Uzi Dollah. (2007). Penerapan nilai dalam pengajaran guru matematik sekolah menengah: Satu kajian kes. (Unpublished Doctoral Dissertation). Universiti Sains Malaysia, Pulau Penang. National Council of Teachers of Mathematics (NCTM). (2000). Principles and standards for school mathematics. Reston, VA: NCTM. Nik Mohd Rahimi Nik Yusoff, Zamri Mahamood, & Kamarulzaman Ab. Ghani. (2008). Motivasi pembelajaran kemahiran mendengar Bahasa Arab dan hubungannya dengan pencapaian pelajar [Electronic version]. Jurnal Pendidikan, 33, 3-18. Nisbet, S., & Warren, E. (2000). Primary school teachers beliefs relating to mathematics, teaching and assessing mathematics and factors that influence these beliefs. Mathematics Teacher Education and Development, 2, 34-47. Noorlyda Abd Latib. (2012). Mengapa hasil pembelajaran harus ditentukan? Retrieved from http://www.slideshare.net/NoorlydaAbdLatip/hasilpembelajarankpd4033presentation Nor Haiza. (2010). Rekod berterusan untuk mengesan perkembangan kanak-kanak. Retrieved from http://www.scribd.com/doc/37749354/rekod-berterusan-untuk-mengesan-perkembangan-kanak-kanak Noraini Idris. (2005). Pedagogi dalam pendidikan matematik. Kuala Lumpur, Malaysia: Utusan. Popham, W. J. (2007). Classroom assessment what teachers need to know. (5th ed.). Boston, MA: Allyn & Bacon. Russell, M. K., & Airasian, P. W. (2012). Classroom assessment concepts and applications. (7th ed.) Dubuque, Iowa: McGraw-Hill. Sapsford, R. (2007). Survey research. (2nd ed.). Thousand Oaks, CA: Sage. Senk, S. L., Beckmann, C. E., & Thompson, D. R. (1997). Assessment and grading in high school mathematics classrooms. Journal for Research in Mathematics Education, 28(2), 187-215. Shulman, L. S. (1986). Those who understand: Knowledge growth in teaching. Educational Researcher, 15(2), 4-14. Shulman, L. S. (1987). Knowledge and teaching: Foundations of the new reform. Harvard Educational Review, 57(1), 1-21. Sidek Mohd Noah. (2002). Reka bentuk penyelidikan: Falsafah, teori dan praktis. Serdang, Selangor: Penerbit UPM. Song, E., & Kim, K. (2010). Assessment for learning: Understanding teachers beliefs and practices. Retrieved from http://www.iaea.info/documents/paper_2fb234cf.pdf Stergar, C. (2005). Performance tasks, checklists, and rubrics. Glenview, Illinois: LessonLab. Suah, S. L., & Ong, S. L. (2012). Investigating assessment practices of in-service teachers. International online journal of educational science, 4(1), 91-106. Suah, S. L., Ong, S. L., & Shuki Osman. (2010). Pentaksiran pembelajaran pelajar: Amalan guru-guru di Malaysia. Malaysian Education Deans Council Journal, 5, 68-83. Suurtamm, C., Martha, K., & Arden, A. (2010). Teachers assessment practices in mathematics: Classrooms in the context of reform. Assessment in Education: Principles, Policy & Practices, 17(4), 399-417. Utusan online. (2012). Wajar beri pengiktirafan kepada pelajar cemerlang. Retrieved from http://www.utusan.com.my/utusan/Selatan/20121109/ws_04/Wajarberi-pengiktirafan-kepada- pelajarcemerlang Walshaw, M. (2012). Teacher knowledge as fundamental to effective teaching practice. Journal of Mathematics Teacher Education, 15(3), 181-185. Watt, H. M. G. (2005). Attitudes to the use of alternative assessment methods in mathematics: A study with secondary mathematics in Sydney, Australia. Educational Studies in Mathematics, 58(1), 21-44. Wilson, N. S. (2008). Teachers expanding pedagogical content knowledge: Learning about formative assessment together. Journal of In-service Education, 34(3), 283-298. Zhang, Z., & Burry-Stock, J. A. (2003). Classroom assessment practices and teachers self-perceived assessment skills. Applied Measurement in Education, 16(4), 323-342. Zulkefly Othman. (2011). Bengkel headcount ketua bidang. Retrieved from http://www.slideshare.net/inovatifinsan/headcount
Copyright statement
Copyright 2013 Shafini Suhaimi, Mohd Faizal Nizam Lee Abdullah, Mohd Sahandri Gani Hamzah, Che Nidzam Che Ahmad, Mazlini Adnan, Noraini Mohamed Noh: The author/s assign to 2 nd International Higher Education Teaching and Learning Conference 2013 a non-exclusive licence to use this document for personal use and in courses of instruction provided that the article is used in full and this copyright statement is reproduced. The author also grants a non-exclusive license to the organisers of the 2nd International Higher Education Teaching and Learning Conference 2013 to publish this document as part of the conference proceedings. Any other usage is prohibited without the express permission of the authors.