OECD Programme for International Student Assessment 2012

PISA 2012 MAIN SURVEY SCHOOL CO-ORDINATORS MANUAL

(CORE PISA)

Final version September 2011

Consortium: Australian Council for Educational Research (ACER, Australia) cApStAn Linguistic Quality Control (Belgium) Deutsches Institut fr Internationale Pdagogische Forschung (DIPF, Germany) Educational Testing Service (ETS, USA) Institutt for Lrerutdanning og Skoleutvikling (ILS, Norway) Leibniz - Institute for Science and Mathematics Education (IPN, Germany) National Institute for Educational Policy Research (NIER, Japan) The Tao Initiative: CRP - Henri Tudor and Universit de Luxembourg - EMACS (Luxembourg) Unit d'analyse des systmes et des pratiques d'enseignement (aSPe, Belgium) Westat (USA)

School Co-ordinator s Manual MS 12

School Co-ordinator s Manual MS 12

TABLE OF CONTENTS
INTRODUCTION .........................................................................................................................4 What is PISA? .................................................................................................................................... 4 The PISA assessment ........................................................................................................................ 4 STAGE I: PREPARATION ..............................................................................................................6 1. Nominate dates for the assessment ............................................................................................. 6 2. Prepare List of Eligible Students ................................................................................................... 9 3. Complete School s Computer System Diagnostic Report ............................................................ 10 STAGE II: BEFORE THE ASSESSMENT ....................................................................................... 111 4. Receive and check materials sent by EPRD .............................................................................. 111 5. Notify teachers, students and parents of the assessment ....................................................... 111 6. Review and update Student Tracking Form .............................................................................. 111 7. Give School Questionnaire to appropriate person at your school ............................................. 17 8. Organise and confirm assessment plans with school staff and with Test Administrator........... 17 9. Collect completed School Questionnaire ................................................................................. 177 STAGE III: DURING/AFTER THE ASSESSMENT ........................................................................... 188 10. Provide Test Administrator with Student Tracking Form and School Questionnaire............... 188 11. Be present throughout the assessment to assist Test Administrator (if possible) ................. 188 12. Arrange for follow-up session (if needed) .............................................................................. 188 13. Arrange assistance for Test Administrator for Computer-Based Assessment session ........... 188 Appendix 1: PISA 2012 Main Survey - Assessment Dates Form ....................................................... 200 Appendix 2: PISA 2012 Main Survey - School's Computer System Diagnostic Report ..................... 211 Appendix 3: PISA 2012 Main Survey - List of Eligible Students........................................................... 32

Field Code Changed

Field Code Changed

Field Code Changed

Field Code Changed

Field Code Changed

Field Code Changed

Field Code Changed

Field Code Changed

Field Code Changed

Field Code Changed

Field Code Changed

Field Code Changed

Field Code Changed

Field Code Changed

Field Code Changed

Field Code Changed

Field Code Changed

Field Code Changed

Field Code Changed

Field Code Changed

Field Code Changed

School Co-ordinator s Manual MS 12

Introduction
1. Thank you for agreeing to be a School Co-ordinator for PISA. The purpose of this manual is to help you step by step to understand how you can assist with the successful implementation of this large scale international assessment programme.

What is PISA?
2. PISA stands for the Programme for International Student Assessment, sponsored by the Organisation for Economic Co-operation and Development (OECD). PISA 2012 is the fifth PISA study that has been conducted since 2000 and involves more than 60 participating countries. PISA has the following characteristics: y y y y 4. 5. It is the world s largest international survey in education; It surveys students aged about 15 years; It assesses students preparedness for adult life; and It measures students Mathematical, Scientific, Problem Solving and Reading Literacy.

3.

Typically, between 4,500 and 10,000 students in each country take part in this programme. Undertaking PISA is important because the results of the assessment can be used: y y y to indicate how well prepared students in Malaysia are for learning once they leave school; by schools, education systems and governments to identify areas for improvement over time; and to allow a comparison of student performance and the learning environment between different countries.

6.

For more details about PISA, please refer to the information materials that Educational Planning and Research Division (EPRD) has provided or the OECD website at: http://www.pisa.oecd.org/. Note 1: All data collected in PISA are kept strictly confidential. No individual or school is identified in any report released from the assessment programme.

The PISA assessment

7. Thirty-five students from your school will be sampled to complete a two-hour paper-based test consisting of Mathematics, Science and Reading questions. These students will also be asked to complete a questionnaire. After the Paper-Based PISA and questionnaire, a sub sample of 14 students who participated in the print assessment will also participate in a forty-minute Computer-Based Assessment of Problem Solving. In addition, schools will be requested to complete one School Questionnaire. As a School Co-ordinator you will be responsible to act as the liaison between your school and EPRD. The key activities you will carry out are listed in Table 1Table 1 with their associated dates. Please note that the actual assessment will be conducted by a Test Administrator who will be appointed by EPRD.

8. 9.

Field Code Changed

Formatted: Font: No

School Co-ordinator s Manual MS 12

Table 1: Key Activities of PISA School Co-ordinators Stage I. Preparation Date 6 weeks prior to assessment Activity 1. Nominate dates for the assessment 2. Prepare a List of Eligible Students 3. Complete School s Computer System Diagnostic Report II. Before the Assessment 2 weeks prior to assessment 4. Receive and check materials sent by EPRD 5. Notify teachers, students and parents of the assessment 6. Review and update Student Tracking Form 1-2 weeks prior to assessment 7. Give School Questionnaire to appropriate person at your school 8. Organise and confirm assessment plans with school staff and with Test Administrator A few days prior to the assessment III. During/ After the assessment Assessment Day 9. Collect completed School Questionnaire 10. Provide Test Administrator with Student Tracking Form and School Questionnaire 11. Be present throughout the assessment to assist Test Administrator (if possible) 12. Arrange assistance for the Test Administrator for the Computer-Based Assessment session Same week as the main session if possible 13. Arrange for follow-up session (if needed) Page number 6 9 10 11 11 11 17 17

Field Code Changed

Field Code Changed

Field Code Changed

Field Code Changed

Field Code Changed

Field Code Changed

Field Code Changed

Field Code Changed

17 18 18 18

Field Code Changed

Field Code Changed

Field Code Changed

Field Code Changed

18

Field Code Changed

If you have any questions, please contact us at: EPRD, MOE. Tel: 03-88846548/6595, Fax:03-88846579, email:pisa@moe.gov.my

School Co-ordinator s Manual MS 12

Stage I: Preparation
1. Nominate dates for the assessment
10. PISA will take place between 09.04.2012 and 11.05.2012. Please provide details of your preferred assessment dates by completing the attached Assessment Dates Form (see Appendix 1Appendix 1). Then, e-mail or fax this form to pisa@moe.gov.my or 03-88846579 before 20.02.2012. Please take into account the issues listed on the Assessment Dates Form when nominating assessment dates. You will receive details of your assigned assessment date and time by 19.03.2012. Please inform the Principal and school staff of the details. The Paper-Based PISA will take approximately 3 hours and 15 minutes to administer as outlined in Table 2Table 2, and the Computer-Based Assessment (including preparation and administration time) will take about 1 hour 30 minutes as outlined in Table 3Table 3. Table 2: Timing of Paper-Based PISA Administration Activity Distributing the materials and reading the General Directions The Test Booklet Short break The Test Booklet Break The Student Questionnaire Collecting the materials and ending the session Total Time 10-15 minutes (approximately) 1 hour (exactly) No more than 5 minutes 1 hour (exactly) 15 minutes 35 minutes (approximately) 3-5 minutes (approximately) 3 hours and 15 minutes (approximately)

Formatted: Font colo

Formatted: Font: 11

Field Code Changed

11.

Field Code Changed

Formatted: Font: No

Field Code Changed

Formatted: Font: No

Table 3: Timing of Computer-Based Assessment Administration Activity Computer room setup Student log-ons, passwords and introducing the test Computer-Based Assessment practice session Completing the Computer-Based Assessment Ending the assessment session, collection and transfer of data Total 12. Time 15 minutes (approximately) 5 minutes (approximately) 15 minutes (approximately) 40 minutes (exactly) 15 minutes (approximately) 1 hour 30 minutes (approximately)

Please note that the Computer-Based Assessment session should take place after the Paper-Based PISA, and preferably on the same day. Because some students will participate in both the Paper-Based and Computer-Based sessions, it is best to schedule the start of the Paper-Based PISA at the beginning of the school day. It is recommended that the Computer-Based Assessment be scheduled for the afternoon. Please allow an adequate interval of at least 1 hour after the Paper-Based PISA has finished before beginning the Computer-Based Assessment session. The Test Administrator will need to prepare the Computer-Based Assessment area and run checks and preparations for each computer. Also, students participating in the Computer-Based Assessment will need a break between assessments.

13.

School Co-ordinator s Manual MS 12

[This page is left blank on purpose]

School Co-ordinator s Manual MS 12

Figure 1: Example of List of Eligible Students School Name: SMK Putrajaya 2 Address: Presint 9, Putrajaya Telephone: 03-88887678 List prepared by: Maria Adam Date List prepared: 17.02.2012 Fax: 03-88888768 E-mail: maria@smkp2.moe.gov.my Total Number of Students Listed: 14

DIRECTIONS: Please list ALL students in your school in Form 1 and higher born between 01.01.1996 and 31.12.1996.

(A) Student Name

(B) Form

Ronny Liew M. Muthusamy Salina Ahmad P. Letchumy Ahmad Amir Khaw Meng M. Mahendran Mariah Muda Florence Teoh Lai Kwang Tammy Lim V. Shahrukhkhan Sharifah Azman Kamal Hassan

4 3 4 4 4 4 4 3 4 4 4 4 3 3

(C) Gender (Female=1; Male=2) 2 2 1 1 2 2 2 1 1 2 1 1 1 2

(D) Birth Date (MM-YYYY) 12 11 12 05 04 09 07 04 07 10 12 08 09 07 1996 1996 1996 1996 1996 1996 1996 1996 1996 1996 1996 1996 1996 1996

(E) Study Programme 1 1 1 1 1 1 1 1 1 1 1 1 1 1

(F) SEN*

3 3 1

SEN = Special Education Needs

School Co-ordinator s Manual MS 12

2. Prepare List of Eligible Students

Steps for preparing a List of Eligible Students
Step 1: Identify ALL students at your school who meet the eligibility criteria
14. Criteria for PISA student eligibility are: y y 15. Enrolled in Form 1or higher; AND Born between 01.01.1996 and 31.12.1996.

PISA needs accurate information about all students at your school who meet the above criteria for student eligibility. A small number of these students may be unable to complete the PISA assessment. For now, any such students must be added to your list of students. Later, you may need to identify students who cannot participate in PISA. This will be covered in the next chapter.

Step 2: List each of the eligible students using the form in Appendix 3Appendix 3, along with their demographic information (see Figure 1Figure 1 for an example)
y y y y Student name Column A; Grade Column B; Gender Column C: Indicate 1 (female) or 2 (male); and Birth Date Column D: Indicate as MM/YYYY. For example, if an eligible student was born in September 1996, write 09/1996.

Formatted: Font: 10

Field Code Changed

Formatted: Font: Bo grammar

Field Code Changed

Step 3: Write a Study Programme code for each eligible student Column E
16. Table 4Table 4 below shows a code for each study programme. Please choose the appropriate study code for each eligible student and fill in column E. Table 4: Codes for Study Programme Study Programme Science Upper Secondary Arts Upper Secondary Technical/Vocational Secondary Religious Secondary Lower Secondary Code 1 2 3 4 5

Field Code Changed

Formatted: Font: No

Step 4: Identify students with Special Education Needs Column F

17. Column F is used to record the Special Education Needs of eligible students. The codes are defined in Table 5Table 5 below. Please identify any student who has a special education need by placing the appropriate code in column F. Table 5: Codes for Special Education Needs Description Functional disability disability student has a moderate to severe permanent physical Code 1 2

Formatted: Font: No

Field Code Changed

Cognitive, behavioral or emotional disability in the opinion of qualified staff, student has a cognitive, behavioral or emotional disability Limited assessment language experience student is not a native speaker of any of the languages of the assessment in the country and has limited proficiency in these languages

School Co-ordinator s Manual MS 12 Examples of using Special Education Needs codes

18. Please refer to Figure 1: Example of List of Eligible StudentsFigure 1: Example of List of Eligible Students, in which the following codes have been applied: y y Ahmad Amir is not a native English speaker and has less than one year of instruction in English. Therefore, he is coded as 3 in column F. Khaw Meng is not a native speaker of English. While he has received instruction in English for more than one year, he still has some difficulties with English expression and requires some extra tuition. Therefore, he is also coded as 3 in column F. M. Mahendran is blind and is coded as 1 in column F. Lai Kwang is physically disabled so he is coded as 1 in column F. V. Shahrukhkhan has been professionally assessed as cognitively delayed and is coded as 2 in column F. All other students do not have any special education needs and column F is left blank. For most students column F will be left blank, meaning they have no Special Education Needs.

Field Code Changed

Formatted: Font: No

y y y y

3. Complete School s Computer System Diagnostic Report

19. You will need to complete a School s Computer System Diagnostic Report (Appendix 2Appendix 2). This provides information about the computer hardware, operating systems and administrative permissions at your school and will be used to ascertain how many computers are available to implement the Computer-Based Assessment session. This information will be used for student sampling to determine the number of students who will participate in the Computer-Based Assessment. Therefore, it is important that the information you provide is as accurate as possible. The completion of this report will require input from a member of staff who is familiar with the technical aspects of the school s computer resources and the schools IT policies. Ideally this person will have responsibility for the school s IT systems. Please email or fax to the School s Computer System Diagnostic Report, along with the completed List of Eligible Students to pisa@moe.gov.my or 03- 88846579, EPRD, MOE before 20.02.2012.

Formatted: Font colo

Field Code Changed

20.

21.

About Quality Monitors

22. On the day of the assessment, PISA Quality Monitors will visit some randomly selected schools. The main task of the PISA Quality Monitors is to document the extent to which testing procedures have been followed and are being implemented by the Test Administrator. In order to do this, PISA Quality Monitors will visit some schools, observe the PISA assessment sessions and have a short interview (approximately 15 minutes) with you after the assessment session. Their task is strictly limited to these PISA related activities. Should your school be selected for this visit, a Quality Monitor will contact you a few days prior to the assessment to inform you of his/her visit. On the morning of the assessment he/she will report to the school office. Please contact EPRD, MOE if you have any questions about the visit of the Quality Monitor.

23.

10

School Co-ordinator s Manual MS 12

Stage II: Before the Assessment

4. Receive and check materials sent by EPRD, MOE.
24. About two weeks prior to the assessment, EPRD, MOE will send you a package of the following materials: y y y 25. The Student Tracking Form listing the students who have been sampled to participate in the assessment; The School Questionnaire; and Other materials, e.g., promotional materials, form for reimbursement of expenses, participation certificates, etc.

Please check whether the package includes these materials. If you find any materials missing or damaged, please contact EPRD, MOE as soon as possible. Please note also that EPRD, MOE will send all assessment materials to the Test Administrator who has been assigned to your school.

5. Notify teachers, students and parents of the assessment

26. 27. Please inform teachers, students (especially the sampled students), and parents of the sampled students according to the school s policies. It is very important to introduce the aims of PISA to school staff and students prior to the assessment in order to engage students and promote their participation. Therefore, please use the informational and promotional materials that have been provided to assist you with this activity such as posters, brochures as well as sample covering letters. Please request the assistance of a staff member who has knowledge of the school s computers to provide technical support for a short duration (i.e. at the beginning and end of the Computer-Based Assessment session for the initial set-up of the computers, if necessary).

28.

6. Review and update Student Tracking Form

29. You will receive a Student Tracking Form. This form lists sampled students along with their background data. You will need to review this form and complete it with additional information. An example of the Student Tracking Form is provided in Figure 2Figure 2.

Field Code Changed

Formatted: Font: No

11

School Co-ordinator s Manual MS 12 [This page is left blank on purpose]

12

School Co-ordinator s Manual MS 12

Codes to enter into SEN column: 1 Functional disability 2 Cognitive, behavioral or emotional disability 3 Limited assessment language experience

Figure 2: PISA 2012 Main Survey - Student Tracking Form (example after completion) School Name: SMK Putrajaya 2 School Coordinator: Maria Adam Completed by EPRD, MOE To be checked by School Co-ordinator Test Administrator: Rashid Ishak
To be completed by School Coordinator Completed by EPRD, MOE, identifies CBA students

Codes to enter into N/P columns: 2 Parent refusal 3 Excluded on SEN basis 4 Now enrolled at another school 5 Not enrolled in this school, enrolment unknown n Does not meet PISA criteria for eligibility

For use by School Co-ordinator if needed

Region

Stratum ID

School ID

Student ID

LST Line Number

Student Name

Form

Gender (F=1; M=2)

Birth Date

Study Prog.

SEN*

N/P PISA

CBA

COMMENTS

00 00 00 00 00 00 00 00 00 00 00 00 00 00
*

01 01 01 01 01 01 01 01 01 01 01 01 01 01

010 010 010 010 010 010 010 010 010 010 010 010 010 010

00004 00009 00005 00002 00014 00008 00011 00013 00001 00007 00012 00010 00003 00006

00003 00008 00012 00016 00020 00024 00028 00033 00037 00041 00045 00049 00053 00057

Ronny Liew M. Muthusamy Salina Ahmad P. Letchumy Ahmad Amir Khaw Meng M. Mahendran Mariah Muda Florence Teoh Lai Kwang Tammy Lim V. Shahrukhkhan Sharifah Azman Kamal Hassan

4 3 4 4 4 4 4 3 4 4 4 4 3 Y6 3

2 2 1 1 2 2 2 1 1 2 1 1 1 2

12 11 12 05 04 09 07 04 07 10 12 08 09 07

1996 1996 1996 1996 1996 1996 1996 1996 1996 1996 1996 1995 1996 1996 1996

1 1 1 1 1 1 1 1 1 1 1 1 1 1 2 1 n 3 n 3 3 1 3 3 4

3 9 5

7 10 6 2
Corrected her grade, which is lower than the eligibility criterion Corrected her birth year, which is outside the eligibility criterion

SEN = Special Education Needs CBA = Computer-Based Assessment

13

School Co-ordinator s Manual MS 12

Steps for reviewing and completing the Student Tracking Form

Step 1: Check student details and demographics
30. For columns Student Name , Grade , Gender , Birth Date , Study Programme , and SEN , identify and correct missing or incorrect information regarding student name, grade, gender, birth date, study programme and special education needs (SEN).

Step 2: Identify students who CANNOT participate in PISA

31. Column N/P PISA is used to record sampled students who cannot participate in PISA. The intention of PISA is to be as INCLUSIVE as possible. However, some students may not be able to participate for various reasons. These reasons for non-participation have to be recorded in this column of the Student Tracking Form, using the following specific codes:

Code 2 - Student s parents refuse for their child to participate in PISA

32. A small number of parents may refuse to allow their child to participate in PISA. If you have received such notification from parents, record a code 2 for this student in column N/P PISA .

Code 3 - Student with Special Education Needs who cannot be assessed

33. The guidelines in Table 6Table 6 are provided to assist you in deciding whether a student with Special Education Needs can be excluded from PISA. If a student with a special education need is to be excluded from the assessment, record a code 3 for this student in column N/P PISA . Note 2: Many students with a Special Education Need will be able to be included in the assessment. Table 6: Guidelines - Exclusion/inclusion of students with Special Education Needs Type of SEN Functional disability Students who may be excluded The student has a moderate to severe permanent physical disability such that she/he cannot participate in the PISA testing situation. Code 3 in column N/P PISA . The student has a cognitive, behavioral or emotional disability such that in the opinion of qualified staff, she/he cannot participate in the PISA testing situation. This includes students who are cognitively, behaviorally or emotionally unable to follow even the general instructions of the assessment. Code 3 in column N/P PISA . The student meets ALL of the following three criteria: - not being a native speaker in the assessment language, - having limited proficiency in the assessment language, and - having received less than one year of instruction in the assessment language. Code 3 in column N/P PISA . Students who should be included The student can respond to the assessment. The student can respond to the assessment. He/she should NOT be excluded solely because of poor academic performance or disciplinary problems.

Field Code Changed

Formatted: Font: No

Cognitive, behavioral or emotional disability

Insufficient assessment language experience

The student meets ONLY one or two of the criteria.

15

School Co-ordinator s Manual MS 12

Codes 4 and 5 - Student was transferred to another school or left the school
y y If a student has transferred to another school, record a code 4 for this student in column N/P PISA . If a student has left school, but it is not known which school the student has transferred to, record a code 5 for this student in column N/P PISA .

Code n - Student does not meet student eligibility criteria

y y If a student is enrolled in Std 6 or lower, he/she is not eligible to participate in PISA. Record a code n for this student in column N/P PISA . If a student was NOT born between 01.01.1996 and 31.12.1996, he/she is not eligible to participate in PISA. Record a code n for this student in column N/P PISA .

Summary of non-participant codes

34. All codes to be used in column N/P PISA for non-participants are summarised in Table 7Table 7. If in doubt about which code to use, leave that column blank and insert a brief note in the Comments column. Table 7: Codes for Non-Participant Reason for non-participation Parent refusal Excluded on SEN (Special Education Needs) basis Now enrolled at another school Not enrolled in this school, and enrollment is unknown Does not meet PISA criteria for eligibility Code 2 3 4 5 n

Field Code Changed

Formatted: Font: No

Examples of using Non-Participant codes

35. Please refer to Figure 2Figure 2, in which the following example codes have been applied: y y M. Muthusamy has recently transferred to another school. Therefore, he is coded as 4 in column N/P PISA . Ahmad Amir is coded as 3 in column SEN . He has been taught in English for less than one year and his teachers believe that he will not be able to participate in the assessment. Therefore, he is coded as 3 in column N/P PISA . While Khaw Meng s experience with English is limited, he has been taught in English for more than one year and therefore does not meet the PISA criteria for exclusion on the basis of insufficient language experience. Therefore, column N/P PISA is left blank, and Khaw is considered to be able to participate in the assessment. M. Mahendran is blind and is coded as 1 in column SEN . PISA is not offered in Braille. Therefore, he is unable to participate in the assessment and is coded as 3 in column N/P PISA . Even though Lai Kwang is functionally disabled, it does not affect his ability to participate. So, although he is coded as 1 in column SEN , column N/P PISA is left blank. The following two students are ineligible according to the PISA eligibility criteria. However, their wrong demographic information was recorded in the List of Eligible Students, and these students were subsequently sampled. The School Co-ordinator found the incorrect information on the Student Tracking Form and corrected it: - Tammy Lim does not meet the age definition and is coded as n in column N/P PISA . - Sharifah Azman is in Grade 6 and so is coded as n in column N/P PISA .

Formatted: Font: No

Field Code Changed

y y

16

School Co-ordinator s Manual MS 12

V. Shahrukhkhan has been professionally assessed as cognitively delayed and is coded as 2 in column SEN . She does not participate in other testing programs. Therefore, she will not participate in the assessment and is coded as 3 in column N/P PISA . Note 3: Column N/P PISA of Student Tracking Form The purpose of setting a column N/P PISA is to record the reasons of non-participation for those students who have no possibility to participate in PISA under any circumstances even if a follow-up session is provided after the assessment. Therefore, you should NOT use a code 2 if a student is unable to attend the PISA assessment, for example, due to a competing activity in/outside the school or illness, as he/she might be able to attend a follow-up session.

Students to be assessed in PISA

36. Only those students who have NO codes placed in column N/P PISA can be assessed in PISA. Important! Under No circumstances may a sampled student be replaced with any other student.

7. Give School Questionnaire to appropriate person at your school

37. The School Questionnaire should be completed by the school principal, but she/he may delegate the task to another person in the school.

8. Organise and confirm assessment plans with school staff and with Test Administrator
38. During the two weeks preceding the assessment, you will need to confirm these plans: y y y y arranging for a technical support person to prepare computers for the Computer-Based Assessment; asking participating students to bring a: (a) pencil; (b) eraser; (c) ruler; and (d) calculator on the day of the assessment or preparing the above stationery at the school; obtaining a few spare items of the stationery listed above and some books or magazines for students who finish the assessment ahead of schedule; and informing those concerned of any change in the date or time of the assessment. In particular, it is important to advise EPRD, MOE, so that the Test Administrators and Quality Monitors can be advised.

39.

The Test Administrator assigned to your school should contact you approximately one week before the assessment date to confirm the updates of the Student Tracking Form and details of the assessment session.

9. Collect completed School Questionnaire

40. Please collect the completed Questionnaire A FEW DAYS BEFORE THE ASSESSMENT DAY. It will need to be returned to the Test Administrator on the day of the assessment.

17

School Co-ordinator s Manual MS 12

Stage III: During/After the Assessment

10. Provide Test Administrator with Student Tracking Form and School Questionnaire
41. Please make sure that you have updated the Student Tracking Form before the assessment. There might be some students who are eligible to participate in the assessment, but are absent from the assessment. For such cases, please leave column N/P PISA blank. The Test Administrator will record student attendance during the assessment, using a Session Attendance Form. The Test Administrator will collect your Student Tracking Form as well as the completed School Questionnaire to return to EPRD, MOE. You should make a photocopy of the Form for yourself to keep at your school. Please store that copy until the end of the school year.

42.

11. Be present throughout the assessment to assist Test Administrator (if possible)
43. If it is not possible for you to be present throughout the assessment, you should be present at least at the beginning of the assessment session. This will help set the tone of the session and demonstrate the importance of the study. Note 4: y Because PISA materials may be used in the future, their security is very important; y You and other school staff members are allowed to look through unused booklets only after the assessment, and in the presence of the Test Administrator; and y DO NOT photocopy or copy the test booklet and Computer-Based Assessment materials in any way under any circumstances.

12. Arrange for follow-up session (if needed)

44. PISA requires a high level of participation amongst the sampled students. If several students are absent from the Paper-Based PISA session or the Computer-Based Assessment session, it may therefore be necessary to arrange a follow-up session. The Test Administrator will guide you through the process of determining whether or not you will need to arrange a follow-up session. If you and the Test Administrator decide that a follow-up session should be arranged, please select a date and time that will maximize the number of students attending the session and that is convenient for the Test Administrator. Note 5: There can only be ONE follow-up session. Please make every attempt to ensure that absent students attend the follow-up session.

45.

13. Arrange assistance for Test Administrator for Computer-Based Assessment session
46. If the Paper-Based PISA and Computer-Based Assessment sessions will be conducted by different Test Administrators, please make sure that the Test Administrator for the Computer-Based Assessment will obtain from the other Test Administrator: y The Student Tracking Form to check the consistency; and y A copy of the Session Attendance Form used for the Paper-Based PISA. This will enable her/him to check that only students who were present for the Paper-Based PISA are assessed in the Computer-Based Assessment.

18

School Co-ordinator s Manual MS 12

Your help with PISA is very important to its success. EPRD, MOE and the International Project Team appreciate your time and effort. Thank you!

19

School Co-ordinator s Manual MS 12

Appendix 1: PISA 2012 Main Survey - Assessment Dates Form

To: From: Phone No.: School Name: Date: ( ) Dr Ihsan Ismail Fax or email: Fax No.: E-mail: State: 03-88846579 or ihsan.ismail@moe.gov.my ( )

Issues to consider when selecting assessment dates for your school

y The PISA will take place between 09.04.2012 and 11.05.2012; y The Paper-Based PISA session will take about 3 hours and 15 minutes, including breaks and time for distributing and collecting materials; y The Computer-Based Assessment session will take approximately 1 hour 30 minutes, including time to set-up the test area; y You are encouraged to schedule the Paper-Based PISA to commence early in the morning, and the Computer-Based Assessment to be carried out in the afternoon. The Computer-Based Assessment session should be conducted after the Paper-Based PISA; y EPRD, MOE will send a trained Test Administrator to the school to carry out both the Paper-Based PISA and the Computer-Based Assessment sessions; y For the Computer-Based Assessment session, it is advisable that a member of staff familiar with the school s computers and IT policies is available to assist the Test Administrator. Otherwise, you should be available for the beginning of the assessment and/or periodically; y Please check your School Calendar to ensure there are no other events that coincide with PISA (for example, tests/exams, sports festival or excursions); and y Please ensure you have made arrangements for the assessment location. For the Paper-Based PISA, the room should have a capacity for 35 sampled students sitting at single desks. For the ComputerBased Assessment, a single room able to accommodate 14 computers should be available.

Please nominate three alternative dates for the PISA assessment to take place at your school:
Preferences First Second Third Date Time of commencement

Please specify any dates which will NOT be suitable for testing:

Please e-mail, fax or post back this form to EPRD, MOE by 20.02.2012. Thank you!

20

School Co-ordinator s Manual MS 12

Appendix 2: PISA 2012 Main Survey - School's Computer System Diagnostic Report

Introduction
1. Fourteen (14) students in your school will participate in the Computer Based Assessment (CBA) as part of the PISA assessment. It is important to establish whether the computers students will use for the test have the required capabilities to run the CBA test successfully. 2. This manual provides instructions for the completion of a Systems Diagnostic (SD) which will test computer capabilities for running the CBA test. The information gained from this diagnostic will then enable the completion of the CBA Systems Diagnostic test results forms so that the number of suitable computers available in one room can be determined. This information will need to be provided to EPRD, MOE.

Running the Systems Diagnostic

3. The CBA Systems Diagnostic (SD) runs from a single USB drive. 4. The user must have an account with administrative rights to run the SD. This is explained further below in the section Reporting results (item d). 5. The operating system must be Windows XP, Windows Vista or Windows 7. 6. The SD provides a variety checks. As a minimum, the test input, basic check and virus scan of the USB MUST be completed. A virus scan of the local hard drive and an advanced check are also recommended but are not mandatory. Each test is described in this manual. 7. To run the SD, turn on the computer and close any applications running on the computer. 8. Insert the SD USB and open Windows Explorer. Locate and run the executable file in the root directory of the USB called PISAmenu.exe (e.g. by double-click on the file). 9. The following control panel will appear:

10. Click on the Start Diagnosis button and the following screen will appear:

21

School Co-ordinator s Manual MS 12

11. The top section of the screen About the System provides general information about the computer being tested. "OS" refers to the operating system. "Service Pack" is the latest installed service pack, if there is no service pack installed -- N/A -- is displayed. "User" is the user currently logged into the computer. "USB key size" displays the amount of free memory and total capacity of the USB drive. 12. The buttons on the right of the screen are the three different diagnostic tests that can be implemented: the basic check, the virus scan, the advanced check and a button for exiting the diagnostic. Please note that in the home edition of Windows XP the Advanced Check button will be disabled.

Basic check
13. Click on the Basic check button on the right of the screen. This check may take a few seconds to run. Below is an example of what will appear when the basic check is complete.

22

School Co-ordinator s Manual MS 12

23

School Co-ordinator s Manual MS 12

14. The following table indicates each type of basic check, what it is checked, the associated requirement for the CBA test and the possible messages that will appear in the centre section of the screen above:
Label CPU What it tests The speed of the CPU installed in MHz The operating system Requirement for CBA Must be at least 1500Mhz Possible message OK : CPU speed is "n" MHz OR Failed : CPU speed is "n" MHz Must be Windows XP, Windows Vista or Windows 7 OK : Operating system is "one of Win XP, Win Vista, Win 7" OR Failed : Operating system is "system other than Win XP, Win Vista, Win 7" System memory The memory installed on the computer in MB Required memory for Windows XP = 512 MB Required memory for Windows VISTA = 1024 MB Required memory for Windows 7 = 1024 MB Screen resolution The current screen resolution (width x height) Whether the current user is logged in as an administrator If Skype is running on the computer Minimal screen width = 1024 px Minimal screen height = 768 px Must be logged in as an administrator OK : System memory OR Failed : System memory (required "n" MB) OK : Screen : Width x Height OR Failed : Screen : Width x Height OK : Account with admin rights OR Failed : Account with admin rights required The CBA assessment cannot run with Skype running OK : Skype is not running OR Failed : Skype should not be running System s DPI setting The computer s current DPI setting The USB key data transfer rate in MB/s DPI setting must be 96 DPI (100%) OK : DPI setting is 96 DPI OR Failed : DPI setting is "n" DPI Required USB data transfer rate = 1 MB/s OK : Speed "n" MB/s OR Failed : Speed "n" MB/s OR Can't run the test - in cases where the program could not detect the transfer rate

OS

Account with admin rights

Skype running?

USB key data transfer rate

15. If all basic checks are highlighted in green and labelled OK, the computer is suitable for running the CBA assessment. If any appear in red, the computer may not be suitable.

24

School Co-ordinator s Manual MS 12

16. For each computer tested, please complete the test results form on page 29. Please refer to the section on Reporting Results for more information.

Field Code Changed

Virus scan
17. The virus scan can be used to check any drive on your computer. The scan must be completed on the USB drive and will take approximately 30 seconds. 18. It is recommended (but not mandatory) to also run the scan on the local hard drive. Please note that this may take between two and five hours for each computer. 19. To run the virus scan, click on the Virus Scan button on the bottom right of the Systems Diagnostic check screen. All local and network drives will appear as available for scanning. The following screen shot is an example of what will appear when 8 drives (including USB) are available for scanning.

20. To scan the USB drive, ensure that the correct drive is selected, then click on the Scan button at the bottom of the screen. The following screen will appear and may take approximately 30 seconds to run on the USB (or up to 5 hours for the local hard drive scan).

25

School Co-ordinator s Manual MS 12

21. Please note that while the virus scan is running the computers local antivirus software may detect the virus scan software being used for PISA (ClamWin Free Antivirus) as a virus. This is normal and should be ignored. Below is an example of the screen that appears when Trend Micro OfficeScan is the local antivirus software. Dialog screens and options will vary depending on the type of antivirus software installed locally.

22. In this example, the local virus scanner (Trend Micro Office Scan) has detected the Systems Diagnostic virus scanner (ClamWin Free Antivirus) as a virus. If this occurs click OK and the virus scan will continue. 23. The following screen is an example of what will appear when the virus scan is complete.

24. In this example the scan was successful and the computer is suitable for the CBA. It indicates no infected files were found.

26

School Co-ordinator s Manual MS 12

25. If, however, the scan indicates that there are infected files, you will need to clean the USB (or local hard drive) with professional antivirus software or return the USB to EPRD, MOE. Until the virus scan reveals that there are no infected files on the USB drive, the USB should not be used to run the CBA. 26. Click on the Close button to exit the virus scan.

Advanced check optional

27. The advanced check is a more detailed scan of the memory of the computer. By selecting this option, a text file is written to the USB that contains information regarding the computers memory. This may take a few minutes to complete. While the check is running, the cursor changes into an hourglass icon. It is not a requirement to run this check unless requested to do so by your national center who may then request some specific information from this text file.

Test input
28. Click on the Exit button and return to the control panel which has the Test input button as indicated in the screen shot below.

29. Click on the Test input button. An automatic virus scan of the memory will commence. This scan will take two or more minutes depending on the size of the computers memory. 30. If the software indicates that a virus is detected a popup window will appear informing you of this and asking whether you would like to proceed with the CBA. You may choose to proceed but it is recommended to clean the computer with professional antivirus software first. This reduces the likelihood that a virus will be transferred to the USB. 31. Once the virus scan is complete, enter the password: 123456789

27

School Co-ordinator s Manual MS 12

The following screen will appear:

32. If this page does not appear, press ALT TAB to return to the PISA 2012 Control Panel and click on the Exit button. Eject the USB from the computer and start the process of running the Test input check again. If the error continues to appear, re-start the computer before trying again. 33. This mock item is designed to test if the functionality of the CBA will work on the computer. Please complete the following: 1. 2. Drag and drop the box as indicated and record whether this function works or not. Enter text into the text box and record whether this function works or not.

34. Once complete press ALT TAB to access the Pisa 2012 Control Panel, click on the Exit button and remove the USB from the computer.

28

School Co-ordinator s Manual MS 12

Reporting results
35. For each computer tested, please complete the test results table below. Make photocopies as required. 36. Please then collate the information from all test results and determine how many computers are suitable in a single room for CBA testing. Then pass this information onto the <NC>.

CBA SYSTEMS DIAGNOSTIC TEST RESULTS FORM

BASIC CHECKS CPU OS System memory Screen resolution Account with admin rights Skype running? Systems DPI setting USB key data transfer rate VIRUS SCAN USB Drive Local Hard Drive ADVANCED CHECK TEST INPUT Drag and drop Enter text COMMENTS OK OK FAILED FAILED OK OK OK OK OK OK OK OK OK OK OK FAILED FAILED FAILED FAILED FAILED FAILED FAILED FAILED FAILED FAILED but corrected FAILED but corrected OK FAILED FAILED Not Applied

Not Applied

FAILED

IS THE COMPUTER SUITABLE?*

YES

NO

29

School Co-ordinator s Manual MS 12

* Instructions for establishing if the computer is suitable

Use the information in the summary section of the test results to establish if the computer is suitable for the CBA test: a. b. c. If all checks are OK (or Not Applied) then the computer is suitable for the CBA test. If CPU, OS or System memory failed the test, the computer is not suitable for the CBA test. If the screen resolution test fails, you may attempt to alter the screen resolution. To do this, go to Settings and then Control Panel from the Start menu. Open Display and go to the Settings tab. Ensure that the resolution is set to at least 1024 x 768 pixels. If this level of resolution is not available the computer is not suitable for running the CBA test. If this level of resolution is available, apply the new settings and run the basic check again. If the screen resolution test is ok, the computer is suitable with this setting; if it fails the computer is not suitable for the CBA test. If the account with admin rights test fails it may be possible to change the administration settings of the computer. To do this, go to Settings and then Control Panel from the Start menu. Open User Accounts and change the user group membership to Administrator. Note that this setting should be altered using the main administrator account for the computer. After adjusting the administrator settings on the computer, run the basic check again. If the account with admin rights test is ok, the computer is suitable with this setting, if it fails the CBA test can not be launched. For more detailed instructions on adjusting the administration rights on the computer, visit one of the following websites depending on your operating system.
Creating user accounts
http://www.microsoft.com/windowsxp/ using/setup/winxp/accounts.mspx http://windows.microsoft.com/enUS/windows-vista/Create-a-user-account http://windows.microsoft.com/enus/Windows7/Create-a-user-account

d.

Operating system Windows XP

Creating user account group membership

http://www.microsoft.com/resources/docume ntation/windows/xp/all/proddocs/enus/usercpl_change_group.mspx?mfr=true http://windows.microsoft.com/enUS/windows-vista/Change-a-users-accounttype http://windows.microsoft.com/enus/windows7/Change-a-users-account-type

Windows Vista

Windows 7

e.

If the Skype test fails, ensure that Skype is shut down on the local computer and run the basic check again. If the Skype test still fails the computer is not suitable for running the CBA test.

30

School Co-ordinator s Manual MS 12

f.

If the Systems DPI setting fails, it is possible to adjust these settings. The systems DPI must be set to 96 DPI (100%). For instructions on adjusting the systems DPI setting visit one of the following websites depending on your operating system.
Website for instructions for altering the system s DPI settings http://www.microsoft.com/windowsxp/using/setup/personalize/look.mspx http://windows.microoft.com/en-US/windows-vista/Make-the-text-on-yourscreen-larger-or-smaller http://windows.microsoft.com/en-us/windows7/Make-the-text-on-yourscreen-larger-or-smaller

Operating system Windows XP Windows Vista Windows 7

After making this adjustment, run the basic check again. If the Systems DPI setting is ok, the computer is suitable however if the test still fails, the computer is not suitable for the CBA test. g. If the USB Key data transfer rate test fails, the USB is not suitable for running the test and you will need to contact your <NC>. A USB that has a data transfer rate of 1 MB/s must be used.

Exit
37. Once the basic check, virus scan, advanced check and mock item check have been completed, please exit the SD by clicking on the Exit button on the main screen and remove the USB from the computer.

Return of materials
38. Once all computers have been tested using the Systems Diagnostic, please return the USBs to the national centre.

31

School Co-ordinator s Manual MS 12

Appendix 3: PISA 2012 Main Survey - List of Eligible Students

School Name: ___________________________________________________________________________ Address: _______________________________________________________________________________ Telephone: ________________________________ List Prepared By: ____________________________ Total Number of Students Listed: _______________ Fax: ______________________________________ Date List Prepared: __________________________

DIRECTIONS: Please list ALL students in your school in Form 1 or higher born between 01.01.1996 and 31.12.1996. (A) Student Name (B) Grade (C) Gender (Female=1; Male=2) (D) Birth Date (MM-YYYY) (E) Study Programme (F) * SEN

SEN = Special Education Needs

32