JITT just in time teaching is a teaching method that stimulates active learning and collaborative learning both inside

e and outside the classroom. The aim of JITT is to encourage student preparation for class by assigning them online activities related to the upcoming lecture topic. The students complete the assignment prior to arrival, or just in time for class participation. JITT also allows the instructors to gauge student understanding of the specific material before the class begins. The rationale is that students are accustomed to more passive classroom learning, and acquiring knowledge by having it passed from instructor to students. In order to have students participate more actively in class and enjoy the excitement of discovery, we should develop an online assignments designed around explorations of real-world data, and requires that they be completed before class. Implementation of JITT is simple and easy. Teachers start by figuring out what they want students to know and what the learning objectives are and then design questions to be as clear and straightforward as possible. If possible auto-graded online assignments is good to find which questions students find the most difficult, so that you can addresss them specifically in class. Interaction. JITT is an effective teaching method that engages students and reinforces what they have learned. Students are exposed to same topic several times-through preclass assignments, the teachers lecture, in-class discussion with peers, and preparation for quizzes after class. At the end of the class, challenge your students by asking then questions relating to covered topics. The students write their responses on 3 by 5 note cards, and earn credit for their participation. Their responses gives you a better understanding of what the students have learned, so that you can follow up on it in the next class.

JiTT techniques are consistent with research on effective teaching practices, in particular Chickering and Gamson's (1987) Seven Principles of Good Teaching, a widely-accepted benchmark of "best practices" for college teaching distilled from decades of research on undergraduate education. These Seven Principles remain useful guideposts for faculty interested in improving the effectiveness of their teaching. JiTT promotes each of Chickering and Gamson's Seven Principles, listed below:

1.

Increasing student-instructor (and student-student) contact

Hide JiTT enhances connections between students and instructors, both through personal feedback on JiTT responses and classroom activities that are informed by those responses. Students see an immediate link between their out-of-class JiTT exercises and follow-up in-class activities, which are developed "just-in-time" to target student learning challenges made visible

in students' JiTT responses. Interactive, cooperative-learning-based in-class activities promote student-student contact and provide students with additional opportunities for formative assessment of their learning. Novak, et al. (1999) report a more personal and intimate bond between instructors and students and an increase in communication from students in JiTT courses. 2. Encouraging active learning

Hide JiTT exercises are designed so that students must "do something" beyond simply reading the textbook in order to answer JiTT questions. Typically, these questions require higher-order thinking skills, such as analysis, problem-solving, and application, that go beyond mere familiarity and memorization.

3.

Encouraging cooperative learning Hide

In-class follow-up exercises based on students' JiTT responses emphasize cooperative learning strategies whenever possible to promote active engagement with course concepts and provide additional opportunities for formative assessment and monitoring of student learning. 4. Providing prompt feedback

Hide JiTT promotes real-time feedback for both students and instructors on student learning challenges. Students' JiTT responses are used in class as the basis for in-class discussion and follow-up small-group activities. Both directly target learning gaps highlighted in the JiTT responses. As a result, students get immediate and frequent feedback on their learning in the course. 5. Encouraging time on task

Hide JiTT exercises intentionally structure and focus out-of-class studying and encourage regular interaction with course materials. Additionally, because JiTT exercises are relatively brief, they help to break down the learning process into manageable chunks of time. 6. Communicating high expectations

Hide JiTT exercises signal high expectations and model good learning processes by prompting students to actively engage with course materials, process new information, and connect it in meaningful ways to their prior knowledge. JiTT exercises generally require students to practice higher-order thinking and problem-solving skills in a variety of settings. Consequently, students not

only learn course content, but also develop transferable learning skills that promote learning in other contexts. 7. Providing a variety of teaching styles to increase the learning effectiveness for students with diverse different learning styles. Hide While learning sciences research provides little empirical support for the concept of "learning styles," one clear benefit of JiTT is that it makes student learning processes visible for instructors. As a result, in-class instruction can specifically target student learning challenges and provide multiple pathways for learning. JiTT provides a scaffold to help students structure their learning, link new information to previous knowledge, and practice new skills and concepts through "hands on, minds on" learning in the classroom.

In particular, How People Learn highlights five key factors related to improving student learning. JiTT pedagogy is especially well-suited to incorporate these factors into your teaching practices. 1. Understanding students' pre/misconceptions

Hide Students don't come into our classes as blank slates; their current mental models, developed in a variety of ways, affect what they learn. As a result, what students learn is often different from what we're teaching. Effective JiTT exercises uncover student pre/misconceptions, which then can be directly addressed in the follow-up class session through hands-on, interactive activities that help students discover inconsistencies in their thinking processes. 2. Developing expert-like performance and learning

Hide Experts use structured mental models to order and categorize new information; novices tend to focus on surface properties and often fail to develop the structured mental models that characterize expert learning. JiTT exercises make visible to instructors naive, novice-like thinking processes in time to develop follow-up in-class activities that help to scaffold expert-like thinking. 3. Encouraging the development of transferable knowledge

Hide For students to develop the ability to transfer knowledge to new situations they must be given the opportunity to apply new knowledge in a variety of

situations and contexts. This takes repeated practice, guided learning, and feedback "about the degree to which they know when, where, and how to use the knowledge they are learning." (p. 47) JiTT exercises should provide students with opportunities to apply course concepts in a variety of new ways, helping to build transferable knowledge. Carefully constructed in-class activities provide additional opportunities to practice key course concepts, focusing on areas of conceptual challenge highlighted in students' JiTT responses.

4.

Emphasizing formative assessment Hide

Assessment is critical to provide students with feedback on their learning; assessment should include both content knowledge and process knowledge (e.g. how students solve problems). Effective JiTT exercises require students to reflect on both course content and the processes needed to solve problems, carry out critical analyses, evaluate information, and synthesize ideas, all higher-order thinking skills. In-class, follow-up cooperative learning activities provide additional opportunities for students to obtain immediate feedback on their learning processes. 5. Helping students become reflective (metacognitive) learners.

Hide To promote life-long learning, we need to help students become selfdirected, reflective learners. This requires directed practice that encourages student reflection about the learning process and how learning is being achieved. JiTT exercises typically include at least one question asking students to reflect on learning gaps uncovered in their pre-class preparation; in turn, instructors can use this knowledge to assist students in monitoring and assessing their own learning and seeking assistance to close those gaps when needed. Learn more about how to foster metacognition in students.

Benefits for Students

This is what students have to say about JiTT:

... it made me read on my own rather than wait for you to teach me.

The exercises allowed you to become familiar with the material so that you would be able to participate in class discussions. It made me better disciplined.

... I also liked the fact that the JiTT assignments were used along with class discussion. Even if you got the JiTT wrong you were still able to learn through the discussion.

JiTT supports the effective teaching practices outlined by Chickering and Gamson (1987) and the research on how students learn [see, for example, Bransford, Brown, and Cocking (1999, 2000)] in the following ways [see also Research on JiTT]:

Improving Pre-Class Preparation

Structuring out-of-class learning to increase time on task- JiTT exercises are designed to focus student attention, scaffold effective learning processes, promote interactive engagement with course material, and encourage reflective learning. All of these behaviors are positively associated with effective learning.

Increasing motivation for out-of-class preparation- Students see a direct connection between their out-of-class effort and in-class instruction. Students' JiTT responses inform classroom activities, which are focused on the learning challenges highlighted in their responses, building a positive feedback loop that encourages completion of future JiTT exercises and greater student-faculty interaction. In short, students see that their out-ofclass efforts make a real difference in what happens in class, which in turn is focused on addressing their most important learning challenges.

Improving In-Class Learning

Increasing learning in the classroom- Students who come to class prepared are able to learn more and contribute more to others' learning in the classroom. JiTT exercises require students to interact with course material prior to the upcoming class, increasing the efficiency and effectiveness of classroom activities.

Providing frequent and immediate feedback on learning- JiTT makes learning gaps visible to both instructors and students through review and posting of students' JiTT responses in class. Follow-up in-class exercises based on these responses provide additional opportunities for students to obtain feedback on their thinking processes and gauge their level of understanding relative to their peers.

Confronting mis/preconceptions- JiTT exercises are most helpful when they make visible students' current mental models and thinking processes. Often these thought processes inhibit student learning and lead to the all-too-common result that what students are learning is not the same as what we are teaching. Once student pre/misconceptions are identified, inclass exercises can be developed that lead students to contradictory outcomes, providing an opportunity for effective knowledge-building

(see Interactive Lecture Demonstrations for examples of how to develop exercises that confront student preconceptions).

Promoting Long-term Learning

Developing metacognitive skills- Including a JiTT question such as "After completing this assignment, I am still unsure of..." helps students to develop reflective skills that promote self-directed and self-monitoring learning, skills critical for advanced and life-long learning.

Improving transfer of knowledge- Effective JiTT exercises require students to explain not only their answers, but also the thought process that led them to those answers. By focusing on thinking processes as well as content, students begin to develop transferable thinking skills that they can apply in a variety of settings and to problems that they have not encountered previously.

Linking new information and concepts to prior knowledge and real-world issues- JiTT exercises are often most effective when students are asked to relate new course material and concepts to prior experiences and relevant real-world issues and events. This helps to create a knowledgemaking bridge that promotes durable, long-term learning.

Benefits for Instructors

Read an interview with JiTT pioneer Gregor Novak on how JiTT provides instructors with valuable insights on student understanding of course concepts prior to class. Also, seeResearch on JiTT. While JiTT provides a variety of benefits for students and their learning processes, it also benefits faculty members, improving both the practice and enjoyment of teaching. In particular, JiTT provides a window into student learning processes- Student responses to JiTT exercises provide a never-ending stream of surprises about student thinking processes that would otherwise remain hidden until an exam, quiz, or homework assignment. Making these learning processes visible allows us to develop in-class activities aimed at improving those skills.

JiTT is a flexible teaching technique- JiTT can be implemented incrementally and in conjunction with current teaching practices; used in any discipline and any course level, including both undergraduate and graduatelevel courses; and combined with other pedagogical innovations.

JiTT increases teaching efficiency and effectiveness- Instructors are often concerned that JiTT will take more time relative to lecturing, leading to less content coverage. On the contrary, using JiTT exercises to determine areas where students are having the most difficulty means that you can focus classroom attention where it has the biggest impact rather

than using a "shotgun approach" that may waste students' time and reduce motivation. JiTT improves student preparation for class- At the top of most instructors' list of "teaching problems" is insufficient student preparation for class. JiTT provides a positive incentive for students to complete JiTT exercises, knowing that their responses will be used to inform the structure and activities of the next class. Students in JiTT-based courses come to class better prepared, which in turn increases the level of in-class engagement and learning, whether in a small seminar course or a large lecture-based course.

JiTT transforms the classroom- Starting a class by showing a sample of JiTT responses completely changes the classroom learning environment. Students love to see their work used as the basis for the day's activities, whether a classroom discussion or small-group, hands-on activities. In either case, students are engaged and ready to learn, using their own words and thought-processes as starting points.

How to Use Just-in-Time Teaching

Just-in-Time Teaching (JiTT) intentionally links students' responses to out-ofclass web-based questions with in-class activities. To begin, students answer a small set of web-based questions on upcoming course material outside of class and submit their responses online a few hours before class begins. Once submitted, the instructor reviews the students' JiTT responses and develops inclass active-learning

exercises targeting learning gaps identified in the JiTT responses - "just in time" for class.

A Step-by-Step Approach to Implementing JiTT in your Course

Because of its flexibility, Just-in-Time Teaching (JiTT) can be applied to a variety of assignments, courses, and disciplines, and can be easily combined with other

pedagogies. JiTT is an excellent way to get students prepared for class, but it can also be used to uncover student misconceptions, provide examples for classroom activities, extend textbook/classroom learning, and develop disciplinary skills. Follow the steps below to get started using JiTT in your courses.

STEP 1: Getting Started with JiTT - Pre-planning is the Key to JiTT Success
Some questions that you should consider before actually implementing JiTT in your course: In what course(s) do you want to use JiTT? What are the characteristics of those courses (large or small classes, introductory or higher-level, theoretical or applied, discussion or lecture-oriented)?

What do you want your students to be able to do and know at the end of your course? That is, what learning goals do you have for this course, and how will JiTT help you and your students more effectively achieve those goals?

How often will you use JiTT exercises? What percentage of the course grade will JiTT account for?

How will you use students' JiTT responses in class? What activities and teaching strategies will you use as a follow-up to students' JiTT responses? How will you prepare students for completing JiTT exercises?

For classroom-based advice on the questions above...

STEP 2: Developing Effective JiTT Questions

The key to achieving success with JiTT is developing effective JiTT questions. Making decisions about JiTT questions is somewhat different than writing homework or exam questions because the intent is not to determine whether the students have mastered the subject. Rather, the purpose is to elicit a rich set of responses that illustrate students' thinking processes and inform follow-up inclass activities.

As a result, good JiTT questions are typically open-ended and leave room for multiple explanations and interpretations; often, they ask students to apply new concepts or ideas in ways that cannot simply be looked up in a textbook. From a pragmatic standpoint, JiTT questions should focus on key ideas to be discussed in the upcoming class and allow students to answer with relatively short responses so that you'll have time to read them prior to class. Effective JiTT questions align with student learning goals for your course, facilitate higher-order thinking skills, and are grounded in learning sciences research.
For detailed guidance on developing effective JiTT questions, including examples...

STEP 3: Reviewing Student Responses

Timeline for Reviewing Student Responses
You will need time to read and process student responses to the Just-in-Time Teaching questions you've posed before the class period in which you will use their answers. How much time that takes will depend on the complexity of the questions, how many students are in the class, and how quickly you can process the information.

Using Student Responses to Inform In-Class Activities

The most immediate task is to use student responses to inform the activities you will include in your next class. You don't know how you're going to use students' JiTT responses until you read them. Typically there are patterns in the responses that allow you to cluster those responses in groups. These "response clusters" provide valuable insights that can be used to develop in-class activities that directly target learning gaps highlighted in the JiTT responses.
For additional information on reviewing students' JiTT responses...

STEP 4: Implementing In-Class Follow-up Activities

Implementing JiTT in the classroom can be as simple as showing a sample of students' responses (anonymously) at the front of the room and following up with classroom discussion asking students to point out incomplete or incorrect thought processes, expand on submitted responses, or extend the highlighted concept. However, JiTT is particularly effective when student responses are used to develop interactive, cooperative-learning exercises that target learning gaps made visible in students' responses or extend the concepts included in the exercise. The student responses can be used directly to create small-group exercises ("Which one of these is right? Why?" or "Use the sample of responses shown here to develop a new, comprehensive response.") or used to inform both

the form and practice of the activity. Maintaining a close linkage between out-of-class responses and in-class activities provides motivation for students to complete JiTT exercises, supports ongoing formative assessment in the classroom, and promotes learning while concepts are fresh in students' minds.
For ideas on how to develop and implement JiTT-based in-class activities...

Pre-Planning: Getting Started using JiTT in the Classroom

Before implementing Just-in-Time Teaching in your course, it's helpful to think through responses to the questions below. We've included some some tips based on our classroom experience, but you'll need to consider your own teaching style, classroom setting, use of technology, and the like before jumping in. In addition, you'll need to think about what changes using JiTT might make in your own teaching practices and how you will communicate the role of JiTT in your class to your students.

What do you want your students to be able to do and know at the end of your course?
Before you implement JiTT, determine the student learning outcomes you want to achieve and how JiTT can help you and your students more effectively achieve those outcomes. Once you decide on what learning outcomes you want to emphasize, intentionally link JiTT questions to these outcomes. For more information on creating effective JiTT questions linked to learning outcomes...

What instructional technology tools will you use to implement JiTT?

JiTT is easily integrated in most standard web-based course/learning management systems such as Blackboard, Moodle, and Sakai (among others) that you and your students are already using, eliminating the need for learning any additional instructional technology tools. All include options for both openended and multiple-choice assessments, which can be made available to students for specific periods of time. Using learning management systems also makes it easy for you to both review and grade student responses, cut-andpaste student responses for viewing in class, or make comments directly to students online.

In what course(s) do I want to use JiTT?

What are the characteristics of those courses in which you plan to implement JiTT? Are they large or small classes, introductory or higher-level, theoretical or applied, discussion or lecture-oriented? JiTT can easily be implemented in any type of course, but how you implement JiTT, both in the types of questions you use and the way that you use student responses in class will depend on your answer to these questions. No matter the course, JiTT will require some additional time, especially the first time you use it, but the increase in student engagement is generally worth it.

What other activities do you plan to include in your course?

JiTT does increase the workload, both for students and instructors, so you need to think about what other activities you also include in your course. Overloading either your students or yourself will defeat the purpose of incorporating JiTT in the first place. Including JiTT may mean having to leave some other activities out, or you might integrate JiTT into existing activities to increase their learning potential.

What activities and teaching strategies will you use as a follow-up to students' JiTT responses?
JiTT is especially effective when combined with in-class hands-on, active-learning activities carried out in small groups using structured cooperative learning techniques. Students' JiTT responses provide a context or focus for inclass activities and teaching practices that actively involve students in the learning process. For more information on effective in-class activities and teaching strategies to use with JiTT...

How often will you use JiTT exercises?

Ultimately, you may want to use JiTT exercises every week, or possibly even more often. But when you first implement JiTT in a course, you may wish to build exercises in gradually. You can begin with a few JiTT exercises in one semester and add more each time you teach the course. In time, as you add more JiTT exercises in, you will see reduced course preparation work, the ability to grade more efficiently, and greater facility with using student responses to develop effective, student-centered in-class activities. JiTT works best when it is used regularly throughout the semester to provide systematic feedback to students on their learning and included as a formal part of course assessment.

Can JiTT be used in large-enrollment courses?

JiTT is a valuable addition to large enrollment lecture-based courses. JiTT exercises provide prompt feedback about student learning, feedback that is difficult to achieve with other teaching methods, especially in large classes. Moreover, JiTT can be combined with other pedagogical innovations such as Peer Instruction (see references on Peer Instruction) to promote interactive student engagement, even in classes with hundreds of students. For more information on using JiTT with other pedagogies in large lecture courses...

What percentage of the course grade will JiTT account for?

Experience has shown that JiTT works best when it counts as part of the course grade. But how much? Making JiTT count for around 10% of the course grade is typically enough to promote student completion of JiTT exercises while maintaining its primary function as a formative assessment tool. Some faculty members have made JiTT count for as much as 30% of the course grade, however. Of course, the percentage of the final grade that comes from JiTT exercises should be proportional to the number of them that you use. Taking into account that JiTT questions focus on material that has not yet been covered in class, grading is based largely on effort. Sharing the rubric used to evaluate JiTT responses with students will help them to better understand this formative assessment process.

For more information on assessing JiTT exercises...

How will you prepare students for completing JiTT exercises?

Most students will be unfamiliar with the JiTT process and may initially view JiTT exercises as "busy work." Explaining the role of JiTT as an effective, studentcentered teaching/learning tool in your course syllabus will help students better understand their active role in the learning process. Once understood, students generally view JiTT quite favorably and wish that other instructors also used this teaching technique.

What do students say about JiTT?

Developing Effective Questions for JiTT Exercises

The key to achieving success with JiTT is developing effective JiTT questions. Good JiTT questions are typically open-ended and leave room for multiple explanations and interpretations; often, they ask students to apply new concepts or ideas in ways that cannot simply be looked up in a textbook. From a pragmatic standpoint, JiTT questions should focus on key ideas to be discussed in the upcoming class and align with student learning goals for the course.

Some Practical Considerations

Before you write your questions, ask yourself: "What do I want my students to know, understand, apply, analyze, synthesize, or evaluate prior to class?" In answering this question, it is useful to work backwards from the in-class activity you plan to use. In other words, what do your students need to have thought about or done in order to be fully prepared for that activity?

JiTT questions should be relatively brief and take students only 15-30 minutes to answer.

The more involved the problems, the fewer questions should be given to the students. Three to four questions is a typical number, and sometimes they include both multiple choice and short response questions. It is useful to include a question asking what was most important, interesting, or confusing about the reading. Responses to these questions help to target in-class teaching and can jump-start engaging classroom discussions.

Linking JiTT Questions to Course Learning Objectives

Starting with the question "At the end of this course, what do I want my students to know and be able to do?" focuses student and instructor attention on the most important concepts, ideas, and skills in the course, reinforcing course and program goals. For example, do you want to use JiTT exercises to:

develop and extend students' critical and analytical thinking skills?

improve students' quantitative reasoning skills?

increase students' facility using multiple representations (verbal, graphical, quantitative, and/or analytical) of models? scaffold disciplinary thinking processes e.g. the use of evidence in making causal claims? enhance students' problem-solving skills?

The course learning objectives you have for your course will determine what kinds of JiTT questions will be most effective in achieving those objectives.

Linking JiTT Questions to Bloom's Taxonomy

How can JiTT exercises be used to move students from one level of Bloom's taxonomy to higher levels?

Before you write your questions, ask yourself: "What do I want my students to know, understand, apply, analyze, synthesize, or evaluate prior to class?" You may very well want to work backwards from the in-class activity you plan to use. In other words, what do your students need to have thought about or done in order to be fully prepared for that activity?

Knowledge and Comprehension Show JiTT examples promoting knowledge and comprehension

Application and Analysis Show JiTT examples promoting application and analysis

Synthesis and Evaluation Show JiTT examples promoting synthesis and evaluation

Making use of Learning Sciences Research

Learning sciences research suggests the importance of helping students uncover (and confront) pre/misconceptions, develop expert-like thinking processes, transfer knowledge to new, unfamiliar situations, and build metacognitive skills. Intentionally linking JiTT questions to the principles summarized in How People Learn (2000) and Angelo and Cross (1993) increases the impact of JiTT pedagogy on student learning outcomes. Making visible student pre/misconceptions about important course concepts and topics

Show JiTT examples aimed at making student misconceptions visible

Developing expert-like thinking processes Show JiTT examples aimed at developing expert-like thinking

Improving transfer of knowledge from one learning environment to another

Show JiTT examples aimed at improving transfer of knowledge

Promoting self-regulated, reflective learning Show JiTT examples aimed at promoting reflective learning

Using Questions Developed by Others ConcepTests are conceptual multiple-choice questions that are used for in-class formative assessment, often incorporating classroom response systems and used in conjunction withPeer Instruction (references). ConcepTests often make good questions for JiTT exercises.

Reviewing and Assessing Student Responses

You will need time to read and process student responses to Just-in-Time Teaching questions before the upcoming class period. How much time that takes will depend on the complexity of the questions, how many students are in the class, and how quickly you can process the information. Most instructors require students to submit responses a few hours prior to class, or perhaps the evening before, especially for an early morning class.

Looking for Patterns to Inform In-Class Exercise Development

After students submit their JiTT responses you will need to review them soon thereafter, looking for misconceptions, incorrect reasoning processes, or other learning gaps that can inform the activities you will include in your next class. As you go through the responses you will most likely find clusters of responses that highlight similar learning challenges - some you were not even aware of! Select representative responses from each of these clusters to show (anonymously) at the start of the next class and to use in developing related hands-on in-class learning activities. It is useful to start the class period with a discussion of the representative responses as a warm-up for these in-class activities aimed at improving students' understanding of the concepts included in the JiTT exercise. The short time span between student-response submission and in-class activities developed from those responses is what gives Just-in-Time Teaching its sense of immediacy. In general, by the time you begin reviewing students' JiTT responses you will already have done most of the planning for your upcoming class period. Students' JiTT responses will inform your decisions about what activities to include, how much time to spend on them, and how to involve all students in the learning process. For example, if you see a pattern of misconceptions or a lack of understanding of a key concept as you read their answers, you should plan active-learning activities that directly address those topics. This can be done in a variety of ways - showing sample student responses representing conflicting ideas at the start of class and using them to lead whole-class or small-group discussions, asking student groups to expand on incomplete answers, or designing activities that directly confront students with the learning gaps highlighted in their responses.

Evaluating Students' Responses - Grading for Effort

JiTT works best when the exercises are explicitly included as a percentage of students' course grades, typically at least 10%. Instructors use a variety of methods to assign credit for JiTT responses. However, given that JiTT exercises refer to material not yet covered in class, responses to JiTT questions are generally graded more on the level of effort than on the accuracy of the answer. A JiTT Rubric Kathy Marrs (IUPUI) has developed a 4-point rubric for scoring JiTT responses adapted from de Caprariis et al., 2001 - and illustrated below - that blends both correctness and effort in its criteria. Using a rubric (that is shared with students ahead of time) greatly speeds the grading process and makes JiTT assessment

clear and transparent to students. You can grade the responses while reviewing them or wait until after the upcoming class - but the grades should be posted as soon as possible to provide prompt feedback to students on their performance.

Ideas for Managing JiTT Grading in Large Classes If you have very large classes, it may be difficult for you to grade all of your students' responses for every JiTT exercise. There are several ways to decrease the amount of grading you do, while maintaining the number of JiTT exercises you use: You can grade only a random selection of the JiTT exercises you assign during the course.

You can grade responses from a random selection of students for each exercise. You can award grades for completion, rather than accuracy of answers.

Using JiTT Responses to Develop Classroom Activities

Students' JiTT responses are used to develop structured interactive classroom activities for the next class. These activities can be simple or complex but should focus on student learning gaps highlighted in students' JiTT responses. Handson, interactive, cooperative-learning activities that intentionally address student learning challenges work best.

Designing Follow-up Classroom Activities

When you know that most of your students have completed the required reading, reflected on it, and answered questions about it, your options for what to do during class expand exponentially. The language used in class to present and discuss the content comes directly from the student responses. Each class session is unique because the students in each class are unique. Some Practical Advice: Student responses will usually fall into a fairly well-defined set of categories. Before going to class, select representative examples for class discussion from the full set of responses. Make sure that all the students get their day in class.

Revise the lesson flow now that you have the actual responses. The lesson does not have to be elaborately written out. Just a flow of ideas is usually sufficient. The fact that the wording actually comes from the class makes the lesson fresh and interesting to students.

Go to class and be ready to improvise if necessary. The lesson flow is pretty much predetermined, but the words used in class will flow from the student responses and, most importantly, will be influenced by the feedback from the live class.

A Few Possibilities to Consider:

Discussion If student answers to one or more of your questions show differences of opinion, you can capitalize on those differences to spark a discussion. Make a slide or overhead showing two of the contrasting answers, project them, and ask your

students to comment on them. Seeing one anothers' responses - and asking students to analyze them - helps to sharpen learning and communication skills. If comfortable, ask the author of a particular response to defend the response. The class can then be asked to respond. If several alternatives emerge the activity can move into a peer-instruction mode where students vote on the alternatives, converse in small groups and re-vote. Classroom response system technology (clickers) can be very effective here. You can use the contrasting answers for a think-pair-share exercise, mediated through the use of clickers.

See also: Ways to use Socratic Questioning in your courses

Problem-Solving If the topic of the day is pertinent to a real-world problem, ask students to analyze real data or use an investigative case study. Tying course material to real-world problems helps students to see the relevance of what they're learning and increases their interest in the course.

See also: Ways to use Coached Problem Solving in your courses See also: Ways to use Guided Discovery Problems in your courses

Role-Playing If the topic of the day lends itself to a role-playing activity, use one. In a small class, assign each student or small groups of students to particular roles. In a large class you can have students form small groups with each playing a particular role or you can ask for volunteers to take on roles in front of the class.

See also: Ways to use Role Playing Exercises in your courses

While the examples above can be carried out in a variety of ways, JiTT is most effective when implemented using structured cooperative learning techniques.

Combining JiTT with Other Pedagogies

One of the greatest strengths of Just-in-Time Teaching is its flexibility as a teaching tool. JiTT exercises can easily be combined with a variety of innovative, student-centered teaching practices, including: Classroom Experiments - JiTT exercises can be used to generate predictions ahead of the actual in-class experiments. Committing to a prediction prior to an experiment has been shown to improve learning in physics (Crouch, Fagen, Callan, and Mazur, 2004).

Cooperative Learning - JiTT is particularly effective when combined with in-class cooperative learning exercises. Student responses to JiTT questions provide a context for in-class cooperative learning exercises such as think-pair-share, jigsaw, or send-a-problem.

Peer Instruction - Harvard physicist Eric Mazur uses JiTT to uncover student pre/misconceptions, using student responses to generate in-class ConcepTest questions that students answer in pairs using personal response systems.
o o o

References on Peer Instruction

YouTube videos of Peer Instruction presentations by Eric Mazur: Long (80:09) | Short(2:22) Blog post by Derek Bruff on Peer Instruction Case Method - Responses to JiTT questions provide information to the instructor on areas of confusion or misunderstanding prior to class. This information can be used to develop cases designed to address the confusion or misunderstanding. Context Rich Problems - JiTT exercises can be used to help instructors better understand students' learning gaps so that in-class context-rich problems can be developed that directly address those gaps. Alternatively, context-rich problems can be used as JiTT exercises to build students' problem-solving skills.

Interactive Lecture Demonstrations - Student responses to JiTT questions can help uncover student mis/preconceptions, which can then be addressed via in-class interactive lecture demonstrations. In addition, i interactive lecture demonstrations can be adapted for use as JiTT exercises.

Example JiTT WarmUp Exercises

Help

Refine the Results

Earth System Topics
Atmosphere 2 matches Biosphere 10 matches Climate 2 matches Evolution 4 matches Geography 1 match Human Dimensions 2 matches Hydrology 3 matches Oceans 2 matches Solar System and Astronomy 1 match Solid Earth 4 matches Time/Earth History 1 match

Results 1 - 10 of 16 matches

JiTT - Dam Removal - A Good Idea or Not? part of Examples 1) What are some of the biological effects of dam removal (good and bad)? 2) What are some of the more pressing/compelling reasons to remove a dam? Explain. 3) The Stanley and Doyle (2003) article states that, ... JiTT - Ethics of Fossil Collecting part of Examples 1) What do you think it means for a fossil resource to be "abused"? 2) What's the issue with fossil hunting on federal land (such as National Parks)? Explain what your interpretation of the conflict ... JiTT - La Brea Tar Pits part of Examples 1) What is "tar" and how does it form? 2) List the animals that have been uncovered in the tar pits that you didn't know were native to North America. Why do you think these animals are now extinct? ... JiTT - The Big 5 Extinctions and Then Some part of Examples 1) What are the three leading ideas for the cause of the Permian mass extinction? What is the evidence for and against each? 2) Why are tropical forest species going extinct the quickest? 3) What are the ... JiTT - Marine Archaeology and Technology part of Examples 1) How has technology helped marine archaeological investigations? How has it hurt? 2) What are some of the "costs" involved with using technology for marine archaeology? Are the "costs" worth ... JiTT - The Future of Africa's Health with Technology part of Examples 1) What are the benefits and opportunities handheld technology can offer the health sector in Africa? Answer the question from the viewpoint of a health care provider and a patient. Does anyone else benefit? ... JiTT - Are Primates Worth Saving? part of Examples 1) What are some of the reasons extinction rates are so high for modern primates? 2) Why are some primates worth more dead than alive? Is there a way we as a society can change this? Explain your thoughts. 3) If ... JiTT - Life on the Moon and Mars part of Examples 1) Based on "Microbial Colony in US..." article, what are Archaea? 2) From the same article, which of the following is NOT evidence to support life on Mars? a) Mars' subsurface groundwater system b) ... JiTT - Darwin Garden part of Examples 1) Why does biodiversity in an ecosystem "matter"? 2) Describe Darwin's garden experiment and the significance of it. 3) So Darwin figured out the connection between biodiversity and ecosystem health ... JiTT - The Legs of Snakes and Whales part of Examples 1) What are the adaptations needed to move from water to land (whether it be an arthropod or a "pioneering amphibian")? 2) Could snakes be linked to marine lizards? What your view and interpretations of ...

Example JiTT WarmUp Exercises

Help

Refine the Results

Earth System Topics
Atmosphere 2 matches Biosphere 10 matches Climate 2 matches Evolution 4 matches Geography 1 match Human Dimensions 2 matches Hydrology 3 matches Oceans 2 matches Solar System and Astronomy 1 match Solid Earth 4 matches Time/Earth History 1 match

Results 11 - 16 of 16 matches

JiTT - Threats to Biodiversity part of Examples 1) How is climate change a threat to biodiversity? 2) What are the impacts of pesticides on animals (including insects) and humans? 3) Can human population growth really impact biodiversity? Explain your viewpoint. JiTT - Life in the Sahara Desert part of Examples 1) What the Sahara always a desert? How do we know? (*NOTE: for this question, I want you to focus on the physical environment) 2) What modern-day animals can be found in the desert? How do they survive? 3) Why ... JiTT - When "Modern" Human Behavior Appeared in Early Hominids part of
Examples

1) What is this controversy in regards to the emergence of "modern" human behavior? In your opinion, what do you think "modern" human behavior means? 2) How do beads symbolize modern human ... JiTT - Groundwater and Archaeology part of Examples 1) What is causing the groundwater to rise to the foundations of Egypt's archaeological structures? What damage is the groundwater doing? 2) Describe at least two different solutions that have been proposed to ... JiTT - Water Issues and the Aswan High Dam part of Examples 1) What are some of the GOOD changes for the environment and positive impacts on the local people from construction of the Aswan High Dam? 2) What are some of the BAD changes for the environment and negative ... JiTT - The Future of Global Climate part of Examples 1) According to NASA, why is Earth's climate warming? 2) Is the Geritol solution the solution to global warming? Explain what the "Geritol solution" is, and then state your opinion as to why you ...

What is Socratic Questioning

Named for Socrates (ca. 470-399 B. C.), the early Greek philosopher/teacher, a Socratic approach to teaching is based on the practice of disciplined, rigorously thoughtful dialogue. The instructor professes ignorance of the topic under discussion in order to elicit engaged dialogue with students. Socrates was convinced that disciplined practice of thoughtful questioning enables the scholar/student to examine ideas logically and to be able to determine the validity of those ideas. Also known as the dialectical approach, this type of questioning can correct misconceptions and lead to reliable knowledge construction. Although "Socratic questioning" appears simple, it is in fact intensely rigorous. As described in the writings of Plato, a student of Socrates, the teacher feigns ignorance about a given subject in order to acquire another person's fullest possible knowledge of the topic. Individuals have the capacity to recognize contradictions, so Socrates assumed that incomplete or inaccurate ideas would be corrected during the process of disciplined questioning, and hence would lead to progressively greater truth and accuracy.

An Example of the Socratic Approach to Teaching

Named for the early Greek philosopher/teacher Socrates, a Socratic approach to teaching is one in which the instructor poses thoughtful questions to help students learn.

Short Example of Socratic Questioning

The following series of questions and hypothetical answers illustrates how a teacher might engage students in Socratic questioning during class. This particular example on plate tectonics, faulting, and earthquakes is designed for an introductory Earth sciences course in which the students have some background to the topic, e.g., through lectures and/or text readings.

The image to the left illustrates the San Andreas fault and its plate tectonic setting along the coast of western North America. Show it to students, then call on various students to answer the following questions: Q. What is an essential feature of this fault? Draw upon material from course readings, lectures, or other sources of information. A. The plates are moving past one another about 5 cm/yr. Q. How do we know that the relative plate velocity is 5 cm/yr? A. Scientists can measure the motion with satellites. Q. Could an earthquake occur along the northernmost San Andreas fault, from San Francisco northward to the fault's termination? A. Yes. Q. What is your reasoning for saying an earthquake could occur? A. A large earthquake occurred in San Francisco in 1906. Q. What assumption are you making, if you state that earthquakes can occur in the future? A. If an earthquake already occurred, then it could happen again. Q. Is that a reasonable assumption, and why or why not? A. Yes, because if the tectonic plates are moving, then earthquakes must occur to accommodate that motion, so they're not likely to stop now, unless the plates stop moving.

Why Use Socratic Questioning?

Socratic questioning helps students to think critically by focusing explicitly on the process of thinking. During disciplined, carefully structured questioning, students must slow down and examine their own thinking processes (i.e., reflective thinking). Thoughtful, disciplined questioning in the classroom can achieve the following teaching and learning goals:

Model scientific practices of inquiry Support active, student-centered learning Facilitate inquiry-based learning

Help students to construct knowledge Help students to develop problem-solving skills

Improve long-term retention of knowledge (c.f., Tools for Teaching by Barbara Davis)

How to Use the Socratic Method in the Classroom

Role of the Teacher
During Socratic questioning, the teacher is a model of critical thinking who respects students' viewpoints, probes their understanding, and shows genuine interest in their thinking. The teacher poses questions that are more meaningful than those a novice of a given topic might develop on his or her own. The teacher creates and sustains an intellectually stimulating classroom environment and acknowledges the value of the student in that environment. In an intellectually open, safe, and demanding learning environment, students will be challenged, yet comfortable in answering questions honestly and fully in front of their peers.

Tips for the Teacher

Plan significant questions that provide structure and direction to the lesson.

Phrase the questions clearly and specifically.

Wait Time: Maintain silence and wait at least 5 to 10 seconds for students to respond. Keep the discussion focused. Follow up on students' responses and invite elaboration. Stimulate the discussion with probing questions.

Periodically summarize (e.g., on blackboard or overhead projector) what has been discussed. Draw as many students as possible into the discussion.

Do not pose yes/no questions, as they do little to promote thinking or encourage discussion. Do not pose questions that are vague, ambiguous, or beyond the level of the students. Further information on developing and guiding questioning.

Role of the Student

Before an exercise in thoughtful questioning, it is helpful if the teacher tells students that they are expected to do the following:

Participate when called upon. Answer questions as carefully and clearly as possible. Address the whole class so that everyone can hear their answers.

Be as succinct as possible in the interest of maximizing classroom time and effectiveness.

General Examples of Socratic Questions

Socratic questions can be phrased in three general ways:

1. 2.

To explore a general aspect of course material. "Describe different types of tectonic movement along plate boundaries." To encourage creativity and brainstorming. "Think of as many causes as possible for the origin of a large boulder found perched upon a nearly flat plain that is underlain by a rock type different from that of the boulder."

3.

To focus attention on a specific problem. "Compare the evidence used by scientists to support the idea of biological evolution (or modern global warming) with that used by others who reject the possibility of evolution (or global warming)." Educational researchers refer to specific categories of questions, indicated below with examples. Questions of Clarification

What is your main point? Can you give me an example? What is the source of that idea or information? Can you summarize what we discussed?

Questions that Probe Assumptions

What are you assuming? How would you support your assumption?

Questions that Probe Reasons and Evidence

What did you observe in the demonstration/experiment? What evidence supports your hypothesis?

Questions that Probe Implications and Consequences

What effect would that have?

What could you generalize from this observation? What does that remind you of? What do you predict will happen next?

Guiding Socratic Questioning

Example of Guiding Students to Better Answers and Greater Understanding during Socratic Questioning
Note: This example differs from one given in the Geoscience Examples: Hydrosphere/Cryosphere module, in that the first student response is incorrect, the student continues to have difficulty answering questions, and the teacher must make more effort to guide the student towards understanding. Only the beginning of the lesson is presented here. A fuller version is given in the Geoscience Examples part of the module for Socratic questioning. The instructor/teacher is identified as T, and the student as S. T. Imagine that it has been raining for 2 days in an area on the outskirts of Denver, Colorado, during July. Total rainfall for the event is 2 inches. The nearly level terrain is covered with wheat fields. What happens to the rain? [Wait at least one minute to give students time to reflect, then call for students to raise their hands if they have an answer. If not, choose and call on someone at random.] S. It runs off the ground. T. What are you assuming about the ground if you say that all of a 2-inch rainfall runs off a wheat field? [No answer from the student. Student looks perplexed. Teacher poses the question in slightly different way.] T. Are you assuming that the ground is like a sponge that soaks up the water, or a counter top from which water runs off? S. That the ground is like a counter top. T. Can you give one or more reasons for why your assumption might be valid? S. No. Not really. I guess it wouldn't be like a counter top, though. T. Why not? S. Well, the counter top is solid and hard, and the ground isn't.

T. I could argue that the ground is solid (it's not liquid or gas, right?), and it is hard, just like the counter top. Something else must be different from a counter top. What else is different? Imagine that you're the farmer digging up the soil after a Fall harvest. What does the ground look like? S. It's dirt. T. And how is dirt different than a counter top? S. It's crumbly, and loose. T. Can a pile of dirt, which consists of "crumbs" that actually are small pieces of mineral and organic matter, absorb water? And if so, how does it do so? S. Yes, it could absorb water I think. Water can fill the spaces between the crumbs. T. So the ground is actually porous and permeable if it consists of loose dirt rather than solid rock. Going back to our original question then, what would happen to the 2 inches of rainfall? S. It could soak into the ground. T. And what if it kept raining and raining, for days and days? Would the dirt be able to keep soaking up more and more water? S. I guess that the holes would fill up with water. [Teacher pauses to wait for more response from the student.] So, I guess that eventually the water would run off, like the counter top.

Examples of Socratic Questions for Introductory Geoscience Courses

Refine the Results
Earth System Topics
Atmosphere 2 matches Biosphere 3 matches Climate 2 matches Hydrology 1 match Oceans 1 match Solar System and Astronomy 1 match Solid Earth 2 matches Surface Processes 1 match Time/Earth History 1 match

Results 1 - 10 of 11 matches

Global Warming: Detailed Example part of Examples Detailed, annotated example of Socratic questioning for topics of climate change, global warming, and greenhouse gases in the atmosphere. Earth System Topics: Climate, Atmosphere

Hydrosphere: Detailed Example part of Examples Detailed, annotated example of Socratic questioning for topics of hydrologic cycle, streamflow hydrographs, and impacts of land-use change on streamflow. Earth System Topics: Hydrology, :Surface Water, Ground Water

Solar Radiation: Sample Socratic Questions part of Examples Cluster of questions on variations in solar radiation influx to Earth's surface. Earth System Topics: Climate, Atmosphere

Solar System: Sample Socratic Questions part of Examples Solar system and Universe Socratic questions and answers. Earth System Topics: Solar System and Astronomy

Earth History and Time: Sample Socratic Questions part of Examples Time and Earth History sample Socratic questions and answers. Earth System Topics: Time/Earth History

The Biosphere: Sample Socratic Questions part of Examples Sample Socratic questions and answers regarding the biosphere. Earth System Topics: Biosphere

Earth Surface: Sample Socratic Questions part of Examples Earth surface processes sample Socratic questions and answers. Earth System Topics: Surface Processes

Oceans: Sample Socratic Questions part of Examples Oceans Socratic questions and answers. Earth System Topics: Oceans:Biological Oceanography, Solid Earth, Oceans:Ocean Basin Topography

Are Viruses Alive: Sample Socratic Questions part of Examples This example Socratic questioning page provides an outline for leading a classroom discussion regarding whether or not viruses are alive. Sample questions, resources for background information, and tips and ...
Earth System Topics: Biosphere

Are Nanobacteria Alive: Sample Socratic Questions part of Examples This example Socratic questioning page provides an outline for leading a classroom discussion regarding whether or not nanobacteria exist. Sample questions, resources for background information, and tips and ...

Assessment
This material is replicated on a number of sites as part of the SERC Pedagogic Service Project
Author Profile

William Slattery, Departments of Geological Sciences and Teacher Education, Wright State University, Dayton, Ohio

Related Links

Cutting Edge: Observing and Assessing Student Learning Course Design

What is Assessment?

Assessment is the process of observing and measuring learning. Assessments provide faculty with a better understanding of what your students are learning and engage students more deeply in the process of learning geoscience content. By using assessment strategies that draw students into the assessment process it is more likely that they learn more of the geoscience content that you want them to learn while getting the added benefits of learning skills that will be useful to them in the future. As the National Science Education Standards (NRC, 1996 ) points out, assessments don't take time from learning, they are learning experiences by themselves. Click on the following link for additional information on the Domains of Learning.

Why Is Assessment Important?

By deliberately using different Functions of Assessments at specific times during the learning process students will have a clearer vision of what is expected of them and generally will be more positive about their course experiences. They will also learn to use skills that will help them understand how scientists analyze and present findings. In addition, a program of sustained student assessment is the foundation for evaluation of courses and geoscience programs.

How to Use an Assortment of Assessment Strategies

Imagine that you have just finished a lecture on atmospheric dynamics, ocean currents or plate tectonics. Students anxiously ask "will we be tested on this?" They are concerned about getting a good grade. There might be some questions you have yourself, such as, "How do I know my students learned anything? Have they met my learning objectives for the unit? Are they ready to tackle more abstract conceptual ideas in the geosciences?" By learning how to use the assessment strategies in this module, you will reduce student anxiety and it is likely that they will learn and remember more of what you want them to know.

Examples of Assessment in Various Learning Settings

Geoscientists teach and learn in the laboratory, classroom and in the field. Students learning in those varied settings can get value added learning experiences by engaging in some of the same activities that geoscientists use to gather and analyze data and report their findings to their colleagues. Click on the following link to see examples of assessment in several learning settings.

Domains of Learning

How We Learn
Humans are lifelong learners. From birth onward we learn and assimilate what we have just learned into what we already know. Learning in the Geosciences, like all learning, can be catagorized into the domains of concept knowledge, how we view ourselves as learners and the skills we need to engage in the activities of geoscientists. As early as 1956 Educational Psychologist Benjamin Bloom divided what and how we learn into three seperate domains of learning.

Cognitive Domain
Cognitive Domain - This domain includes content knowledge and the development of intellectual skills. This includes the recall or recognition of specific facts and concepts that serve developing intellectual abilities and skills. There are six major categories, starting from the simplest behavior (recalling

facts) to the most complex (Evaluation). The University of Washington's Geography Department website Major Categories in the Taxonomy of Educational Objectives has a detailed explanation of Bloom's Six Levels of Cognitive Development (more info)

Affective Domain
Affective Domain- How does one approach learning? With confidence, a can do attitude. The Affective domain includes feelings, values, appreciation, enthusiasms, motivations, and attitudes. The University of Dayton, School of Law Affective Domain website describes each catagory in the domain and provides illustrative examples and keywords for the cognitive, affective, and psychomotor domains.

Psychomotor Domain
Psychomotor Domain- The psychomotor domain includes physical movement, coordination, and use of the motor-skill areas. Development of these skills requires practice and is measured in terms of speed, precision, distance, procedures, or techniques in execution. For a more detailed treatment of this domain see the Penn State Teaching and Learning with Technology website Psychomotor Domain Taxonomy (more info)

Connecting Learning and Assessment

To see how assessments are built from these domains of learning and to learn how to build effective assessments go to the Hallmarks of Effective Assessment page.

Resources
Bloom Benjamin S. and David R. Krathwohl. Taxonomy of Educational Objectives: The Classification of Educational Goals, by a committee of college and university examiners. Handbook I: Cognitive Domain. New York, Longmans, Green, 1956.

Hallmarks of Effective Assessment

The Analogy Between Maps and Assessment
If you have ever taken a road trip, you recognize the utility of maps. We can hardly expect to get to our destination without knowing where we started from. Maps allow us make our journey from point A to point B in the most efficient way possible. Assessment is a lot like using a map. It allows us to: know where our students are conceptually before we begin a learning activity.

decide the most efficient way for them to meet the learning objectives we set.

measure their progress and/or achievement of the learning objectives.

To continue our map analogy, just as knowing that different colored lines on the map represent the difference between a major highway and a dirt road can help a map user plan an easier journey, there are a few "eternal truths" concerning assessment. Your students will value and focus on material that will impact their grade.

Their motivation to learn will increase if you tell them "up front" what skills and content will be assessed with clearly stated learning objectives. Always align your assessment to the learning objectives of the activity.

Learning Objectives

Learning objectives are the building blocks to conceptual content knowledge and the skills that enable students to build their own understandings of geoscience. Learning objectives are measurable statements of what students should recall, understand and be able to do as a direct outcome of a particular learning activity. Each learning activity students engage in has at least one learning objective. Some activities will have more than one. Learning Objectives are defined as what you want the students to know or understand after they have finished the learning activity. The learning objective could be an important geoscience concept such as the sequence of crystallization in Bowen's Reaction Series, or it could be a method of collecting data in the field, such as using a Brunton Pocket Transit to measure strike and dip. Deepen your understanding of how learning objectives are linked to assessment by viewing a short case study. Click on the following link to see how to assess your learning objectives using a variety of assessment strategies

Resources
Huba, Mary E. and Freed, Jann E.(2000) Learner-Centered Assessment on College Campuses. Allyn and Bacon, Boston, Massachusetts. Chapter 1 provides a systemic overview of assessment and the role of learning outcomes in the process.

How Learning and Assessment are Linked

Learning Objectives and Assessment
Mini Case Study

In order to reach a course goal that states, "students will understand the relationship between local and regional folding in the Appalachian Orogenic Belt" a graduate student teaching a geology lab section decides to take her class into the field to visit several folded Devonian outcrops. She will ask students to find the strike and dip of outcrops at several different locations. Students will then compare the data they collected with regional scale geologic maps of the Appalachian Orogenic Belt. They will then discuss similarities and differences between the regional and local strikes. There is only one slight difficulty. The students have never used a Brunton before. As a preliminary learning activity she sets up several boards outside that have been anchored against trees on the quad. Before they begin she hands out a worksheet that has the following learning objective:Students will be able to use a Brunton Pocket Transit to find strike and dip to within 5% accuracy.After demonstrating the use of the instrument, students work to gather strikes and dips. She assesses their work and assists students until all of them achieve the learning objective. Now the students can apply this new skill to build their understandings of the relationships they will see in the field and on the maps

Elements of Effective Assessment

The brief case study above provides examples of some of the elements of assessment that actively engage students in the assessment process and provides additional opportunities for them to learn concepts and the processes of science. These are: Instructors precede learning activities with a diagnostic preassessment of student knowledge and/or skills

Instructors design learning objectives aligned with the educational needs of their students Students are actively involved in the assessment process and receive prompt feedback

Learners apply the skills and knowledge gained during assessment in new situations

The Graduate Teaching Assistant in the hypothetical case study performed a prelearning activity assessment by asking the students if any of them had ever used a Brunton to measure strike and dip. She set an attainable learning objective, demonstrated the process and then gave students prompt feedback, working with them until all had attained the learning objective. The students then used the new skill to build their conceptual knowledge of geoscience. This can be considered the model for assessment. There are several methods that instructors may use to assess introductory geoscience activities and courses but all use the same framework as in the dotpoints above.

Assessment in Different Learning Settings

Examples of effective assessment strategies below are paired with activities using different learning settings. Additional assessment strategies are provided in the resources listed on each page:

Assessing Interactive Lectures Assessing Laboratory Activities Assessing Field-based Activities Assessing Project-based learning Assessing Web enhanced and On-line Learning

A Short Glossary of Assessment Terms

Affective Domain Outcomes of education involving feelings more than understanding; likes, pleasures ideals and/or values. Assessment The Latin root assidere means to sit beside. In an educational context, the process of observing learning; describing, collecting, recording, scoring, and interpreting information about a student's or one's own learning. At its most useful, assessment is an episode in the learning process; part of reflection and autobiographical understanding of progress. Traditionally, student assessments are used to determine achievement of learning objectives and grades. Authentic Assessment Assessment strategies that require students to directly reveal their ability to think critically and to apply and synthesize their knowledge. Cognitive Domain

Outcomes of education involving thinking and content knowledge, logic, classification and problem solving. Evaluation Both qualitative and quantitative descriptions of student behavior, plus value judgments concerning the desirability of that behavior. Using collected information (assessments) to make informed decisions about continued instruction, programs, and activities. Formative Assessment Observations which allow one to determine the degree to which students know or are able to do a given learning task, and which identifies the part of the task that the student does not know or is unable to do. Outcomes suggest future steps for teaching and learning. Learning Goal See Learning Objective Learning Objective What you want students to know and understand after they complete a learning experience, usually a culminating activity, product, or performance that can be measured. Mean One of several ways of representing a group with a single, typical score. It is figured by adding up all the individual scores in a group and dividing them by the number of people in the group. Can be affected by extremely low or high scores. Median The point on a scale that divides a group into two equal subgroups. Another way to represent a group's scores with a single, typical score. The median is not affected by low or high scores, as is the mean. Metacognition The knowledge of one's own thinking processes and strategies, and the ability to consciously reflect and act on the knowledge of cognition to modify those processes and strategies. Mode The most frequently occurring score in a group {1233345}. Not all sets have a mode {1234567}, and some sets may have more than one mode {1123445}. Performance Assessment See Authentic Assessment.

Portfolio A systematic and organized collection of a student's work that exhibits to others the direct evidence of a student's efforts, achievements, and progress over a period of time. The collection may involve the student in the selection of its contents, and should include information about the performance criteria, the rubric or criteria for judging merit, and evidence of student self-reflection or evaluation. It should include representative work, providing a documentation of the students' performance and a basis for evaluation of the student's progress. Portfolios may include a variety of demonstrations of learning and have been gathered in the form of a physical collection of materials, videos, CD-ROMs, reflective journals, etc. Rubric In general a rubric is a scoring guide used in subjective assessments. A rubric implies that a rule defining the criteria of an assessment system is followed in evaluation. A rubric can be an explicit description of performance characteristics corresponding to a point on a rating scale. A scoring rubric makes explicit expected qualities of performance on a rating scale or the definition of a single scoring point on a scale. Summative Assessment Evaluation at the conclusion of a unit or units of instruction, or an activity or plan to determine or judge student skills and knowledge. Also an evaluation of the effectiveness of a plan or activity.

The Functions of Assessment

Varieties of Assessment
It's been said that in life, timing is everything. As in life, assessments performed at crucial times in the learning process can spell the difference between gathering data to evaluate students and using assessments to enhance learning. Based on timing and purpose, four functions of assessment data are:

Formative assessment provides diagnostic feedback to students and instructors at short-term intervals (e.g., during a class or on a weekly basis)

 Summative assessment provides a description of students' level of attainment upon completion of an activity, module, or course

Evaluative assessment provides instructors with curricular feedback (e.g., the value of a field trip or oral presentation technique)

Educative assessment Integrated within learning activities themselves, educative assessment builds student (and faculty) insight and understandings about their own learning and teaching. In short, assessment IS a form of learning NRC, 1996 , p. 76)

At its most useful, educative assessment (sometimes termed active assessment) is an episode in the learning process; part of reflection and autobiographical understanding of student progress. Click on the following hotlink to learn more about assessment strategies you can use in your introductory geoscience courses, laboratories, field based activities and projects.

Diagnostic and Formative Assessment

Diagnostic Pre-Assessments

Diagnostic assessments (also known as pre-assessments) provide instructors with information about student's prior knowledge and misconceptions before beginning a learning activity. They also provide a baseline for understanding how much learning has taken place after the learning activity is completed. Instructors usually build concepts sequentially throughout a course. For example, the Coriolis effect may be taught prior to a unit on ocean currents. A diagnostic pre-assessment given after the Coriolis effect activity but before the Ocean current activity will provide an opportunity to determine if students remember the concepts they need. If some students don't remember, then a refresher will make the Ocean current activity more meaningful to your students. Diagnostic assessment data may be gleaned from:

Summative assessments of the previous learning activity.

Short assessments that focus on key knowledge and concepts such as ConcepTests and Minute Papers (more info)

Formative Assessments

Formative assessments take place during a learning activity to provide the instructor with information regarding how well the learning objectives of a given learning activity are being met. The value of formative assessment is pointed out by Black and William'(1998) paper "Inside the Black Box: Raising Standards Through Classroom Assessment" (Phi Delta Kappan, October 1998) They point to evidence that high quality formative assessment has a powerful impact on student learning. In addition, formative assessment is particularly effective for students who have not done well in school, narrowing the gap between low and high achievers while raising overall achievement. Most instructors intuitively use questioning as a method of formative assessment but in large lecture classes not

every student can be questioned because of time constraints. Formative assessment is also useful in virtually all learning activities such as preparing oral and written reports, fieldwork and as projects and case studies progress. Here is an example of using on-going formative assessment in a large lecture course.

Resources
The Geoscience Concept Inventory WebCenter includes a collection of questions you can use for diagnostic or formative assessment.

Assessing How Students Learn is a short article that describes several methods for finding out what learning strategies your students are using. Gather Evidence of Learning from the Lesson Study Project for College Teachers. This site describes how to gather evidence of student learning, thinking, and engagement in your classroom.

Using Assessments to Evaluate Geoscience Courses

Beyond Student Evaluations

Most colleges and universities use student evaluatiuons of faculty teaching as a measure of what students are learning and their perceptions of how much content was learned in a given course. These student evaluations are sometimes used to judge faculty teaching effectiveness and for decisions on promotion and tenure. Students are sometimes not very good judges of what they have learned, and evaluations of faculty teaching usually are completed at the end of the course, too late to be used by faculty to make mid-course corrections in teaching strategies. Other SERC modules contain examples of Knowledge surveys and ConcepTests given at the start, middle and end of a course to provide valuable feedback from students and provide a measure of teaching and learning independent of student evaluations.

Related Links

Setting Course Goals, from the Course Design Tutorial

Developing Course Instructional Goals

How well are my students learning? Is the course effective in developing the skills and content knowledge students need to take more advanced courses and become lifelong learners? A sustained program of formative and summative assessment can bring data to bear on answering these and other questions. A Teaching Goals Inventory and Self-Scorable Worksheet Angelo and Cross, 1993 pp.393-397) provides a model for defining the essential instructional goals of your course. Responses are clustered into several catagories. Among these are:

Discipline-specific knowledge and skills Higher-order thinking skills Work and career skills

Teaching Goals Worksheets are but one instrument that can be used for course evaluation. For more information on other types of assessments that can be used to measure changes in student knowledge, visit"How to Use Assesment Strategies" To get more information on strategies you can use to weave individual course evaluations into the fabric of program evaluation, visit "Using Assessments to Evaluate Geoscience Programs."

Resources
Classroom Assessment Techniques: A Handbook for College Teachers. (1993) Thomas A. Angelo and K. Patricia Cross Jossey-Bass, San Francisco 2nd Edition

Using Assessments to Evaluate Geoscience Programs

Related Links
Program Assessment and Review

A Framework for Knowing

Course instructors that use assessment to better understand changes in student content knowledge and attitudes provide a foundation on which to gather direct measures of how students are meeting overall program goals. Increasingly, Geoscience departmental program evaluations are required for both internal administrative purposes and for use by outside accreditation organizations. [Palomba and Banta, 1999] provide a framework for faculty wishing to begin to assess programs in their book, Assessment Essentials: Planning Implementing and Improving Assessment in Higher Education.

Seeing the Big Picture

When assessing programs, one very useful kind of assessments to use are those that may be aggregated across all sections of a given course and across different courses. Access more information on assessment of individual courses. A well constructed plan is provided by Palomba and Banta (1999, pg. 6-18). They suggest six essential steps to develop a comprehensive assessment of an educational program: Agree on learning goals and objectives: Consider asking each faculty member to complete a Teaching Goals Inventory to determine the goals and objectives for each of the courses in the program, then finding common goals and objectives for all courses.

Design and Implement a Thoughtful Approach to Assessment Planning: Is the purpose of the assessment to improve the program (formative) or to make judgments about it, or both? (summative)

Involve all Stakeholders: Students, alumni, employers and other stakeholders provide insight on goals and objectives and provide support for the assessment process.

Select/Design and Implement Data Collection: Will you employ direct measures where students display their knowledge and skills as they respond to the instrument itself, or indirect measures that ask students to reflect on their learning but not display it. Will the assessment instruments be qualitative or quantitative?

Examine the data, Share it and Act on it: How will the data collected be presented? Different groups have different needs.

Reexamine the Assessment Process: Do the results lead to additional questions that may require further assessment?

References and Resources

Instructional Assessment Resources: Program Evaluation This site, published by the Division of Instructional Innovation and Assessment at the University of Texas - Austin, provides guidance on creating and implementing an evaluation of an academic program. Other assessment information is also available on this site.

How to Use Assessment Strategies

Assessment need not take time away from learning; assessments can be learning experiences in themselves. Active assessment strategies enhance student content understanding and promote skills that will be beneficial to students throughout their lives. The ability to see the big picture, develop effective oral and written reports and the ability to work cooperatively with their peers are skills that are promoted by active assessment. In this module you can access the following Assessment strategies: Scoring Rubrics help students focus on content and Instructional Rubrics to guide them in developing presentations, written and oral reports.

Concept Maps assist students in "seeing the big picture".

Portfolios document student learning and improve student metacognition. Rebecca Teed's Starting Point module Co-operative Learning: Assessment contains strategies that encourage peer-to-peer learning, individual and group grading. The SERC Cutting Edge Assessment site also includes information on the following assessment strategies: ConcepTests - Conceptual multiple-choice questions useful in large classes.

Knowledge Survey - Students answer whether they could answer a survey of course content questions. Exams - Find tips on how to make exams better assessment instruments. Oral Presentations - Tips for evaluating student presentations. Peer Review - Having student assess themselves and each other. Written Reports - Tips for assessing written reports.

Other Assessment Types including concept sketches, case studies, seminar-style courses, mathmatical thinking and performance assessments. Click on the link to see a short glossary of assessment terms.

Developing Scoring Rubrics

"Learning increases when learners have a sense of what they are setting out to learn, a statement of explicit standards they must meet and a way of seeing what they have learned." Loaker, Cromwell and O'Brien (1986) pg.47 One of the timeless verities of student psychology is that students will focus on learning material that will impact their grade. Rubrics are a way to make explicit our expectations of what students will need to know and be able to do in order to receive a given grade. Rubrics help instructors to develop clear and attainable

learning objectives for their students and if provided to students prior to the activity, serve to guide their efforts.

Scoring Rubrics Focus and Promote Learning

Assessment sometimes carries a sense of the mysterious for students. They may be told to take notes in class, read the chapter and answer the questions at the end, but they may get few specifics regarding what material will be assessed, and at what depth. In contrast, rubrics given to students before the learning activity starts helps them get a clear sense of what knowledge and skills they need in order to achieve a given grade. In their book Learner-Centered Assessment on College Campuses, Hubba and Freed (2000) point out that Scoring rubrics usually contain the following elements: Clear statements of the level of knowledge you expect the student to achieve for them to receive a given grade.

The dimensions of the quality of work you expect the student to achieve. Commentaries describing your expectations of knowledge and quality that distinguishes each grade band (e.g. ABCDF). Keep a few questions in mind while developing an instructional rubric.

What are the essential elements of high quality work? How many levels of achievement are to be described? Are the criteria for each level clearly described?

Diane Ebert-May's website titled Classroom Assessment Techniques-Scoring Rubrics (more info) contains detailed information regarding the development of these valuable tools. Click on the following link to see an example of using Scoring Rubrics for assessment of field-based activities in the geosciences and the resources below for other assessment ideas.

Resources

Browse Information about Rubrics. Browse Rubric Examples

Developing Instructional Rubrics

Instructional Rubrics That Address Presentations/Reports

Assessing student work by means of oral and written reports are golden opportunities to teach students skills that they will use throughout their working lives. Both oral and written reports share common themes and therefore can be addressed together. Introductory thesis paragraphs, a body of evidence with support for assertions and summaries of findings are common elements in both oral and written reports. In beginning to develop a rubric for written reports or oral presentations ask yourself these questions: What prior experience do my students have in preparing oral/written reports?

What do I feel are the essential elements I will expect them to include (e.g. citations, supporting evidence)? Considering the experience of students in guiding their work will determine how much support they will need to be able to achieve your expectations. Providing samples/models of what you consider excellent work will provide them with a mental framework to build their own reports. Elements of the framework include:

What are the essential elements of a high quality report? How many levels of achievement are to be described? Are the criteria for each level clearly described?

The following hotlinks will bring you to specific examples of using an instructional rubric to assess a geoscienceproject-based learning activity and a Laboratory activity See the resources below for other assessment tools.

Resources
Classroom Assessment Techniques: A Handbook for College Teachers (Second Edition).Angelo and Cross, 1993 This book by Thomas Angelo and K. Patricia Cross provides a practical guide to help faculty develop a better understanding of the learning process in their own classrooms and assess the impact of their teaching upon it. The authors offer detailed how-to advice on classroom assessment - from what it is and how it works to how to plan, implement, and analyze assessment projects. Their approach is illustrated through numerous case studies. The book features fifty Classroom Assessment Techniques, each presented in a format that provides an estimate of the ease of use, a concise description, step-by-step

procedures for adapting and administering the technique, practical advice on how to analyze the data and other useful information. (citation and description) Classroom Assessment Techniques: Attitude Surveys. This page describes attitude surveys, one of a series of Classroom Assessment Techniques (CATs) provided by the Field-tested Learning Assessment Guide (FLAG) website. The CATs of FLAG were constructed as a resource for science, technology, engineering and mathematics instructors to emphasize deeper levels of learning and to give instructors valuable feedback during a course. The attitude surveys consist of a series of statements in which students are asked to express their agreement or disagreement using a scale, thus providing information on the students? perceptions of their classroom experience. The site provides an overview of this assessment instrument, including information about how to use an attitude survey. This site is also linked to a set of discipline-specific "tools" that can be downloaded for immediate use, as well as supplementary links and sources are included to further explore this assessment technique. (more info)

Classroom Assessment Techniques: Conceptual Diagnostic Tests. This page describes conceptual diagnostic tests, one of a series of Classroom Assessment Techniques (CATs) provided by the Field-tested Learning Assessment Guide (FLAG) website. The CATs of FLAG were constructed as a resource for science, technology, engineering and mathematics (STEM) instructors to emphasize deeper levels of learning and to give instructors valuable feedback during a course. Conceptual diagnostic tests are used to assess how well students understand key concepts in a STEM field prior to, during, and after instruction. They assess student understanding using a multiple-choice or short-answer format that has been designed to address misconceptions. This site provides an overview of this assessment instrument including information about why conceptual diagnostic tests are beneficial to use and how to use them. The site is also linked to a set of discipline-specific "tools" that can be downloaded for immediate use, as well as supplementary links and sources to further explore this assessment tool. (more info)

Classroom Assessment Techniques: Interviews. This page describes the technique of using interviews to assess student understanding. The assessment tool is one of a series of Classroom Assessment Techniques (CATs) provided by the Field-tested Learning Assessment Guide (FLAG) website. The CATs of FLAG were constructed as a resource for science, technology, engineering and mathematics (STEM) instructors to emphasize deeper levels of learning and to give instructors valuable feedback during a course. Interviews enable instructors to judge the extent of understanding students have developed with respect to a series of well-focused, conceptually-related scientific ideas. This site provides an overview of this assessment instrument including information about how to use classroom interviews to their maximum benefit. The site is also linked to a set of

discipline-specific "tools" that can be downloaded for immediate use, as well as supplementary links and sources to further explore this assessment tool. (more info) A Data Rich Exercise for Discovering Plate Boundary Processes. [Sawyer et al., 2005] This article in the Journal of Geoscience Education describes a classroom exercise based on four world maps containing earthquake, volcano, topographical and seafloor age data. Students participate in this exercise by using a "jigsaw" approach, in which they break into four groups and become specialists on one of the map types. After being organized into new groups with one specialist from each map represented, the groups present their data from the class. This exercise (assessment tool) has shown that students come away with knowledge of the key features of each type of plate boundary and a sense of why it looks the way it does. (Full Text Online)

An Investigation of Student Engagement in a Global Warming Debate. [Schweizer and Kelly, 2005] This article in the Journal of Geoscience Education investigates how using debate as a pedagogical tool for assessing earth system science concepts can promote active student learning, present a realistic and dynamic view of science, and provide a mechanism for integrating the scientific, political and social dimensions of global environmental change. This is achieved by using the causes of global warming as an example of earth system science for the debate. (citation and

description)

Classroom Assessment Techniques: A Handbook for College Teachers (Second Edition).Angelo and Cross, 1993 This book by Thomas Angelo and K. Patricia Cross provides a practical guide to help faculty develop a better understanding of the learning process in their own classrooms and assess the impact of their teaching upon it. The authors offer detailed how-to advice on classroom assessment - from what it is and how it works to how to plan, implement, and analyze assessment projects. Their approach is illustrated through numerous case studies. The book features fifty Classroom Assessment Techniques, each presented in a format that provides an estimate of the ease of use, a concise description, step-by-step procedures for adapting and administering the technique, practical advice on how to analyze the data and other useful information. (citation and description)

Classroom Assessment Techniques: Minute Paper. This page describes the minute paper, one of a series of Classroom Assessment Techniques (CATs) provided by the Field-tested Learning Assessment Guide (FLAG) website. The CATs of FLAG were constructed as a resource for science, technology, engineering and mathematics (STEM) instructors to emphasize deeper levels of learning and to give instructors valuable feedback during a course. The minute paper is a concise note, taking one minute and written by students, that focuses on a short question presented

by the instructor to the class. It provides real-time feedback from a class to find out if students recognized the main points of a class session and also helps the instructor make changes for the next class. This site provides an overview of this assessment instrument including information about how to use minute papers in the classroom. The site is also linked to a set of discipline-specific "tools" that can be downloaded for immediate use, as well as supplementary links and sources to further explore this assessment tool. (more info) Classroom Assessment Techniques: Weekly Reports. This site describes the use of weekly reports as an assessment tool for student learning. It is one of a series of Classroom Assessment Techniques (CATs) provided by the Field-tested Learning Assessment Guide (FLAG) website. The CATs of FLAG were constructed as a resource for science, technology, engineering and mathematics instructors to emphasize deeper levels of learning and to give instructors valuable feedback during a course. Weekly reports provide rapid feedback about what students think they are learning and what conceptual difficulties they are experiencing. This site provides an overview of this assessment technique including information about how to use it. The site is also linked to a set of discipline-specific "tools" that can be downloaded for immediate use, as well as supplementary links and sources to further explore this assessment tool. (more info)

A Cohort-Driven Assessment Task for Scientific Report Writing. [Chuck and Young, 2004] This article from the Journal of Science Education and Technology describes a formative assessment task that was developed to improve the scientific report writing skills of university students. The assessment task involved feedback from instructor to students before final submission of their reports, as well as the instructor's use of a cohort-specific marking scheme based on the deficiencies that were evident within the class group. Using a mixture of peer and self-review against specific criteria, the students were required to resubmit an amended report. This technique proved to be efficient for both parties and also resulted in improvement of skills of the entire student population. (citation and description)

Weekly Reports: Student Reflections on Learning. An Assessment Tool Based on Student and Teacher Feedback. [Etkina and Harper, 2002] This article from the Journal of College Science Teaching details the use of weekly reports; a structured journal form of formative assessment that allows instructors to receive information from students and alter their instruction based on student needs. (citation and description)

Weekly Reports: A Two-Way Feedback Tool. [Etkina, 2000] This article from Science Education describes how to use weekly reports written by students as a two-way feedback tool in teaching science. The weekly reports help students to reflect on their knowledge, learn how to ask questions, and predict what questions their teacher is likely to ask. The

reports help teachers to identify the difficulties their students experience while learning new material, to adjust their teaching to the students? needs, and to match the levels of difficulty of learning and testing. The authors of this study conclude that there is a common mismatch between learning and assessment and offers a solution through weekly journals. (citation and description) Assessing Science Understanding: A Human Constructivist View. [Novak, Mintzes and Wandersee, 2000] This book by Joel J. Novak, James H. Mintzes, and Joseph D. Wandersee describes different kinds of assessments for measuring student understanding of science concepts. The book explores many assessment types and how they can be used in the classroom to improve instruction and learning. Topics include assessment concept maps, structured interviews, observations, portfolios and written products. The book also provides useful examples, data, and extensive references to the literature. (citation and description)

Applying Argumentation Analysis to Assess the Quality of University Oceanography Students' Scientific Writing. [Takao, Prothero and Kelly, 2002] This article from the Journal of Geoscience Education describes a study which examined 24 student papers from an introductory oceanography class and analyzed the quality of their written arguments. The article discusses ways of using argumentation to help students understand how to tie data to theoretical assertions and to provide ways for students and teachers to assess the uses of evidence in scientific writing. Included is an argumentation analysis model that describes argument structure according to epistemic levels. (Full Text Online)

Developing Concept Maps

Making Thinking Visible
Concept maps are visual representations of linkages/connections between a major concept and other knowledge students have learned. Concept maps are excellent tools to provide instructors with diagnostic pre-assessment prior to beginning a unit and formative assessments during learning activities. Concept maps also provide immediate visual data to geoscience instructors on student misconceptions and their level of understanding. Angelo and Cross (1993) indicate that concept maps develop student abilities in certain critical areas. Among these are:

The ability to draw reasonable inferences from observations The ability to synthesize and integrate information and ideas The ability to learn concepts and theories in the subject area

Getting Started With Concept Mapping

Students may not be familiar with concept maps and it is suggested that they practice with familiar concepts. Michael Zeilik's website Concept Maps provides step by step directions on the construction and use of concept maps in the college classroom. Concept maps can provide a springboard for classroom discussions of systems and relationships among major and sub-concepts in the geosciences. They can be used to focus learning if concept maps are developed by the instructor, or may be used as pre-diagnostic assessment tools or formative assesments as concepts are developed.

Formative Assessment Using Concept Mapping: A Geoscience Application

At the beginning of an Introductory Metroeorology unit on Moisture in the Atmosphere the instructor passes out copies of a concept map (Acrobat (PDF) 450kB May2 05) to her students. The major concepts are identified, but detail is missing. As the unit progresses the instructor asks students to add to the original concept map. For example the students could add:

the types of reservoirs that occur on land different types of precipitation additional mechanisms such as transpiration

As the unit progresses the students continue to see the major concepts repeatedly, and the instructor can track student understandings of the relationships of parts to the whole (or misconceptions) as they arise by collecting and reviewing the concept maps.

Resources
Classroom Assessment Techniques: A Handbook for College Teachers (Second Edition).Angelo and Cross, 1993 This book by Thomas Angelo and K. Patricia Cross provides a practical guide to help faculty develop a better understanding of the learning process in their own classrooms and assess the impact of their teaching upon it. The authors offer detailed how-to advice on classroom assessment - from what it is and how it works to how to plan, implement, and analyze assessment projects. Their approach is illustrated through numerous case studies. The book features fifty Classroom Assessment Techniques, each presented in a format that provides an estimate of the ease of use, a concise description, step-by-step

procedures for adapting and administering the technique, practical advice on how to analyze the data and other useful information. (citation and description) Classroom Assessment Techniques: Concept Mapping. This page describes concept maps, one of a series of Classroom Assessment Techniques (CATs) provided by the Field-tested Learning Assessment Guide (FLAG) website. The CATs of FLAG were constructed as a resource for science, technology, engineering and mathematics instructors to emphasize deeper levels of learning and to give instructors valuable feedback during a course. A concept map is a diagram of nodes adjoined by directional lines and organized in hierarchical levels that move from general to specific concepts. Concept maps are used to assess how well students see the "big picture," and to illustrate students' conceptual knowledge. This site provides an overview of this assessment instrument that includes information about how to use them. This site is also is linked to a set of discipline-specific "tools" that can be downloaded for immediate use, as well as supplementary links and sources to further explore this assessment tool. (more info)

Assessment and Active Learning Strategies for Introductory Geology Courses. McConnell et al. (2003) This article describes several techniques to promote active learning in the classroom and compares classes taught using these with those using traditional lectures. General education Earth Science classes was evaluated using formative assessment exercises conducted by students in groups. (Full Text Online)

Using Concept Maps to Plan an Introductory Structural Geology Course. [Clark and James, 2004] This article in the Journal of Geoscience Education describes a method of designing a structural geology course by using concept mapping. A detailed description of concept mapping is provided as well as an explanation as to why this approach is used and effective in planning a structural geology course. Also included in this article are examples of concept maps referring to structural geology topics. (Full Text

Online)

Concept Map Assessment of Classroom Learning: Reliability, Validity, and Logistical Practicality. [McClure and Sonak, 1999] This article in the Journal of Research in Science Teaching evaluates the characteristics and practicality of concept mapping as a technique for classroom assessment. Results indicate that the time required to provide training in concept mapping, have students produce maps, and then score them justifies the use of concept mapping as an effective and efficient classroom assessment technique. (citation and description)

Assessing Science Understanding: A Human Constructivist View. [Novak, Mintzes and Wandersee, 2000] This book by Joel J. Novak, James H. Mintzes, and Joseph D. Wandersee describes different kinds of assessments for measuring student understanding of science concepts. The

book explores many assessment types and how they can be used in the classroom to improve instruction and learning. Topics include assessment concept maps, structured interviews, observations, portfolios and written products. The book also provides useful examples, data, and extensive references to the literature. (citation and description) Comparison of the Reliability and Validity of Scores From Two Concept Mapping Techniques. [Ruiz-Primo et al., 2001] This paper from the Journal of Research in Science Teaching reports the results of a study that compared two concept-mapping techniques; one high-directed, ? fill-inthe-map,? and one low-directed, ? construct-a-map-from-scratch.? The article examines whether skeleton map scores were influenced by the sample of nodes or linking lines to be filled in; if the two types of skeleton maps were equivalent; and if the two mapping techniques provided similar information about students' understanding. The authors conclude that the construct-amap technique better reflects differences among students' knowledge structure.(citation and description)

Novel Assessments: Detecting Success in Student Learning. [Williams et al., 2004] This journal article from Frontiers in Ecology and the Environment illustrates how multiple methods can be used to assess student understanding. The authors describe four assessment techniques, including a knowledge probe, paired questions, concept mapping and experimental problems, and suggest that these methods will engage students in diverse ways to demonstrate their understanding. (citation and

description)

Developing Portfolios to Assess Student Learning

Metacognition is an awareness of one's own thought processes. As such, it involves an almost simultaneous, concious degree of self-awareness. "This is how I approach or think about a situation"; "this is how I might best approach this particular concept in order to more fully understand it"; "this is how I am thinking about this issue and it is or is not effective"; "these are other possible approaches I might take instead" (Courts and McInerney, 1993 p.57)

Portfolios Defined
Portfolios are personalized long term documentation of student mastery of course material. An essential element of portfolios are student reflections on their own learning and progression towards the mastery of the material documented in the portfolio. As such, portfolios are windows on the metacognitive process of students.

Types of Portfolios
In order to be useful as assessment tools the portfolio should include entries that demonstrate a progression of student understandings and ultimately mastery of the concepts. Huba and Freed (2000) identify two different portfolio types. All-Inclusive Portfolios that contain a complete record of all work done by a student in a course or program.

Selection Portfolios that are focused on documenting the achievement of mastery of specific course goals/objectives set by the course instructor.

Assessment Using Portfolios

Because portfolios are by their nature long term records of student progress and achievement they can be used to assess programs, courses or projects. Although the aspect of long-term change in student content content knowledge and metacognition may not be as marked when portfolios are used to assess courses, the process of portfolio development tends to focus students on deciding what the essential elements of learning are and what samples of their work best display these elements. In Chapter 5 of [Palomba and Banta, 1999]they discuss three critical elements of student portfolios in promoting and supporting student learning. These are:

Student involvement in the selection of entries Student preparation of written reflections about learning Continuing discussion with faculty regarding written reflections

Timothy Slater's portfolio website (more info) provides an in depth discussion of the use of portfolios in introductory science courses, including an example for an Introductory Environmental Science course. Click on the following link to learn how portfolios are used in an on-line Earth Systems Science course, or see the resources below for more on portfolios and other assessment tools.

Resources
Browse Additional Information about Portfolios

Assessment in Various Learning Settings

Building Scientific Literacy through Assessment
Teaching introductory geoscience is a challenge and an opportunity. In settings as diverse as large lecture classes, field trips and laboratory sessions we strive to build student geoscience content knowledge and an understanding of how science works. Of course, most of the students we teach won't become geoscientists but they need to be scientifically literate citizens. Quite possibly they will vote on global issues affecting the environment, regional public policy on such issues as water resources and local issues as well. Using assessment to engage their interest in geoscience, to focus their learning and to develop communication skills are excellent opportunities to increase their scientific literacy and to give them a view of science as a way of knowing.

Specific examples of assessment are provided for each of the learning settings below. Additional assessment strategies are provided in the resource lists on each page.

Interactive Lectures Laboratory Settings Field based Activities Project based Activities Web-enhanced and On-line Courses

Assessing Interactive Lectures

Assessment is often viewed as a process that takes time from teaching but since most students are grade concious and will focus on learning what will be assessed, the process of formative assessment can be a powerful driver for enhancing content understanding among introductory geoscience students. Using long-term formative assessment strategies, such as mapping plate boundaries using interactive lectures, can deepen content understandings and engage students in how data is used in the geosciences. Additional assessment strategies of interactive lectures are provided in the resource list at the bottom of this page.

Using Formative Assessment to Drive Learning

Black and William'(1998) paper "Inside the Black Box: Raising Standards Through Classroom Assessment"(Phi Delta Kappan, October 1998) points out that high quality formative assessment has a powerful impact on student learning, and that formative assessment is particularly effective for students who have not done well in school, narrowing the gap between low and high achievers while raising overall achievement. Allowing students to build their scientific understandings by interpreting data sets will strengthen their ability to use scientific data and provide the instructor with an array of opportunities to use the data sets in formative assessment. For example, The USGS Earthquake Hazards Program maintains a website that lists earthquakes that have occurred worldwide during the past several days. Visit the site, print the page and make a transparency of the week's earthquakes.

Ask students to plot the longitude and latitude of the earthquakes on a physiographic chart.

Make a new overhead transparency of the new data once each week, as more recent earthquakes replace older ones on the list At the end of several weeks, students will see that most, but not all earthquakes they have plotted are at the edges of the tectonic plates. You can increase the complexity of the interactive lecture activity by asking students to also plot depth of the earthquakes using pens of different color. Use an arbitrary depth such as 5 kilometers to differentiate between "shallow" and "deep" earthquakes. Then, prior to teaching plate tectonic concepts (e.g. the relative thicknesses of oceanic versus continental crust) the instructor can refer students to their physiographic charts to determine what the data suggests in terms of the relative thicknesses of continental versus oceanic crust.

More Examples
Minute Papers/ConcepTests-

The example provided above is not the only way to assess interactive lectures. Other strategies that increase the interaction between students and the instructor are Minute Papers (more info)and ConcepTests (more info) . Student may either work together to answer these questions, or individual responses may be graded. Whatever method is chosen, long term incremental assesssment of

interactive lectures will make it more likely that students will have had repeated opportunities to interact with and learn essential geoscience concepts.

Resources
Addressing the Challenges of Inquiry-Based Learning through Technology and Curriculum Design. Edelson et al., 1999 The authors explore the challenges of implementing inquiry-based learning through a program of research on the use of scientific visualization technologies to support inquiry-based learning in the geosciences. (citation and description)

Learning-for-Use: A Framework for the Design of TechnologySupported inquiry Activities. Edelson, 2001 This article discusses a way to integrate the teaching of content with the teaching of process, which have in the past been seen as competing priorities. General guidelines are presented for the design of inquiry activities that support content learning taking advantage of modern technologies. (citation and description)

Effects of Student-Generated Diagrams versus StudentGenerated Summaries on Conceptual Understanding of Causal and Dynamic Knowledge in Plate Tectonics. Gobert and Clement, 1999 This article examines the beneficial effects of student-generated diagrams as assessment tools versus student-generated summaries on conceptual understanding on the topic of plate tectonics. (citation and description)

Pre-/Post-Knowledge Assessment of an Earth Science Course for Elementary/Middle School Education Majors. Gosselin and MaklemHurst, 2002 This article describes a course for elementary and middle school education majors at the University of Nebraska-Lincoln. The authors of the study developed an assessment instrument that allowed them to determine the success of this class in improving the content knowledge of the students and showed an average gain of 30%. (citation and description)

Assessment and Active Learning Strategies for Introductory Geology Courses. McConnell et al. (2003) This article describes several techniques to promote active learning in the classroom and compares classes taught using these with those using traditional lectures. General education Earth Science classes was evaluated using formative assessment exercises conducted by students in groups. (Full Text Online)

Active-Learning Methods to Improve Student Performance and Scientific Interest in a Large Introductory Oceanography Class. Yuretich et al., 2001 This article in the Journal of Geoscience Education provides information about a study that was conducted at the University of Massachusetts. A 600-student oceanography class was transformed by modifying lectures to include cooperative learning via interactive in-class exercises and directed discussion. Assessments were redesigned as "two-stage" exams with a significant collaborative component. Results of the student surveys, course evaluations, and exam

performance demonstrate that learning of the subject under these conditions has improved. (Full Text Online) Innovation in Large Lectures - Teaching for Active Learning. [Ebert-May, Brewer and Allred, 1997] This article from BioScience discusses studies conducted by Northern Arizona University and the University of Montana that were designed to improve biological literacy among education majors. The study highlights the importance of active, inquiry-based learning. (citation and description)

The Use of a Mock Environment Summit to Support Learning about Global Climate Change. [Gautier and Rebich, 2005] This article in the Journal of Geoscience Education discusses why a learner-centered environment is particularly suitable for earth system science learning, and how it provides students with motivation and the opportunity to understand this complex area of scientific inquiry. Through a course that explores global change from both and earth science and human perspective, students use role playing, writings, class discussions, and presentations to develop and negotiate a mock international environmental agreement. Rubrics are used both to assess student learning and to provide feedback to students about their work. (citation and description)

Scientific Teaching. [Handelsman et al., 2004] This article from Science offers a guide to learning how to do scientific teaching. The guide discusses the need for active learning strategies to engage students in the process of science as well as the use teaching methods that have been systematically tested and shown to reach a diverse audience of students. Topics include implementing changes in lectures, students as scientists, and how universities can promote change. (citation and description)

Research Methodologies in Science Education: Mental Models and Cognition in Education.[Libarkin, Beilfuss and Kurdziel, 2003] This article in the Journal of Geoscience Education identifies and discusses four types of cognitive models: nave, unstable, and conceptual mental models, and conceptual frameworks. The author includes illustrations to show these different types of cognitive models. (Full Text Online)

The Two Paradigms of Education and the Peer Review of Teaching. [McManus, 2001] This article in the Journal of Geoscience Education provides information about two models of education, the Teaching-Centered Paradigm and the Learning-Centered Paradigm. The paper examines how to align the appropriate tools for peer review with the teaching implications of paradigm choice. (Full Text Online)

Active Learning in Secondary and College Science Classrooms: A Working Model for Helping the Learner to Learn. [Michael and Modell, 2003] This book by Joel Michael and Harold Modell is designed for professionals interested an active learning approach to teaching students. The main topics covered in this book are how to build the foundation for

active learning, roles for the teacher in creating an active learning environment and creating active learning environments.(citation and description) Assessment Essentials: Planning, Implementing, and Improving Assessment in Higher Education. [Palomba and Banta, 1999] This book by Catherine Palomba and Trudy Banta is a step-by-step guide that provides the most current practices for developing assessment programs on college and university campuses. Each chapter of the book addresses a specific aspect of assessment and is designed to walk users through various steps of the assessment process. The authors describe effective assessment programs and offer a thorough review of the most up-to-date practices in the field. (citation and description)

The Case for a Cooperative Studio Classroom: Teaching Petrology in a Different Way.[Perkins, 2005] This article in the Journal of Geoscience Education discusses the effects on student learning when a petrology course is changed from a lab-lecture format to one that emphasizes studio and cooperative learning. Changes were instigated in order to emphasize the development higher order thinking skills rather than traditional knowledge-based learning. The article illustrates assessment stratgies by comparing a cooperative studio classroom with a more traditional one. After one semester, an assessment revealed that students like this format and believe they learn more than in a traditional course setting. (Full Text Online)

Studio vs. Interactive Lecture Demonstration--Effects on Student Learning. [Roy, 2003] This article from Bioscene compares two methods of teaching a course in genetics and evolution. The first method is studio teaching involving teamwork by students, hands-on exercises, and minimal lecturing. The second is an interactive lecture demonstration method. The study uses pre- and post-testing of basic concepts to evaluate the effectiveness of each method. (citation and description)

A Data Rich Exercise for Discovering Plate Boundary Processes. [Sawyer et al., 2005] This article in the Journal of Geoscience Education describes a classroom exercise based on four world maps containing earthquake, volcano, topographical and seafloor age data. Students participate in this exercise by using a "jigsaw" approach, in which they break into four groups and become specialists on one of the map types. After being organized into new groups with one specialist from each map represented, the groups present their data from the class. This exercise (assessment tool) has shown that students come away with knowledge of the key features of each type of plate boundary and a sense of why it looks the way it does. (Full Text Online)

A Student-Centered Project for Earth System History. [Teed, 2005] This article in the Journal of Geoscience Education describes a project in which students play the role of class lecturer. The purpose of the exercise

is to reduce instructor lecture time and increase student involvement in learning about earth system history. Each student researches a specific era of geologic history and then presents their findings to the class, as well as providing a written summary and bibliography of resources used in their research. Rubrics for assessing the student presentations are included in the article. (Full Text Online) Workshop Biology: Demonstrating the Effectiveness of Active Learning in an Introductory Biology Course. [Udovic et al., 2002] This article in BioScience describes the University of Oregon? Workshop Biology. s The program was created to engage students in open-ended, interactive activities and projects designed to develop their understanding of essential biological concepts, the process of scientific discovery, and their critical thinking skills. Workshop students displayed more improvement in conceptual learning and understanding of scientific reasoning, a greater appreciation of science and its role in their lives, and greater motivation and involvement in learning activities than did students in a comparison course taught in a more traditional, passive style. (citation and description)