Using the Medium Effectively: Rubrics

Dan Feinberg SUNY Learning Network Summer 2012

About rubrics Tips for using Examples Create own Grade with them

What is a rubric?
A scoring tool that lists the criteria for a piece of work and articulates gradations of quality for each criterion.

Formative Assessment
Have you used rubrics?

Have you created your own rubrics?

Have you used the ANGEL/Bb rubric tool?

Why Use a Rubric?

Increase an assessments validity
Content validity (skills measured represent the skills in the broader domainwould an expert agree?) Construct validity (criteria reflects the knowledge/skills you are trying to measure) Face validity (assessment instrument appears to be valid to users)

Why Use a Rubric?

Increase an assessments reliability
Intra-rater reliability (an individual rater is consistent in his/her scoring) Inter-rater reliability (different raters are consistent in scoring the same item)

Why Use a Rubric

Helps establish clear criteria
Reduce bias
Prevent grading drift Increase transparency in the grading process

Why Use a Rubric

Helps students assume responsibility for their own learning
Able to set goals

Accurately self and peer assess

Helps students know what counts

Why Use a Rubric

Provides rich feedback
Students can see exactly where they lost credit and why
Should still be accompanied with narrative

Disadvantages
Time consuming to create Convergent production

Types of Rubrics
Holistic
Assess students' work globally "as a whole Have fewer details to analyze, and are easier to integrate into the schema for students

Do not provide detailed information about students' performance in specified areas within the assignment

Holistic Rubric

Types of Rubrics
Analytic
Break down the various objectives of the final product into specified components Evaluate individual components independently Possess extra details that allow multiple grades to emphasize the same criteria

Analytic Rubric

Analytic
Useful for assessing each criterion separately Better at weighting criteria Greater level of specificity

vs.

Holistic
Useful for quick, gross judgments Often used when criteria cannot be disentangled (writing) More flexible

Creating a Rubric
1. Define the purpose and goals of your assignment 2. Choose your rubric type
1. 2. 1. 2. 3. 4. Holistic Analytic observable and measurable important and essential distinct from other criteria phrased in precise, unambiguous language

3. Define the criteria

4. Design the rating scale 5. Write the descriptors for each scale point

Creating a Rubric
Define the purpose and goals of your assignment
What are your learning outcomes? How will students demonstrate they have learned these outcomes? Why are you assigning this activity?

Creating a Rubric
Choose your rubric type

Creating a Rubric
Define the criteria
Will be used to guide student performance Observable and measurable Important and essential Distinct from other criteria Phrased in precise, unambiguous language Most rubrics have 4-8 criteria

Creating a Rubric
Design the rating scale
Example 1: Example 2: Performance Met standards Performance Did Not Meet Standards Items Present Items Not Present

Example 1: Example 2:

Beginning Fail

Developing Fair

Competent Pass

Accomplished Exceed

Example 3:

Never

Sometimes

Usually

Always

More is NOT always better!

Creating a Rubric
Write the descriptors for each scale point
Describe observable and measurable behavior Use parallel language across the scale
The degree changes across the scale, not the criteria

Indicate the degree to which standards are met (amount, frequency, or intensity)

Hint: Use an Even Number of Levels

Two, four, or six performance levels works best With an odd number of levels, the middle level becomes a catch-all. Having an odd number requires more precise judgments in the middle ranges.

Hint: Arrange Achievement Levels from High to Low

Students will first read the description of an exemplary performance

Hint: Limit the Number of Criteria and Achievement Levels

Too many dimensions may lead to a rubric that is hard for students to understand and faculty to efficiently use.

Hint: Use Consistent Descriptors

4 Task Frequency Accuracy All Always No errors 3 Most Usually Few errors 2 Some Some of the time Some errors 1 Very few or none Rarely or never Frequent errors

Understanding Range

Fully Broad

Adequately Adequate

Partially Limited

Minimally Very Limited

Sources
Phoenix College. (2008). Creating Rubrics: A guide to development and use. Retrieved July 16, 2012 from http://www.slideshare.net/chedisky/creating-rubrics-331194 Center for Advanced Research and Language Acquisition. University of Minnesota, n.d. Web. 16 July 2012. <http://www.carla.umn.edu/assessment/vac/Evaluation/p_2.ht ml>. O'Reilly, Lisa and Tom Cyr. Creating a Rubric: An online tutorial for faculty. University of Colorado Denver, 2006. Web. 16 July 2012. <http://www.ucdenver.edu/faculty_staff/faculty/center-forfaculty-development/Documents/Tutorials/Rubrics/index.htm>.