MULTILITERACIES

Designing Assessments
A Multiliteracies Approach
GLORI A E. JACOBS

lthough the concept of assessment in education is broad, the mandates of No Child Left Behind, Race to the Top, and programs such as Response to Invention have been reinforcing the old basics (Kalantzis, Cope, & Harvey, 2003) by narrowly defining assessment as the measurement of discrete cognitive skills used for decoding and comprehending alphabetic texts (Cope, Kalantzis, McCarthey, Vojak, & Kline, 2011). Such a narrow view of assessment fails to help teachers develop deeper understandings of students multiliterate skills. As the New London Group (2000) argued, multiliteracies are integral to a diverse, multimodal, and information-based world. However, it is not enough to provide opportunities for youths to engage in multiliteracies; assessment of multiliteracies must also be meaningfully integrated into instruction. Therefore, in this last entry of the Multiliteracies: Production and Consumption department, I turn attention to the possibilities of assessment within a multiliterate world. There is a wealth of theoretical and empirical literature that explores assessment for multiliteracies and multimodal texts. Scholars in composition studies (Adsanatham, 2012; Peterson, Botelho, Jang, & Kerekes, 2007), the new media studies (Ball, 2006), the new literacies (Callow, 2008; Hansford & Adlington, 2008), multiliteracies (Cope & Kalantzis, 2006; Cope et al., 2011; Kalantzis et al., 2003), and multimodal research (Kress & Van Leeuwen, 2001) have contributed to a vision of assessment for multiliteracies.

Although coming from different disciplinary traditions, the literature generally agrees that tools are needed for assessing multiliteracies and youths understanding and creation of multimodal texts. The table contains an overview of multiliteracies approaches to assessment, and the remainder of this article explores those suggestions of what a multiliteracies approach to assessment can look like in an instructional setting already overly filled with traditional literacy assessments.

Designing Multimodal Assessments

The move from assessment as currently practiced in schools to a multiliteracies assessment is not simply a matter of identifying a set of discrete skills, nor is it a matter of creating rubrics. A pedagogy of multiliteracies is philosophically different from the cognitive approach that dominates assessment and requires a different way of thinking about texts and assessment. Kalantzis et al. (2003) called this the new basics (p. 16). According to Cope and Kalantzis (2006), the new basics are about a kind of learning which facilitates an active engagement with new and unfamiliar kinds of text, without arousing a sense of alienation and exclusion (p. 37). Thus, assessment techniques need to be altered, in many cases quite radically, to promote new learning and to measure more accurately the skills required for success in the twenty-first century (Kalantzis et al., 2003, p. 16). Kalantzis et al. (2003) identified 10 key skills needed for a person to be successful within an information-based society: 1. Autonomy and self-direction 2. Flexibility 3. Problem-solving skills
623

The department editor welcomes reader comments. Gloria E. Jacobs is an independent scholar in Portland, Oregon, USA. She is a former associate professor of literacy at St. John Fisher College, Rochester, New York, USA. Her research focuses on the new literacies of adolescents and the implications of those practices for education. E-mail glojacobs@gmail.com.

Journal of Adolescent & Adult Literacy 56 (8) May 2013 doi:10.1002 /JA AL.189 2013 International Reading Association (pp. 623 626)

MULTILITERACIES
TABLE Overview of Multiliteracies Assessment Approaches
Composition studies (Adsanatham, 2012) Qualities assessed Rhetorical elements The relationship between modes Design choices Coherence Argument strength New literacies (Callow, 2008; Hansford & Adlington, 2008) Affective/social purpose Compositional (holistic structure, technical skills, grammatical components) Critical Multiliteracies (Cope & Kalantzis, 2006; Cope et al., 2011; Kalantzis et al., 2003) Autonomy and self-direction Flexibility Problem-solving skills Multiple strategies for tackling a task Flexible solutionsorientation to knowledge Collaborative and communicative Ability to work productively with linguistic and cultural diversity Intelligent in multiple ways Broadly knowledgeable Ability to engage with different interpretative frameworks and contexts

Assessment approach

Inductiveheuristics/ rubrics based on student identification of standards used for evaluating multimodal works.

Performative based on qualities identified by theory. Includes project assessment performance assessment quantification of collaborative skills portfolio assessment

Performative based on qualities identified by theory. Includes project assessment performance assessment group assessment to measure collaborative skills portfolio assessment

4. Multiple strategies for tackling a task 5. A flexible solutions-orientation to knowledge 6. Collaborative and communicative
5 6 (8 ) M AY 2013

3. How are the meanings organized textually? How and to what extent do they cohere? (organizational) 4. How do the meanings fit into the larger world of meaning? (contextual) 5. Whose interests are the meanings positioned to serve? (ideological) The elements described here provide possible frameworks for assessing performative aspects of text use and creation as well as the multimodal texts used within a multiliterate environment. There are other approaches for assessing how students engage with multimodal texts that acknowledge the social and contextual nature of multiliteracies but also include an operational or skills-based perspective. For instance, Callow (2008) suggested the analysis of three dimensions of text use and creation: the affective, the compositional, and the critical. The affective dimension, according to Callow, includes expressions of enjoyment as students participate in an activity. The compositional includes concepts such as

7. Able to work productively with linguistic and cultural diversity 8. Intelligent in multiple ways 9. Broadly knowledgeable 10. Able to engage with the different interpretative frameworks and contexts of specific information In addition to the 10 performative skills identified by Kalantzis et al. (2003), Cope and Kalantzis (2006) identified five questions that can be asked about multimodal texts: 1. What do the (representational) meanings refer to?

JO URN AL OF AD OL ES CEN T & ADULT L ITE RACY

624

2. How do the meanings connect people? (social)

actions, symbols, shot length, angles, gaze, color, layout, salience, lines, and vectors. The critical includes how an image positions the viewer to think or feel a particular way. Similarly, Hansford and Adlington (2008) argued that digital texts should be assessed for their social purpose, holistic structure, grammatical components, and the technical skills required to create them. These approaches hold promise as the framework for heuristics that teachers can use to guide their students when consuming or producing multimodal texts. The next section provides descriptions of several multiliteracies assessments.

Implementing Multiliteracies Assessments

Scholars have suggested several approaches for multiliteracies assessments. Callow (2008) and Kalantzis et al. (2003) argued for a performative approach to assessment. Callows show me (p. 617) approach to assessment includes (a) project assessment of in-depth learning tasks, (b) performance assessment of how students complete tasks, (c) quantification of collaborative skills, and (d) ongoing documentation through portfolios. Callow (2008) suggested that multiliteracies assessment also should include focused and purposeful student talk in assessment (p. 619), affective engagement with images, inclusion of students in the creation or manipulation of visual texts, and explicit compositional concepts and development of a metalanguage. Similarly, Kalantzis et al. (2003) argued for (a) project assessment to measure in-depth tasks, (b) performance assessment to measure the creative process, (c) group assessment to measure collaborative skills, and (d) portfolio assessment to document the students body of works. Whereas Kalantzis et al. (2003) and Callow (2008) approached development of a multiliteracies assessment framework through the creation of a priori categories based on theory and research, Adsanatham (2012) approached multimodal assessment through an inductive process. In Adsanathams class, the students first engaged in critiquing new media texts as a way to learn what rhetorical elements make up a text and to identify the relationship between modes, design choices, coherence, and argument strength. However, Adsanathams (2012) approach also requires a heuristic and grading as well as an identification of standards used for evaluating multimodal

works. Toward this end, Adsanatham suggested (a) frequent rhetorical viewing and analysis exercises; (b) evaluation criteria that are flexible, adaptable, and open to revision; (c) continuous reflection; (d) student involvement in the design of criteria; and (e) flexibility about how students repurpose criteria for their own purposes. These approaches are obviously time consuming and work intensive. As such, they may seem unrealistic to a teacher already faced with a heavy set of standardized assessments that must be implemented or prepared for. A multiliteracies assessment should not be an add-on but rather an integral part of an existing pedagogy of multiliteracies in which the old basics and new basics are taught in conjunction with each other. Even if a teacher strives to implement a pedagogy of multiliteracies and integrate multimodal text into instruction, assessing their instructional efforts and student work may be problematic. Honan (2010) suggested that teachers struggle with the assessment of multiliteracy projects because the operation of the technology is often foremost in teachers discourse rather than the thinking or creative processes engendered by the technology. Honan posited that the focus on the operational aspects of technology over the creative might be associated with the pressures of standardized testing:
These teachers [the ones in Honans study] know that accountability measures in relation to literacy learning, including nationwide standardised testing, emphasise the skills required to break the code of texts. They also know that the operational skills being learned by their students are as easily identified, measured, and assessed as these literacy skills. In a climate where this codebreaking is seen to be of paramount value in classrooms, it is no wonder that teachers emphasise this area in their teaching surrounding new technologies. (pp. 186187)

Honan (2010) found that the only connection teachers made between literacy teaching and learning and the use of new technologies was the drafting, planning and writing of a print-based text that is then transferred into a digital space (p. 188). Thus, multimodal texts become a supplement or extension of existing ways of using texts rather than opening up opportunities for engaging in the new ways of thinking afforded by the new literacies (Cope & Kalantzis, 2006; Cope et al., 2011; Kalantzis et al., 2003).

625

Designing Assessments: A Multiliteracies Approach

MULTILITERACIES
Conclusion
The prospect of adding assessment of multiliteracies to an educational system crowded with assessments may be daunting or seem undesirable. However, just as multiliteracies require a rethinking of what constitutes literacy and the role of literacy in everyday life, so, too, does implementing an assessment of multiliteracies. The approaches suggested by Cope, Kalantzis, and others are not about creating rubrics for a curriculum. I suggest that the core of a multiliteracies assessment is the core of any meaningful assessment. It is about watching and noticing what students are doing and then using that information to guide the students toward new skills and knowledge. Good teachers do this already; however, multiliteracies assessment asks for a different set of questions. Once a teacher is routinely asking questions that build on a pedagogy of multiliteracies, then a new way of envisioning assessment will emerge.
References
Adsanatham, C. (2012). Integrating assessment and instruction: Using student-generated grading criteria to evaluate multimodal digital projects. Computers and Composition, 29(2), 152174. Ball, C.E. (2006). Designerlyreaderly: Re-assessing multimodal and new media rubrics for use in writing studies. Convergence: The International Journal of Research into New Media Technologies, 12(4), 393412. doi:10.1177/1354856506068366 Callow, J. (2008). Show me: Principles for assessing students visual literacy. The Reading Teacher, 61(8), 616626. doi:10.1598/ RT.61.8.3 Cope, B., & Kalantzis, M. (2006). From literacy to multiliteracies: Learning to mean in the new communications environment. English Studies in Africa , 49 (1), 2345. doi:10.1080/00138390608691342 Cope, B., Kalantzis, M., McCarthey, S., Vojak, C., & Kline, S. (2011). Technology-mediated writing assessments: Principles and processes. Computers and Composition, 28 (2), 7996. doi:10.1016/j.compcom.2011.04.007 Hansford, D., & Adlington, R. (2008). Digital spaces and young peoples online authoring: Challenges for teachers. Australian Journal of Language and Literacy, 32 (1), 5568. Honan, E. (2010). Mapping discourses in teachers talk about using digital texts in classrooms. Discourse: Studies in the Cultural Politics of Education , 31 (2), 179-193. doi:10.1080/01596301003679701 Kalantzis, M., Cope, B., & Harvey, A. (2003). Assessing multiliteracies and the new basics. Assessment in Education: Principles, Policy & Practice, 10 (1), 1526. Kress, G., & Van Leeuwen, T. (2001). Multimodal discourse: The modes and media of contemporary communication. London: Arnold. New London Group. (2000). A pedagogy of multiliteracies. In B. Cope & M. Kalantzis (Eds.), Multiliteracies: Literacy learning and the design of social futures (pp. 937). London: Routledge. Peterson, S.S., Botelho, M.J., Jang, E., & Kerekes, J. (2007). Writing assessment: What would multiliteracies teachers do? Literacy Learning: The Middle Years, 15 (1), 2935.

626

JO URN AL OF AD OL ES CEN T & ADULT L ITE RACY

5 6 (8 ) M AY 2013