MULTIMODAL TEXTS

Multimodal texts combine two or more modes such as written language, spoken language, visual (still and
moving image), audio, gestural, and spatial meaning (The New London Group, 2000; Cope and Kalantzis,
2009). Creating digital multimodal texts involves use of communication technologies, however, multimodal
texts can also be paper based, or live performances.

Examples of texts to create

Below are examples of different forms of texts students might create in the classroom. The complexity of
creating texts increases proportionately with the number of modes involved and the relationships between
the various semiotic, or meaning making, systems in a text, as well as use of more complicated digital
technologies.

Simple multimodal texts include comics/graphic novels, picture books, newspapers, brochures, print
advertisements, posters, storyboards, digital slide presentations (e.g. PowerPoint), e-posters, e-books, and
social media.

Meaning is conveyed to the reader through varying combinations of written language, visual, gestural, and
spatial modes.

Podcasts are also simple to produce, involving combinations of spoken language, and audio modes.

Live multimodal texts include dance, performance, oral storytelling, and presentations. Meaning is
conveyed through combinations of various modes such as gestural, spatial, audio, and oral language.

Complex digital multimodal texts include: live action films, animations, digital stories, web pages, book
trailers, documentaries, music videos. Meaning is conveyed through dynamic combinations of various
modes across written and spoken language, visual (still and moving image), audio, gesture (acting), and
spatial semiotic resources. Producing these texts also requires skills with more sophisticated digital
communication technologies.

Teaching creating multimodal texts: production stages

Teaching creating multimodal texts is based on teaching writing, extended to teaching students how to
produce short, purposeful, and engaging texts in different forms and media formats.

Students need to develop increasing control over the different semiotic contributions of each of the modes
deployed, and at the same time, attend to creatively combining modes into a meaningful whole (Hull, 2005,
p.234). In addition, pedagogic attention to any technological requirements is also essential.

Teaching creating multimodal texts can be structured in stages around the film production approach. This
includes pre-production, production, and post-production.

Pre-production

The pre-production stage includes consideration of the topic, the purpose, the audience and the context.
The story/content is drafted and organised, and manageable boundaries are established. This includes
setting limits to number of pages in a picture book, or slides in a PowerPoint, or time limits for digital
productions – 30 to 90 seconds is long enough for novice podcasts, film or animation productions.

The production process is planned. This might include writing a story outline which provides brief
information about who, what, where, and when; a script which includes information about the text
participants (characters or subjects), dialogue, action, sound effects, and music; and preparing a storyboard
to scope the visual design of the text – what is to be shown and how it will be seen.

The production stage

The production stage is where the text is composed or produced. Production can be a simple process using
familiar tools and resources or can involve learning to use more complex digital tools including cameras,
recording equipment, or digital applications and software.

Complex media production processes can be simplified for the literacy classroom. For example, a simplified
approach to creating live action films involves an ‘in-camera’ edit. This requires the whole sequence to be
carefully planned first. Beginning with the title shot, the film is shot in sequence, shot by shot, pausing the
camera between shots. Sound effects and additional information must be recorded at the same time as the
action. Following the final shot, the film is finished, and there is no further editing or post-production. The
same approach can be used recording simple podcasts, as an ‘in-microphone’ edit.

In contrast, a conventional approach to filmmaking/podcast production involves filming or recording the

content in segments first, and then putting the final text together through post production.

Post-production stage

In the post-production stage filmed shots or recorded audio segments, are edited using a digital editing
program to remove sections, to order information, and to add in introductions, titles, music, visual and sound
effects.

(Source: https://www.education.vic.gov.au/school/teachers/teachingresources/discipline/english/literacy
/multimodal/Pages/createmultimodal.aspx)

Creating Multimodal Texts

Storytelling in any format is about making meaning. While the essence of the stories we tell may remain
the same, the ways in which we can now share these stories have changed dramatically with the
development of digital communication technologies. Access to simple, easy to use media production tools
and resources in conjunction with the potential for immediate and universal online publication has significant
implications for literacy thinking and practice.

What does creating multimodal texts mean?

Creating is defined in the Australian Curriculum as ‘the development and/or production of spoken, written
or multimodal texts in print or digital forms’ and is an embedded literacy expectation across all disciplines.

Multimodal is defined in the Australian Curriculum as the strategic use of ‘two or more communication
modes‘ to make meaning, for example image, gesture, music, spoken language, and written language.

What is a multimodal text?

While the development of multimodal literacy is strongly associated with the growth of digital communication
technologies, multimodal is not synonymous with digital. The choice of media for multimodal text creation
is therefore always an important consideration.
A multimodal text can be paper – such as books, comics, posters.

A multimodal text can be digital – from slide presentations, e-books, blogs, e-posters, web pages, and
social media, through to animation, film and video games.

A multimodal text can be live – a performance or an event.

And, a multimodal text can be transmedia– where the story is told using ‘multiple delivery
channels’ through a combination of media platforms, for example, book, comic, magazine, film, web series,
and video game mediums all working as part of the same story. Transmedia is a contested term and Henry
Jenkins is worth reading for more background. Jenkins argues that transmedia is more than just multiple
media platforms, it is about the logical relations between these media extensions which seek to add
something to the story as it moves from one medium to another, not just adaptation or retelling. Transmedia
enables the further development of the story world through each new medium; for example offering a back
story, a prequel, additional ‘episodes’, or further insight into characters and plot elements. (Jenkins, 2011).
It also can require a more complex production process.

Development of multimodal literacy knowledge and skills

To enable our students to effectively design and communicate meaning through such rich and potentially
complex texts, we need to extend their (and along the way, our own) multimodal literacy knowledge and
skills. Skilled multimodal composition requires new literacy design skills and knowledge to enable students
to make informed choices within and across the available communication modes to effectively construct
meaning.

Creating a multimodal text, a digital animation for example, is a complex meaning design process requiring
the strategic orchestration of a combination of modes such as image, movement, sound, spatial design,
gesture, and language. The process of constructing such texts is also truly a cross-disciplinary literacy
process, drawing on digital information technologies and The Arts (media, music, drama, visual arts,
design) to bring meaning to life.

Print based multimodal texts include comics, picture storybooks, graphic novels; and posters,
newspapers and brochures.

Digital multimodal texts include slide presentations, animation, book trailers, digital storytelling, live
action filmmaking, music videos, ‘born digital’ storytelling, and various web texts and social media. The
level of digital technology requirements range from very simple options such as slide presentations through
to complex, sophisticated forms requiring a higher level of technical and digital media skills. The choice is
yours depending on your skill and experience, level of confidence, and the resources and tools available to
you.

These examples of different types of student multimodal composition are provided as ideas and starting
points; and may also provide models for introducing new forms of ‘writing’ to your students.

The technical construction of digital multimodal texts is always a significant consideration for teachers.
Practical information about communication technology resources and digital media tools is provided where
possible to support the successful implementation of multimodal authoring in the literacy classroom.

(Source: https://creatingmultimodaltexts.com/)