Continued Absences:
Multimodal Texts and 21st Century Literacy Instruction

The demands of today’s digital media and visual texts continue to require complex new ways of coding and decoding image–text relations. Indeed, the way we read is changing , but you typically wouldn’t know it to look at reading instruction in most public secondary schools. Texts considered “high quality” and “complex” continue to be defined by the Western, canonical tradition, and are predominantly narrative and print-centric. It would seem Freytag’s pyramid still reigns supreme in secondary reading instruction ( Hundley, 2008 ).

In the late 1990s to early 2000s, we started hearing the buzz words “multimodal literacies” in our professional communities. As young teacher educators, we weren’t exactly sure what these literacies were, or what they looked like, but we were curious. We attended sessions at NCTE, read theoretical literature (research on adolescents’ multimodal reading processes remains scarce), and appreciated NCTE’s publication of the position statement on “Multimodal Literacies” ( National Council of Teachers of English, 2005 ). We realized we’d learned a new name for the sophisticated kinds of literacy practices that mark a predominantly visual and increasingly digital culture. We grew inspired; maybe NCTE’s formal recognition of the new social contexts that mark the 21st century was a sign that “multimodal literacies” would soon be taken up in secondary English/language arts classrooms.

But, alas, two decades of disastrous federal reading policy (the Common Core reading standards being the most recent manifestation of this) have proven otherwise. The “career- and college-ready” definitions underlying the current federal “Race to the Top” initiative don’t acknowledge the sophisticated social communication contexts in which today’s young people are immersed—contexts that require new literacy practices (e.g., the ability to multitask, to be active readers and communicators, and to negotiate multiple modes of expression simultaneously). The “digital disconnect” that exists between today’s English teachers and adolescents grows larger. And standards-writers and school literacy leaders refuse to acknowledge that 21st century texts—multimodal texts, postmodern texts, “radical change” texts ( Kress & van Leeuwen, 2006 ; Dresang, 2008 ; Groenke & Youngquist, 2011 )— require an entirely new form of reading and, consequently, a place in the ELA classroom.

But we remain hopeful, and continue to seek out opportunities to interact with teens about the ways they engage with multimodal texts. In what follows, we offer a definition of “multimodal texts,” highlight several different multimodal text experiences, and consider the new ways of reading (and teaching) that multimodal texts require. We do so in an initial attempt to show that taking up multimodal literacies in the English classroom puts “complex texts” at the center of literacy instruction, and helps adolescents learn to navigate the sophisticated “real-world” reading processes and literacy practices required in the 21st century.

What Are Multimodal Texts?

Multimodal texts are texts that include multiple modes (or genres) of representation, with combined elements of print, visual images, and design. As Kress (2008) and other literacy theorists have suggested, multi-modal texts decentralize the written word; print is no longer the central mode of communication. As such, readers must consider the juxtaposition of multiple modes of communication, as graphics, images, and other modes “extend, and often replace, the printed word as the primary carrier of meaning” ( Hassett & Curwood, 2009 , p. 271).

Dresang (2008) suggests that multimodal texts must be considered for their digital design and intertextual elements. She explains that the juxtaposition of text and image in multimodal texts “requires, or at least promotes, a hypertextual approach to thinking and reading” as they “incorporate references to or imitations of a preexisting content in another context, often in subtle ways” ( Dresang, 2008 , p. 42). As Hassett and Curwood (2009) explain, multimodal texts “include various pathways to follow, parallel displays of information, [and] extensive cross-referencing elements” (p. 271). We describe some popular young adult novels that exemplify these elements below.

What Do Multimodal Texts Look Like?

We see many of these elements in popular young adult novels being published today. Walter Dean Myers’s (1999) Monster , a story about a young African American male who struggles with his own racial identity, could be considered multimodal, as the story of Steve Harmon is told through journal entries, a screen play, and black-and-white images. Navigating the multiple modes in Monster can be a dizzying experience, but Myers wants the reader to be involved, and doesn’t allow for too-easy conclusions about Steve ( Groenke & Youngquist, 2011 ). Sherman Alexie’s award-winning The Absolutely True Story of a Part-Time Indian (2009) could also be considered multimodal, as it draws on both traditional print and cartoon images to communicate meanings about the protagonist’s conflicted identity and feelings about life on the rez. Graphic novels have always been multimodal, and such works as Art Spiegelman’s Maus (1986), Gene Luen Yang’s American Born Chinese (2007), and Vera Brosgol’s Anya’s Ghost (2011) continue to stretch readers’ skills as they present complex narratives with complex graphics and text structures.

Patrick Carman’s popular Skeleton Creek (2009) series can also be considered multimodal as one character, Ryan, is depicted predominantly in print, while another character, Sarah, is depicted solely through digital video (which can only be accessed by reader via Internet). Readers must engage two very different modes of communication to “read” the two characters and fully realize the story. Carman has said he wrote the Skeleton Creek books to mimic the technological multitasking he sees today’s teens engaged in ( Groenke & Maples, 2010 ; Groenke, Bell, Allen, & Maples, 2011 ). Carman’s latest series, Trackers (first book published in 2010), is another multiplatform narrative where the printed text and a related website create the narrative.

Yet another multimodal text experience could include the Harry Potter franchise ( Rowling, 1997– 2007 ), which has gone well beyond the written words on the page (perhaps challenging the very definition of multimodality that we offer here). It is now possible to read the original seven books in the Harry Potter series. It is now possible to watch all eight film adaptations. It is now possible to travel in the real world to London and take Harry Potter tours to sites that were used in the film or that inspired Rowling’s writings. It is now possible to travel in the real world to Orlando, Florida, and experience “The Wizarding World of Harry Potter” at Universal Studios, drinking a butterbeer as you walk through Hogsmeade. It is now possible to purchase the merchandise, (official and unofficial) artifacts, memorabilia, clothes, etc. from the Harry Potter world—wands, scarves, patches, robes, magnets, mugs, rememberalls. It is now possible to be Harry Potter in many games, including a LEGO version of Harry Potter. It is now possible to log onto the Web and explore the virtual worlds of Harry Potter through fan fiction sites such as http://Mugglenet.com , Universal Studio’s online Wizarding World of Harry Potter , J. K. Rowling’s “personal” website, and, as of this writing, the imminent Pottermore , the latest online incarnation of the world of Harry Potter.

Rowling has said Pottermore is “something unique—an online reading experience unlike any other,” that will “continue the story of the young boy wizard” ( Rowling, 2011 ). Also available on the website will be the seven Potter novels as e-books and audiobooks in several different languages. The website will also reveal background details on characters and settings Rowling says she’s been “hoarding for years’” ( Cooke, 2011 ). Burbules (1998) explains that the practice of reading always takes place “within contexts and social relations . . . [and] significant differ- ences in those contexts and relations alter the practice” (p. 102). Certainly, new economic and social contexts mark the rise of the “convergence culture” that surrounds the Harry Potter franchise expansion. Henry Jenkins (2008) , a popular culture theorist, defines “convergence culture” as a 21st-century phenomenon in which content flows across multiple media platforms, multiple media industries (e.g., music, film) cooperate with each other, and media consumers “will go almost anywhere in search of the kinds of entertainment experiences they want” (p. 2).

Unlike the traditional multimodal experience of moving among print, text, and other elements within the same physically bounded artifact, the Harry Potter franchise extends the reading experience across space and time (Jenkins calls this “transmediation”). Jenkins explains that in a convergent media culture, participation across multiple platforms becomes a way of broadening and deepening the reading experience beyond the confines of the print-bound text. Through the physically real opportunities of Universal Studios or London, through the written fan fiction expansions of characters and stories, and through the online back stories and insights of J. K. Rowling’s websites, the Harry Potter story is told across multiple platforms— page, film, theme parks, locations, Internet, video games, and merchandise. The variety of offerings allows for further nonlinear experiences that combine to enrich and further the original story.

These nonlinear experiences certainly allow for a central characteristic of multimodal reading— “interactive narration,” where readers (not authors) choose how to engage with certain aspects of the text ( Hassett & Curwood, 2009 ). Rowling has created a multimodal world, if you will, where Potterphiles can practice this “interactive narration” and access the world of Harry Potter whenever and however they want. We feel fairly confident that Potter fans—as engaged, sophisticated readers—can and will navigate the multiple modalities of the Harry Potter franchise and make connections among them.

But we know, too, that some students who sit in our classrooms need help from teachers to navigate increasingly complicated textual worlds. Most recently, Susan talked about Deborah Wiles’s popular Countdown (2010) with 8th-grade students at a local middle school. Countdown is another multimodal novel (the first in a planned 1960s trilogy) that intersperses traditional narrative with biography and black-and-white documentary footage from the 1960s. The book’s narrative print mode, portrayed on white, grey-bordered pages, relays the story of Franny Chapman, an 11-year-old girl who is feuding with her best friend and trying to get the attention of the cute boy across the street. Interspersed with Franny’s narrative are evocative black-and-white, archival images that include well-known photographs (e.g., the bombing of the Freedom Riders bus), newspaper advertisements, public safety communications (e.g., fallout shelter signs), newspaper articles, song lyrics, quotes from prominent people (e.g., politicians, popular athletes), and movie posters. In addition, yet another mode of meaning includes four grey-colored, multipage biographical vignettes on President Harry Truman, the Kennedy family, Pete Seeger, and Fannie Lou Townsend Hamer.

Certainly one way to read the book (and the predominant way that Susan’s 8th-grade students read the text) is to privilege Franny’s story—the written mode—and push to the background the images and biographical vignettes. But reading Countdown multi-modally requires the reader to periodically decentralize the written language and foreground the other various visual and textual elements. In an interview with Vicky Smith for Kirkus Books, Wiles describes reading the book as a “weaving in and out” process where the images and biographical elements work to “give readers context” and “show large history, the large arc of history and within that, a person’s smaller story, which is really big, is everything to that character” ( Smith, 2011 ). Wiles further comments that the biographical elements work to provide:

. . . a chance to breathe as a person—you’re reading a really intense story—[and you can] take a step back, and see more about the history of that time. . . . Harry Truman could show us how we got into World War II, and how Vietnam’s little whisperings began and what the Cold War was, and then Pete Seeger was a way for me to talk about the Communist scare and McCarthyism, along with folk music and protest in that way, and pacifism. The Kennedys gave me a chance to say, is it Camelot or not? And Fannie Lou Hamer gave me a chance to talk about the Civil Rights movement, which was going on as 1962 unfolded and the Cuban missile crisis was here and so, I had this overall arcing large history through these biographies and Franny’s story weaving in and out, in and out, in and out. We teach history so often as a series of names and dates and places, but it’s so much more than that. ( Smith, 2011 )

Indeed, reading the text feels a lot like “weaving in and out,” as the juxtaposition among the narrative, images, and biographical elements continually moves us from the personally lived experience of Franny, to the extended, expanded experience of life in the 1960s. But Susan’s 8th graders ignored the pictures and the biographies, and instead focused solely on the narrative. Susan felt the students might have done so because they were more comfortable with the narrative element. Students remarked that the images were “distracting” and “confusing.” In essence, the students didn’t know what to do with the pictures, so they ignored them.

What Do We Do with These Multimodal Texts?

Susan’s experience is instructive for us, as we consider what multimodal reading instruction requires. As Hassett and Curwood (2009) explain, multimodal aspects of texts set forth new roles for readers, but they set forth new roles for teachers as well. It would seem, as Susan’s students’ experience with Countdown suggests, that students need help decentralizing the written word in multimodal texts. Are adolescents so used to traditional print-centric, linear narratives in school (we’re reminded again of Freytag’s pyramid), that it’s difficult for them to imagine reading in different ways? That texts can operate differently? Are adolescents savvy multi-modal readers outside of school, but so aware of the “game of school” that they struggle in school with texts that don’t fit the dominant textual mode? This would account for their focus on the written narrative of Countdown (a school-valued mode) and their rejection of the rest, unless directed there by a teacher.

We think these questions need attention by literacy researchers and teachers. We also think that if today’s adolescents are to grow confident as multimodal readers, they need opportunities in secondary English/language arts classrooms to discuss and reflect upon how texts are changing and becoming more multimodal—and thus, how reading itself is changing. Students also need opportunities to consider how pictures, images, graphics, sound—nonwritten modes of communication—express meaning, and they need opportunities to consider how such nonprint modes extend and expand upon meanings gleaned from written modes. How do multiple modes work together? Why are multiple modes used? What do different modes afford?

Larson and Marsh (2005) explain that many traditional roles for teachers remain necessary in a “new media age”: teacher as facilitator, teacher as instructor, teacher as model, and so on. But they also suggest that teachers will need to take on another role “in order to facilitate adolescents’ navigation of complex, multimodal, electronic worlds” (p. 73)—that of “co-constructor of knowledge.” Indeed, It was only when Susan started working collaboratively with the students, walking through the book together—talking about the story, looking at the pictures together, finding the songs on iTunes and singing along, Googling information about the Cuban Missile crisis that they didn’t know, and going back through the book-making connections— reading recursively , that the students (and Susan) started to feel like they were getting, as one student put it, “the whole story.” By the end of the book “walk-through,” the 8th graders had a new appreciation of the book and wanted to read it again. They gained some confidence in their own skills, it seemed, and wanted to know when Wiles’s second book in the trilogy would be out. They were ready to “try another book like that.”

Like Myers’s Monster , Carman’s Skeleton Creek series, and Wile’s Countdown , Rowling’s Harry Potter series and the forthcoming Pottermore extend what multimodal looks, sounds, and feels like; one can envision many opportunities for students and teachers to explore the multiple layers upon layers upon layers of narrative, insight, and experience through interactive readings. Whether reading about the Student Non-violent Coordinating Committee (SNCC) and the nuclear threat in the 1960s or Professor MacGonagall’s back story, students and teachers interact with not only the written text, but the multiple modes of these texts to create a richer, deeper understanding and reading experience.

As Leu (2000) has suggested, when students engage in multimodal experiences in the classroom, they become critically oriented to ever-evolving digital media and multimodal forms. From Susan’s initial experience, the small amount of research and theory available, and the growing opportunities with multimodal, multiplatform narratives, we do believe that teachers need to begin their multimodal textual explorations—their guiding, supporting, and collaborating—with their students in these new environments. Only then can students begin to acquire the conceptual bases needed to interpret complex text/image/design relationships. And only then will the students begin to acquire a 21st century literacy skill that expands and furthers their understanding of the changing way we read texts, especially multimodal ones.

Dr. Susan L. Groenke is an associate professor of English education at University of Tennessee. Groenke teaches courses in secondary methods, adolescent literature, and action research. Her current research interests center on adolescents’ reading preferences and processes. Groenke is coauthor (with Lisa Scherff) of Teaching YAL through Differentiated Instruction ( NCTE, 2010 ) and editor of NCTE’s English Leadership Quarterly . Groenke is a recent recipient of an ALAN Foundation research grant.

Dr. Robert Prickett is an assistant professor of English education at Winthrop University. Prickett has taught courses in secondary methods, educational technology, and adolescent literature. With a specialized focus on English Education, his most recent contributions to the field are bringing together areas of personal and professional interest: popular culture, media and technology, young adult literature, and teaching.

References

Burbules, N. (1998). Rhetorics of the web: Hyperreading and critical literacy. In I. Snyder (Ed.), Page to screen: Taking literacy into the electronic era (pp. 103–122). New York, NY: Routledge.

Cooke, S. V. G. (2011). “Pottermore” secrets revealed: J. K. Rowling’s new site is e-book meets interactive world. Retreived from http://newsfeed.time.com/2011/06/23/pottermore-secrets-revealed-j-k-rowlings-new-site-is-e-bookmeets-interactive-world/?artId=?contType=?chn= .

Dresang, E. T. (2008). Radical change theory, postmodernism, and contemporary picturebooks. In L. R. Sipe & S. Pantaleo (Eds.), Postmodern picturebooks: Play, parody, and self-referentiality . New York, NY: Routledge.

Groenke, S. L., Bell, R., Allen, E., & Maples, J. (2011). “What is this thing called a ‘vook’?” Using Skeleton Creek to transform students’ reading experiences in (and out) of school. English Journal, 100 (3), 105–108.

Groenke, S. L., & Maples, J. (2010). Young adult literature goes digital: Will teen reading ever be the same? The ALAN Review, 37 (3), 38–44.

Groenke, S. L., & Youngquist, M. (2011). Are we postmodern yet? Reading Monster with 9th graders. Journal of Adolescent and Adult Literacy, 54 , 505–513.

Hassett, D. D., & Curwood, J. S. (2009). Theories and practices of multimodal education: The instructional dynamics of picture books and primary classrooms. The Reading Teacher, 63 , 270–282.

Hundley, M. K. (2008). Hypertext and young adult literature. In A. O. Soter, M. Faust & T. Rogers (Eds.), Interpretative play: Using critical perspectives to teach young adult literature . Norwood, MA: Christopher-Gordon.

Jenkins, H. (2008). Convergence culture: Where old and new media collide . New York, NY: NYU Press.

Kress, G. (2008). “Literacy” in a multimodal environment of communication. In J. Flood, S. B. Heath, & D. Lapp (Eds.), Handbook of researchon teaching literacy through the communicative and visual arts (Vol. II, pp. 91–100). New York, NY: Erlbaum.

Kress, G., & van Leeuwen, T. (2006). Reading images: The grammar of visual design (2nd ed.). London: Routledge.

Larson, J., & Marsh, J. (2005). Making literacy real: Theories and practices for learning and teaching . Thousand Oaks, CA: Sage.

Leu, D. J., Jr. (2000). Literacy and technology: Deictic consequences for literacy education in an information age. In M. L. Kamil, P. B. Mosenthal, P. D. Pearson, & R. Barr (Eds.), Handbook of reading research (Vol. 3, pp. 743–770). Mahwah, NJ: Erlbaum.

National Council of Teachers of English. (2005). Position statement on multimodal literacies . Retrieved from http://www.ncte.org/positions/statements/multimodalliteracies .

Rowling, J. K. (2011). Exclusive announcement [YouTube video]. Retrieved September 29, 2011, from http://www.youtube .com/jkrowlingannounces .

Smith, V. (2011, May 10). Interview with Deborah Wiles. Retrieved from http://www.youtube.com/watch?v=rnRIdMXrleY .

Multimodal Texts Cited in the Text

Alexie, S. (2007). The absolutely true diary of a part-time Indian . New York, NY: Little, Brown.

Brosgol, V. (2011). Anya’s ghost . New York, NY: First Second.

Carman, P. (2009). Skeleton creek: Book 1 . New York, NY: Scholastic.

Carman, P. (2010). Trackers: Book 1 . New York, NY: Scholastic.

Myers, W. D. (1999). Monster . New York, NY: Amistad.

Rowling, J. K. (1997–2007). Harry Potter series. New York, NY: Scholastic.

Rowling, J.K. (2011, October). Pottermore. Retrieved from http://www.pottermore.com .

Spiegelman, A. (1986). Maus I: A survivor’s tale: My father bleeds history. New York, NY: Pantheon.

Wiles, D. (2010). Countdown . New York, NY: Scholastic.

Yang, G. L. (2007). American born Chinese. New York, NY: First Second.