Story Apps as Didactic Multimodal Resources for Multiliteracies Practices. A Case in Point / Aplicativos de histórias digitais multimodais como recursos didáticos para práticas de multiletramentos. Um caso em foco

Íris Susana Pires Pereira; Maitê Moraes Gil; Cristina Maria dos Santos Moreira da Silva Sylla

doi:10.17851/2237-2083.31.4.1916-1951

Story Apps as Didactic Multimodal Resources for Multiliteracies Practices. A Case in Point / Aplicativos de histórias digitais multimodais como recursos didáticos para práticas de multiletramentos. Um caso em foco

Íris Susana Pires Pereira, Maitê Moraes Gil, Cristina Maria dos Santos Moreira da Silva Sylla

Abstract

Abstract: This article aims to contribute to the understanding of the use of interactive story apps as didactic resources for fostering the learning and teaching of multiliteracies. It presents the multimodal discourse analysis of an interactive story app showing a dense net of intermodal meaning relations, such as convergence, complementarity, and divergence, in the representation of ideational and interpersonal meanings. The results allow the characterization of the story app as a complex multimodal ensemble and point to the need of a renewed readers and teacher’s agency in the pedagogical use of interactive story apps in the learning and teaching of multiliteracies. The article finishes by identifying the limitations of the study and some future developments.

Resumo: Este artigo tem como objetivo contribuir para a compreensão do uso de aplicativos interativos de histórias como recursos didáticos para promover a aprendizagem e o ensino de multiletramentos. Apresenta a análise do discurso multimodal de uma aplicação interativa de histórias, que mostra a densa rede de relações de significado intermodais, como a convergência, a complementaridade e a divergência, na representação de significados ideacionais e interpessoais. Os resultados permitem a caraterização desta aplicação de histórias como um conjunto multimodal complexo e apontam para a necessidade de uma renovada agência do leitor e do professor na utilização pedagógica de aplicações interativas de histórias na aprendizagem e no ensino dos multiletramentos. O artigo termina identificando as limitações do estudo e alguns desenvolvimentos futuros.

Keywords

multiliteracies; multimodality; story app; didactic resources; agency; multiletramentos; multimodalidade; aplicativos de histórias digitais interativas; recursos didáticos; agência.

Full Text:

PDF

References

AL-YAQOUT, G.; NIKOLAJVA, M. Re-conceptualising picturebook theory in the digital age. Barnelitterært Forskningstidsskrift, Oslo, v. 6, n. 1, p. 1-7, 2015. DOI: https://doi.org/10.3402/blft.v6.26971

BAL, M. Narratology. Introduction to the Theory of Narrative. 4. ed. London: University of Toronto Press, 2017.

BEZEMER, J.; KRESS, G. Changing text: A Social Semiotic Analysis of Textbooks. Designs for Learning, Stockholm, v. 3, n.1-2, p. 10-29, 2010. DOI: https://doi.org/10.16993/dfl.26

BEZEMER, J.; KRESS, G. The Textbook in a Changing Multimodal Landscape. In: KLUG, N.; STÖCKL, H. (eds.). Language in Multimodal Contexts. New York/Berlin: De Gruyter, 2016a. p. 476-498.

BEZEMER, J.; KRESS, G. Multimodality, learning and communication. A social semiotic frame. New York: Routledge, 2016b.

BUS, A.; TAKACS, Z.; KEGEL, C. Affordances and limitations of electronic storybooks for young children’s emergent literacy. Developmental Review, Fribourg, v. 35, p. 79-97, 2015. DOI: https://doi.org/10.1016/j.dr.2014.12.004

COPE, B.; KALANTZIS, M. Multiliteracies: New Literacies, New Learning. Pedagogies, Singapore, v. 4, n. 3, p. 164-195, 2009. DOI: http://doi.org/10.1080/15544800903076044

GEE, J. P. A personal retrospective on the New London Group and its formation. In: SERAFINI, F.; GEE, E. (eds.). Remixing multiliteracies. Theory and practice from New London to New times. New York: Teachers College Press, 2017. p. 19-30.

GIL, M.; PEREIRA, Í. S.; SYLLA, C. La construcción del posicionamiento lector en una narrativa digital interactiva. Una discusión a la luz de la teoría de las multiliteracidades. Cultura, Lenguaje Y Representación, Castellón de la Plana, v. 30, p. 79-106, 2023. DOI: https://doi.org/10.6035/clr.6773

GUALBERTO, C. L.; SANTOS, Z. B. Multimodalidade no contexto brasileiro: um estado de arte. DELTA: Documentação de Estudos em Linguística Teórica e Aplicada, São Paulo, v. 35, n. 2, p. 1-30, 2019. DOI: https://doi.org/10.1590/1678-460X2019350205

HAGEN, A. The Potential for Aesthetic Experience in a Literary App. An analysis of The Fantastic Flying Books of Mr. Morris Lessmore. Barnelitterært forskningstidsskrift, v. 11, n. 1, p. 1-10, 2020. DOI: https://doi.org/10.18261/issn.2000-7493-2020-01-02

HALLIDAY, M.; MATTHIESSEN, C. An Introduction to Functional Grammar. London: Routledge, 2004.

JEWITT, C. Multimodal, “Reading”, and “Writing” for the 21st Century. Discourse: studies in the cultural politics of education, London, v. 26, n. 3, p. 315–332, 2005. DOI: https://doi.org/10.1080/01596300500200011

JEWITT, C. Multimodality and Literacy in School Classrooms. Review of Research in Education, Washington, v. 32, n. 1, p. 241-267, 2008. DOI: https://doi.org/10.3102/0091732X07310586

KALANTZIS, M; COPE, B. New learning. Elements of a science of education. 2. ed. Cambridge: Cambridge University Press, 2012.

KRESS, G. Multimodality. A social semiotic approach to Contemporary Communication. New York: Routledge, 2010.

KRESS, G.; VAN LEEUWEN, T. Reading images. The grammar of visual design. 2. ed. New York: Routledge, 2006.

LOW, D. E.; PANDYA, J. Z. Issues of Validity, Subjectivity, and Reflexivity in Multimodal Literacy Research and Analysis. Journal of Language and Literacy Education, Athens, v. 15, n. 1, p. 1-22, 2019.

MARTIN, J. R.; ROSE, D. Working with Discourse. Meaning Beyond Clause. London and New York: Bloomsbury, 2007.

MENEGAZZI D.; SYLLA C. Touch to Read: Investigating the Readers’ Interaction Experience in Mediated Reading to Design Story Apps. In: BROOKS A., BROOKS E. (eds). Interactivity, Game Creation, Design, Learning, and Innovation. ArtsIT 2019, DLI 2019. Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering. Springer, 2020. p. 588-600. 328 v. DOI: https://doi.org/10.1007/978-3-030-53294-9_43

NEW LONDON GROUP. A Pedagogy of Multiliteracies: Designing Social Futures. Harvard Educational Review, v. 66, n. 1, p. 60-92, 1996.

NEW LONDON GROUP. A pedagogy of multiliteracies. In: COPE, B; KALANTZIS, M. (eds.), Multiliteracies. London: Routledge, 2000. p.19-37.

O’BRIEN, A. Using Focalisation Choices to Manipulate Audience Viewpoint in 3-D Animation Narratives: What Do Student Authors Need to Know? In: UNSWORTH, L.; THOMAS, A. (eds). English Teaching and New Literacies Pedagogy. Interpreting and Authoring Digital Multimodal Narratives. New York: Peter Lang Gmbh, 2014. p. 123-150.

PAINTER, C.; MARTIN, J. R.; UNSWORTH, L. Reading Visual Narratives: Image Analysis of Children’s Picture Books. Sheffield: Equinox, 2012.

PEREIRA, Í. S. P. Para uma pedagogia da leitura realizada on-line para construir conhecimento. Texto Digital, v. 15, n. 2, p. 28-56, 2019. DOI: https://doi.org/10.5007/1807-9288.2019v15n2p28

PEREIRA, Í.; GIL, M.; SYLLA, C. Para a construção da Teoria dos Multiletramentos. Dimensões ideacionais (e implicações pedagógicas) da multimodalidade textual numa StoryApp para crianças. Calidoscopio, v. 20, n. 1, p. 65-89, 2022. https://revistas.unisinos.br/index.php/calidoscopio/article/view/24438

PEREIRA, Í. S. P.; GIL, M.; SYLLA, C. Meeting the challenges of designing multimodal texts to enact the multiliteracies pedagogy = Enfrentando os Desafios do Design de Textos Multimodais Para Promover a Pedagogia das Multiliteracias. Comunicação e Sociedade, v. 43, p. 1-31, 2023. DOI: https://doi.org/10.17231/comsoc.43(2023).4481

RESNICK, M.; MARTIN, F.; BERG, R.; BOROVOY, R.; COLELLA, V.; KRAMER, K.; SILVERMAN, B. Digital Manipulatives: New Toys to Think With. In: CHI ‘98. Human Factors in Computer Systems, 1998, Los Angeles. Conference Proceedings. Steven Pemberton, 1998. p. 281-287.

ROWSELL, J.; KRESS, G.: PAHL, K.; STREET, B. The Social Practice of Multimodal Reading: A New Literacy Studies–Multimodal Perspective on Reading. In: ALVERMANN, D. E.; UNRAU, N. J.; RUDDEL, R. B. (eds.). Theoretical Models and Processes of Reading. Newark: International Reading Association, 2013. p. 1182–1207. 978 v.

STAKE, R. Case Studies. In: DENZIN, N.; LINCOLN, Y. (eds.). Handbook on qualitative Studies. 2 ed. London: Sage Publications Inc., 2000. p. 435-454.

STEIN, P. Multimodal Pedagogies in Diverse Classrooms. Representations, rights and resources. London: Routledge, 2008.

SYLLA, C.; GIL, M.; PEREIRA, Í. Untangling the complexity of designing tools to support tangible and digital intercultural storytelling in troubled times: a case in point. Literacy, London, v. 56, n. 1, p. 3-17, 2022. DOI: https://doi.org/10.1111/lit.12263

TAKACS, Z.; BUS, A. Benefits of Motion in Animated Storybooks for Children’s Visual Attention and Story Comprehension. An Eye-Tracking Study. Frontiers in Psychology, London, v. 7, p. 1-12, 2016. DOI: https://doi.org/10.3389/fpsyg.2016.01591

TAKACS, Z.; SWART, E.; BUS, A. Benefits and Pitfalls of Multimedia and Interactive Features in Technology-Enhanced Storybooks: A Meta-Analysis. Review of Educational Research, London, v. 85, n. 4, p. 698-739, 2015. DOI: https://doi.org/10.3102/0034654314566989

UNSWORTH, L. Towards a Metalanguage for Multiliteracies Education: Describing the Meaning-Making Resources of Language-Image Interaction. English Teaching: Practice and Critique, Leeds, v. 5, n. 1, p. 55-76, 2006.

WALSH, M. The ‘textual shift’: Examining the reading process with print, visual and multimodal texts. Australian Journal of Language & Literacy, Crows Nest, v. 29, n. 1, p. 24-37, 2006.

WALSH, M. Worlds have collided and modes have merged: Classroom evidence of changed literacy practices. Literacy, London, v. 42, n. 2, p. 101-108, 2008. DOI: https://doi.org/10.1111/j.1741-4369.2008.00495.x

YOKOTA, J.; TEALE, W. Picture Books and the Digital World. Reading Teacher, Newark, v. 67, n. 8, p. 577-585, 2014. DOI: https://doi.org/10.1002/trtr.1262

DOI: http://dx.doi.org/10.17851/2237-2083.31.4.1916-1951