The Multiple Representations Principle in Multimedia Learning

doi:10.1017/9781108894333.016

12 - The Multiple Representations Principle in Multimedia Learning

from Part III - Basic Principles of Multimedia Learning

Published online by Cambridge University Press: 19 November 2021

Shaaron Ainsworth

Edited by

Richard E. Mayer and

Logan Fiorella

Show author details

Richard E. Mayer: Affiliation:
University of California, Santa Barbara
Logan Fiorella: Affiliation:
University of Georgia

Book contents

Get access

Summary

In this chapter, I propose three distinct purposes of multiple representations and suggest that these lead to different design principles and learning activities. Multiple representations can play a complementary role when learners exploit differences in their form and content by switching between and selecting the appropriate representation for the task at hand. Constraining benefits are achieved when learners can profit from the support of a familiar representation to understand a new and complex representation. Finally, if learners abstract across multiple representations, they can construct a deeper understanding of the nature of the representations and the domain they represent. This chapter updates a review of studies that have used multiple representations for these purposes and identifies some of the circumstances that influence the effectiveness of using multiple representations in these ways.

Keywords

multiple representations DeFT framework complementary functions constraining functions constructing functions

Information

Type: Chapter
Information: The Cambridge Handbook of Multimedia Learning , pp. 158 - 170
DOI: https://doi.org/10.1017/9781108894333.016 [Opens in a new window]
Publisher: Cambridge University Press
Print publication year: 2021

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Book purchase

Temporarily unavailable

References

Ainsworth, S. E. (1999). The functions of multiple representations. Computers & Education, 33(2–3), 131–152.CrossRef Google Scholar

Ainsworth, S. E. (2006). Deft: A conceptual framework for considering learning with multiple representations. Learning and Instruction, 16(3), 183–198.Google Scholar

Ainsworth, S. E. (2014). The multiple representation principle in multimedia learning. In Mayer, R. (ed.), The Cambridge Handbook of Multimedia Learning (pp. 464–486). New York: Cambridge University Press.Google Scholar

Ainsworth, S. E., Bibby, P. A., & Wood, D. J. (1997). Evaluating principles for multi-representational learning environments. Paper Presented at the 7th European Conference for Research on Learning and Instruction, 1997, Athens.Google Scholar

Ainsworth, S. E., Bibby, P. A., & Wood, D. J. (2002). Examining the effects of different multiple representational systems in learning primary mathematics. Journal of the Learning Sciences, 11(1), 25–61.Google Scholar

Ainsworth, S. E., & Loizou, A. T. (2003). The effects of self-explaining when learning with text or diagrams. Cognitive Science, 27(4), 669–681.Google Scholar

Anzai, Y. (1991). Learning and use of representations for physics expertise. In Anders-Ericsson, K., & Smith, J. (eds.), Towards a General Theory of Expertise: Prospects and Limits (pp. 64–92). Cambridge: Cambridge University Press.Google Scholar

Berthold, K., Eysink, T. H. S., & Renkl, A. (2008). Assisting self-explanation prompts are more effective than open prompts when learning with multiple representations. Instructional Science, 37(4), 345–363.Google Scholar

Bertin, J. (1983). Semiology of Graphics: Diagrams, Networks, Maps (trans. Berg, W. J.). Madison, WI: University of Madison Press.Google Scholar

Bivall, P., Ainsworth, S., & Tibell, L. A. E. (2011). Do haptic representations help complex molecular learning? Science Education, 95(4), 700–719.Google Scholar

Bodemer, D., Ploetzner, R., Bruchmuller, K., & Hacker, S. (2005). Supporting learning with interactive multimedia through active integration of representations. Instructional Science, 33(1), 73–95.Google Scholar

Boshuizen, H., & van der Wiel, W. (1998). Using multiple representations in medicine: How students struggle with them. In Van Someren, M. W., Reimann, P., Boshuizen, H. P. A., & de Jong, T. (eds.), Learning with Multiple Representations (pp. 237–262). Amsterdam: Pergamon.Google Scholar

Card, S. K., Mackinlay, J. D., & Shneiderman, B. (1999). Readings in Information Visualization: Using Vision to Think. San Diego, CA: Academic Press.Google Scholar

Cheng, P. C.-H. (1996). Functional roles for the cognitive analysis of diagrams in problem solving. Paper presented at the Proceeding of the Eighteenth Annual Conference of the Cognitive Science Society, 1996, Hillsdale, NJ.Google Scholar

Chua, H. F., Yates, J. F., & Shah, P. (2006). Risk avoidance: Graphs versus numbers. Memory & Cognition, 34(2), 399–410.CrossRef Google Scholar PubMed

Cooper, M. M., Corley, L. M., & Underwood, S. M. (2013). An investigation of college chemistry students’ understanding of structure–property relationships. Journal of Research in Science Teaching, 50(6), 699–721.Google Scholar

Corradi, D., Elen, J., & Clarebout, G. (2012). Understanding and enhancing the use of multiple external representations in chemistry education. Journal of Science Education and Technology, 21(6), 1–16.Google Scholar

Danielson, R. W., Schwartz, N. H., & Lippmann, M. (2015). Metaphorical graphics aid learning and memory. Learning and Instruction, 39, 194–205.Google Scholar

Danielson, R. W., & Sinatra, G. M. (2016). A relational reasoning approach to text-graphic processing. Educational Psychology Review, 29(1), 55–72.Google Scholar

de Koning, B. B., Rop, G., & Paas, F. (2020). Learning from split-attention materials: Effects of teaching physical and mental learning strategies. Contemporary Educational Psychology, 61, 101873.Google Scholar

Dienes, Z. (1973). The Six Stages in the Process of Learning Mathematics. Slough: NFER-Nelson.Google Scholar

Eilam, B. (2012). Teaching, Learning, and Visual Literacy: The Dual Role of Visual Representation. New York: Cambridge University Press.Google Scholar

Eitel, A., Scheiter, K., Schüler, A., Nyström, M., & Holmqvist, K. (2013). How a picture facilitates the process of learning from text: Evidence for scaffolding. Learning and Instruction, 28, 48–63.Google Scholar

Fiorella, L., & Zhang, Q. (2018). Drawing boundary conditions for learning by drawing. Educational Psychology Review, 30(3), 1115–1137.CrossRef Google Scholar

Furberg, A., Kluge, A., & Ludvigsen, S. (2013). Student sensemaking with science diagrams in a computer-based setting. International Journal of Computer-Supported Collaborative Learning, 8(1), 41–64.Google Scholar

Goldman, S. R. (2003). Learning in complex domains: When and why do multiple representations help? Learning and Instruction, 13(2), 239–244.Google Scholar

Green, T. R. G., Petre, M., & Bellamy, R. K. E. (1991). Comprehensibility of visual and textual programs: A test of superlativism against the “match–mismatch” conjecture. In Koenemann-Belliveau, J., Moher, T. G., & Robertson, S. P. (eds.), Proceedings of Empirical Studies of Programmers: Fourth Workshop (pp. 121–146). Norwood, NJ: Ablex Publishing Company.Google Scholar

Hannus, M., & Hyona, J. (1999). Utilization of illustrations during learning of science textbook passages among low- and high-ability children. Contemporary Educational Psychology, 24(2), 95–123.Google Scholar

Harp, S. F., & Mayer, R. E. (1997). The role of interest in learning from scientific text and illustrations: On the distinction between emotional interest and cognitive interest. Journal of Educational Psychology, 89(1), 92–102.Google Scholar

Hegarty, M. (2011). The cognitive science of visual-spatial displays: Implications for design. Topics in Cognitive Science, 3(3), 446–474.Google Scholar

Hegarty, M., & Just, M. A. (1993). Constructing mental models of machines from text and diagrams. Journal of Memory and Language, 32(6), 717–742.Google Scholar

Hegedus, S. J., & Roschelle, J. (2013). The SimCalc Vision and Contributions. Netherlands: Springer.Google Scholar

Hilton, A., & Nichols, K. (2011). Representational classroom practices that contribute to students conceptual and representational understanding of chemical bonding. International Journal of Science Education, 33(16), 2215–2246.CrossRef Google Scholar

Kalyuga, S., Ayres, P., Chandler, P., & Sweller, J. (2003). The expertise reversal effect. Educational Psychologist, 38(1), 23–31.Google Scholar

Kaput, J. J. (1992). Technology and mathematics education. In Grouws, D. A. (ed.), Handbook of Teaching and Learning Mathematics (pp. 515–556). New York: Macmillan.Google Scholar

Kozma, R., Chin, E., Russell, J., & Marx, N. (2000). The roles of representations and tools in the chemistry laboratory and their implications for chemistry learning. Journal of the Learning Sciences, 9(2), 105–143.Google Scholar

Kozma, R., & Russell, J. (2005). Students becoming chemists: Developing representational competence. In Gilbert, J. K. (ed.), Visualization in Science and Education (pp. 121–146). Dordrecht: Kluwer Academic Publishers.Google Scholar

Kress, G., & Van Leeuwen, T. (1996). The Grammar of Visual Design. London: Routledge.Google Scholar

Larkin, J. H., & Simon, H. A. (1987). Why a diagram Is (sometimes) worth 10000 words. Cognitive Science, 11(1), 65–99.Google Scholar

Lee, H., Plass, J. L., & Homer, B. D. (2006). Optimizing cognitive load for learning from computer-based science simulations. Journal of Educational Psychology, 98(4), 902–913.Google Scholar

Leinhardt, G., Zaslavsky, O., & Stein, M. M. (1990). Functions, graphs, and graphing: Tasks, learning and teaching. Review of Educational Research, 60(1), 1–64.Google Scholar

Lemke, J. L. (2004). The literacies of science. In Saul, E. W. (ed.), Crossing Borders in Literacy and Science Instruction: Perspectives on Theory and Practice (pp. 33–47). Newark, DE: International Reading Association.Google Scholar

Levin, J. R., Anglin, G. J., & Carney, R. N. (1987). On empirically validating functions of pictures in prose. In Willows, D. M., & Houghton, H. A. (eds.), The Psychology of Illustration: I. Basic Research (pp. 51–85). New York: Springer.Google Scholar

Lohse, G. L., Biolsi, K., Walker, N., & Rueler, H. (1994). A classification of visual representations. Communications of the A.C.M., 37(12), 36–49.Google Scholar

Lowe, R., Schnotz, W., & Rasch, T. (2010). Aligning affordances of graphics with learning task requirements. Applied Cognitive Psychology, 25(3), 452–459.Google Scholar

Madden, S. P., Jones, L. L., & Rahm, J. (2011). The role of multiple representations in the understanding of ideal gas problems. Chemistry Education Research and Practice, 12(3), 283–293.Google Scholar

Mason, L., Tornatora, M. C., & Pluchino, P. (2013). Do fourth graders integrate text and picture in processing and learning from an illustrated science text? Evidence from eye-movement patterns. [Article]. Computers & Education, 60(1), 95–109.Google Scholar

Mayer, R. E. (2014). Cognitive theory of multimedia learning. In Mayer, R. E. (ed.), The Cambridge Handbook of Multimedia Learning: Second edition (pp. 43–71). New York: Cambridge University Press.Google Scholar

McCrudden, M. T., & Rapp, D. N. (2017). How visual displays affect cognitive processing. Educational Psychology Review, 29(3), 623–639.Google Scholar

McDermott, L. C., Rosenquist, M. L., & van Zee, E. H. (1987). Student difficulties in connecting graphs and physics: Examples from kinematics. American Journal of Physics, 55(6), 503–513.Google Scholar

McElhaney, K. W., Chang, H.-Y., Chiu, J. L., & Linn, M. C. (2014). Evidence for effective uses of dynamic visualisations in science curriculum materials. Studies in Science Education, 51(1), 49–85.CrossRef Google Scholar

Meyer, J., Shinar, D., & Leiser, D. (1997). Multiple factors that determine performance with tables and graphs. Human Factors, 39(2), 268–286.CrossRef Google Scholar

Minogue, J., & Borland, D. (2015). Investigating students’ ideas about buoyancy and the influence of haptic feedback. Journal of Science Education and Technology, 25(2), 187–202.Google Scholar

Nemirovsky, R., Tierney, C., & Wright, T. (1998). Body motion and graphing. Cognition and Instruction, 16(2), 119–172.Google Scholar

Olympiou, G., Zacharias, Z., & de Jong, T. (2013). Making the invisible visible: Enhancing students’ conceptual understanding by introducing representations of abstract objects in a simulation. Instructional Science, 41(3), 575–596.CrossRef Google Scholar

Paik, E. S., & Schraw, G. (2013). Learning with animation and illusions of understanding. Journal of Educational Psychology, 105(2), 278–289.Google Scholar

Parnafes, O., & Disessa, A. (2004). Relations between types of reasoning and computational representations. International Journal of Computers for Mathematical Learning, 9(3), 251–280.Google Scholar

Peirce, C. S. (1906). Prolegomena to an apology for pragmaticism. The Monist, 16, 492–546.Google Scholar

Plass, J. L., Homer, B. D., & Kinzer, C. K. (2015). Foundations of game-based learning. Educational Psychologist, 50(4), 258–283.Google Scholar

Prain, V., & Tytler, R. (2012). Learning through constructing representations in science: A framework of representational construction affordances. International Journal of Science Education, 34(17), 2751–2773.Google Scholar

Prain, V., & Waldrip, B. (2006). An exploratory study of teachers’ and students’ use of multi‐modal representations of concepts in primary science. International Journal of Science Education, 28(15), 1843–1866.CrossRef Google Scholar

Rau, M. A. (2018). Sequencing support for sense making and perceptual induction of connections among multiple visual representations. Journal of Educational Psychology, 110(6), 811–833.Google Scholar

Rau, M. A. (2020). Cognitive and socio-cultural theories on competencies and practices involved in learning with multiple external representations. In Van Meter, P., List, A., Lombardi, D, & Kendeou, P. (eds.), Handbook of Learning from Multiple Representations and Perspectives (pp. 17–32). New York: Routledge.Google Scholar

Rau, M. A., Aleven, V., & Rummel, N. (2015). Successful learning with multiple graphical representations and self-explanation prompts. Journal of Educational Psychology, 107(1), 30–46.Google Scholar

Rau, M. A., Aleven, V., Rummel, N., & Pardos, Z. (2013). How should intelligent tutoring systems sequence multiple graphical representations of fractions? A multi-methods study. International Journal of Artificial Intelligence in Education, 24(2), 125–161.Google Scholar

Rau, M., Aleven, V., Rummel, N., & Rohrbach, S. (2012). Sense making alone doesn’t do it: Fluency matters too! ITS support for robust learning with multiple representations. In Cerri, S., Clancey, W., Papadourakis, G., & Panourgia, K. (eds.), Intelligent Tutoring Systems (Vol. 7135, pp. 174–184). Berlin: Springer.Google Scholar

Rau, M. A., Bowman, H. E., & Moore, J. W. (2017). An adaptive collaboration script for learning with multiple visual representations in chemistry. Computers & Education, 109, 38–55.CrossRef Google Scholar

Reed, S. K. (2010). Thinking Visually. New York: Psychology Press.Google Scholar

Roschelle, J., & Teasley, S. D. (1995). The construction of shared knowledge in collaborative problem solving. In O’Malley, C. (ed.), Computer Supported Collaborative Learning (pp. 69–97). Berlin: Springer.Google Scholar

Rouet, J. F., Britt, M. A., & Durik, A. M. (2017). RESOLV: Readers’ representation of reading contexts and tasks. Educational Psychologist, 52(3), 200–215.Google Scholar

Russell, J., Kozma, R., Becker, D., & Susskind, T. (2000). SMV: Chem; Synchronized Multiple Visualizations in Chemistry. New York: John Wiley.Google Scholar

Rutten, N., van Joolingen, W. R., & van der Veen, J. T. (2012). The learning effects of computer simulations in science education. Computers & Education, 58(1), 136–153.Google Scholar

Salomon, G. (1984). Television is easy and print is tough – The differential investment of mental effort in learning as a function of perceptions and attributions. Journal of Educational Psychology, 76(4), 647–658.CrossRef Google Scholar

Scaife, M., & Rogers, Y. (1996). External cognition: How do graphical representations work? International Journal of Human–Computer Studies, 45(2), 185–213.Google Scholar

Scheiter, K., Gerjets, P., Huk, T., Imhof, B., & Kammerer, Y. (2009). The effects of realism in learning with dynamic visualizations. Learning and Instruction, 19(6), 481–494.Google Scholar

Schneider, S., Dyrna, J., Meier, L., Beege, M., & Rey, G. D. (2018). How affective charge and text–picture connectedness moderate the impact of decorative pictures on multimedia learning. Journal of Educational Psychology, 110(2), 233–249.CrossRef Google Scholar

Schnotz, W. (2002). Commentary – Towards an integrated view of learning from text and visual displays. Educational Psychology Review, 14(1), 101–120.Google Scholar

Schoenfeld, A. H., Smith, J. P., & Arcavi, A. (1993). Learning: The microgenetic analysis of one student’s evolving understanding of a complex subject matter domain. In Glaser, R. (ed.), Advances in Instructional Psychology (pp. 55–176). Hillsdale, NJ: Lawrence Erlbaum.Google Scholar

Schwonke, R., Berthold, K., & Renkl, A. (2009). How multiple external representations are used and how they can be made more useful. Applied Cognitive Psychology, 23(9), 1227–1243.Google Scholar

Seufert, T. (2019). Training for coherence formation when learning from text and picture and the interplay with learners’ prior knowledge. Frontiers in Psychology, 10, 193.Google Scholar

Shah, P., & Freedman, E. G. (2011). Bar and line graph comprehension: An interaction of top‐down and bottom‐up processes. Topics in Cognitive Science, 3(3), 560–578.Google Scholar

Stieff, M., Hegarty, M., & Deslongchamps, G. (2011). Identifying representational competence with multi-representational displays. Cognition and Instruction, 29(1), 123–145.Google Scholar

Stull, A. T., Hegarty, M., Dixon, B., & Stieff, M. (2012). Representational translation with concrete models in organic chemistry. Cognition and Instruction, 30(4), 404–434.Google Scholar

Suthers, D. D. (2014). Empirical studies of the value of conceptually explicit notations in collaborative learning. In Okada, A., Buckingham Shum, S. J., & Sherborne, T. (eds.), Knowledge Cartography: Software Tools and Mapping Techniques (pp. 1–22). London: Springer.Google Scholar

Treagust, D. F., & Tsui, C.-Y. (2013). Contributions of multiple representations to biology education. In Treagust, D. F., & Tsui, C.-Y. (eds.), Multiple Representations in Biology Education (pp. 349–367). Dordrecht: Springer.Google Scholar

Tversky, B. (2011). Visualizing thought. Topics in Cognitive Science, 3(3), 499–535.Google Scholar

Tytler, R., Prain, V., Hubber, P., & Waldrip, B. (2013). Constructing Representations to Learn in Science. Rotterdam: Sense Publishers.Google Scholar

van der Meij, J., & de Jong, T. (2006). Supporting students’ learning with multiple representations in a dynamic simulation-based learning environment. Learning and Instruction, 16(3), 199–212.Google Scholar

van der Meij, J., & de Jong, T. (2011). The effects of directive self-explanation prompts to support active processing of multiple representations in a simulation-based learning environment. Journal of Computer Assisted Learning, 27(5), 411–423.Google Scholar

van Joolingen, W. R., & de Jong, T. (2003). SIMQUEST: Authoring educational simulations. In Murray, T., Blessing, S., & Ainsworth, S. E. (eds.), Tools for Advanced Technology Learning Environments (pp. 1–32). Amsterdam: Kluwer Academic Publishers.Google Scholar

van Labeke, N., & Ainsworth, S. (2003). A microgenetic approach to understanding translation between representations. Paper presented at the 10th EARLI Conference, 2003, Padova, Italy.Google Scholar

Virk, S., Clark, D., & Sengupta, P. (2015). Digital games as multi-representational environments for science learning: Implications for theory, research, and design. Educational Psychologist, 50(4), 284–312.Google Scholar

Ware, C. (2008). Visual Thinking for Design. Burlington, MA: Morgan Kaufmann Publishers.Google Scholar

White, T., & Pea, R. (2011). Distributed by design: On the promises and pitfalls of collaborative learning with multiple representations. Journal of the Learning Sciences, 20(3), 489–547.Google Scholar

Wilder, A., & Brinkerhoff, J. (2007). Supporting representational competence in high school biology with computer-based biomolecular visualizations. Journal of Computers in Mathematics and Science Teaching, 26(1), 5–26.Google Scholar

Won, M., Yoon, H., & Treagust, D. F. (2014). Students’ learning strategies with multiple representations: Explanations of the human breathing mechanism. Science Education, 98(5), 840–866.Google Scholar

Wu, S. P. W., & Rau, M. A. (2018). Effectiveness and efficiency of adding drawing prompts to an interactive educational technology when learning with visual representations. Learning and Instruction, 55, 93–104.Google Scholar

Yore, L., & Hand, B. (2010). Epilogue: Plotting a research agenda for multiple representations, multiple modality, and multimodal representational competency. Research in Science Education, 40(1), 93–101.Google Scholar

Zhang, J. J. (1996). A representational analysis of relational information displays. International Journal of Human–Computer Studies, 45(1), 59–74.Google Scholar

Zhang, Z. H., & Linn, M. C. (2011). Can generating representations enhance learning with dynamic visualizations? Journal of Research in Science Teaching, 48(10), 1177–1198.Google Scholar

Accessibility standard: WCAG 2.2 AAA

Why this information is here

This section outlines the accessibility features of this content - including support for screen readers, full keyboard navigation and high-contrast display options. This may not be relevant for you.

Accessibility Information

The HTML of this chapter complies with version 2.2 of the Web Content Accessibility Guidelines (WCAG), offering more comprehensive accessibility measures for a broad range of users and attains the highest (AAA) level of WCAG compliance, optimising the user experience by meeting the most extensive accessibility guidelines.

Content Navigation

Table of contents navigation
Allows you to navigate directly to chapters, sections, or non‐text items through a linked table of contents, reducing the need for extensive scrolling.

Index navigation
Provides an interactive index, letting you go straight to where a term or subject appears in the text without manual searching.

Reading Order & Textual Equivalents

Single logical reading order
You will encounter all content (including footnotes, captions, etc.) in a clear, sequential flow, making it easier to follow with assistive tools like screen readers.

Full alternative textual descriptions
You get more than just short alt text: you have comprehensive text equivalents, transcripts, captions, or audio descriptions for substantial non‐text content, which is especially helpful for complex visuals or multimedia.

Visualised data also available as non-graphical data
You can access graphs or charts in a text or tabular format, so you are not excluded if you cannot process visual displays.

Visual Accessibility

Use of high contrast between text and background colour
You benefit from high‐contrast text, which improves legibility if you have low vision or if you are reading in less‐than‐ideal lighting conditions.