The Modality Principle in Multimedia Learning

doi:10.1017/9781108894333.026

20 - The Modality Principle in Multimedia Learning

from Part V - Principles for Managing Essential Processing in Multimedia Learning

Published online by Cambridge University Press: 19 November 2021

Juan Cristobal Castro-Alonso and

John Sweller

Edited by

Richard E. Mayer and

Logan Fiorella

Show author details

Richard E. Mayer: Affiliation:
University of California, Santa Barbara
Logan Fiorella: Affiliation:
University of Georgia

Book contents

Get access

Summary

When dealing with instructional information, working memory can be divided into auditory and visual processors. The capacity limits of each processor are a major impediment when students are required to learn new material. Nevertheless, there is one strategy that can effectively expand working memory capacity by using the partially independent status of the auditory and visual processors. Under specific and well-defined conditions, presenting some information in visual mode and other information in auditory mode can increase effective working memory capacity and so reduce the effects of cognitive overload. This effect is called the instructional modality effect or modality principle. It is an instructional principle that can substantially increase learning. This chapter discusses the theory and data that underpin the principle and the instructional implications that flow from the principle.

Keywords

modality principle auditiory processor visual processor split-attention effect cognitive load theory

Information

Type: Chapter
Information: The Cambridge Handbook of Multimedia Learning , pp. 261 - 267
DOI: https://doi.org/10.1017/9781108894333.026 [Opens in a new window]
Publisher: Cambridge University Press
Print publication year: 2021

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Book purchase

Temporarily unavailable

References

Baddeley, A. (1992). Working memory. Science, 255(5044), 556–559.CrossRef Google Scholar PubMed

Baddeley, A. (2012). Working memory: Theories, models, and controversies. Annual Review of Psychology, 63(1), 1–29.Google Scholar

Baddeley, A., & Hitch, G. (1974). Working memory. In Bower, G. H. (ed.), Psychology of Learning and Motivation (Vol. 8, pp. 47–89). New York: Academic Press.Google Scholar

Bauhoff, V., Huff, M., & Schwan, S. (2012). Distance matters: Spatial contiguity effects as trade-off between gaze switches and memory load. Applied Cognitive Psychology, 26(6), 863–871.Google Scholar

Brünken, R., Steinbacher, S., Plass, J. L., & Leutner, D. (2002). Assessment of cognitive load in multimedia learning using dual-task methodology. Experimental Psychology, 49(2), 109–119.Google Scholar

Castro-Alonso, J. C. (ed.) (2019). Visuospatial Processing for Education in Health and Natural Sciences. Cham, Switzerland: Springer.Google Scholar

Castro-Alonso, J. C., Ayres, P., & Sweller, J. (2019). Instructional visualizations, cognitive load theory, and visuospatial processing. In Castro-Alonso, J. C. (ed.), Visuospatial Processing for Education in Health and Natural Sciences (pp. 111–143). Cham, Switzerland: Springer.Google Scholar

Castro-Alonso, J. C., Ayres, P., Wong, M., & Paas, F. (2018). Learning symbols from permanent and transient visual presentations: Don’t overplay the hand. Computers & Education, 116, 1–13.CrossRef Google Scholar

Castro-Alonso, J. C., & Sweller, J. (2020). The modality effect of cognitive load theory. In Karwowski, W., Ahram, T., & Nazir, S. (eds.), Advances in Human Factors in Training, Education, and Learning Sciences: Proceedings of the AHFE 2019 International Conference on Human Factors in Training, Education, and Learning Sciences (pp. 75–84). Cham, Switzerland: Springer.Google Scholar

Chandler, P., & Sweller, J. (1992). The split-attention effect as a factor in the design of instruction. British Journal of Educational Psychology, 62(2), 233–246.Google Scholar

Clark, J., & Paivio, A. (1991). Dual coding theory and education. Educational Psychology Review, 3(3), 149–210.Google Scholar

Cowan, N. (2001). The magical number 4 in short-term memory: A reconsideration of mental storage capacity. Behavioral and Brain Sciences, 24(1), 87–185.Google Scholar

Craig, S. D., Gholson, B., & Driscoll, D. M. (2002). Animated pedagogical agents in multimedia educational environments: Effects of agent properties, picture features, and redundancy. Journal of Educational Psychology, 94(2), 428–434.Google Scholar

Crooks, S. M., Cheon, J., Inan, F., Ari, F., & Flores, R. (2012). Modality and cueing in multimedia learning: Examining cognitive and perceptual explanations for the modality effect. Computers in Human Behavior, 28(3), 1063–1071.Google Scholar

De Renzi, E., & Nichelli, P. (1975). Verbal and non-verbal short-term memory impairment following hemispheric damage. Cortex, 11(4), 341–354.Google Scholar

Dinçer, S., & Doğanay, A. (2017). The effects of multiple-pedagogical agents on learners’ academic success, motivation, and cognitive load. Computers & Education, 111, 74–100.Google Scholar

Geary, D. C. (2008). An evolutionarily informed education science. Educational Psychologist, 43(4), 179–195.Google Scholar

Ginns, P. (2005). Meta-analysis of the modality effect. Learning and Instruction, 15(4), 313–331.Google Scholar

Harskamp, E. G., Mayer, R. E., & Suhre, C. (2007). Does the modality principle for multimedia learning apply to science classrooms? Learning and Instruction, 17(5), 465–477.Google Scholar

Horvath, J. C. (2014). The neuroscience of PowerPointTM. Mind, Brain, and Education, 8(3), 137–143.Google Scholar

Inan, F. A., Crooks, S. M., Cheon, J., Ari, F., Flores, R., Kurucay, M., & Paniukov, D. (2015). The reverse modality effect: Examining student learning from interactive computer-based instruction. British Journal of Educational Technology, 46(1), 123–130.Google Scholar

Jeung, H. J., Chandler, P., & Sweller, J. (1997). The role of visual indicators in dual sensory mode instruction. Educational Psychology, 17(3), 329–343.Google Scholar

Kirsner, K., & Smith, M. C. (1974). Modality effects in word identification. Memory & Cognition, 2(4), 637–640.CrossRef Google Scholar PubMed

Kroll, N. E. A., Parks, T., Parkinson, S. R., Bieber, S. L., & Johnson, A. L. (1970). Short-term memory while shadowing: Recall of visually and of aurally presented letters. Journal of Experimental Psychology, 85(2), 220–224.CrossRef Google Scholar PubMed

Kühl, T., Scheiter, K., Gerjets, P., & Edelmann, J. (2011). The influence of text modality on learning with static and dynamic visualizations. Computers in Human Behavior, 27(1), 29–35.CrossRef Google Scholar

Leahy, W., Chandler, P., & Sweller, J. (2003). When auditory presentations should and should not be a component of multimedia instruction. Applied Cognitive Psychology, 17(4), 401–418.Google Scholar

Leahy, W., & Sweller, J. (2011). Cognitive load theory, modality of presentation and the transient information effect. Applied Cognitive Psychology, 25(6), 943–951.CrossRef Google Scholar

Lee, H., & Mayer, R. E. (2018). Fostering learning from instructional video in a second language. Applied Cognitive Psychology, 32(5), 648–654.Google Scholar

Margrain, S. A. (1967). Short-term memory as a function of input modality. Quarterly Journal of Experimental Psychology, 19(2), 109–114.Google Scholar

Mayer, R. E., Fiorella, L., & Stull, A. (2020). Five ways to increase the effectiveness of instructional video. Educational Technology Research and Development, 68(3), 837–852.CrossRef Google Scholar

Mayer, R. E., & Moreno, R. (1998). A split-attention effect in multimedia learning: Evidence for dual processing systems in working memory. Journal of Educational Psychology, 90(2), 312–320.Google Scholar

Miller, G. A. (1956). The magical number seven, plus or minus two: Some limits on our capacity for processing information. Psychological Review, 63(2), 81–97.Google Scholar

Moreno, R. (2006). Does the modality principle hold for different media? A test of the method-affects-learning hypothesis. Journal of Computer Assisted Learning, 22(3), 149–158.Google Scholar

Moreno, R., & Mayer, R. E. (1999). Cognitive principles of multimedia learning: The role of modality and contiguity. Journal of Educational Psychology, 91(2), 358–368.Google Scholar

Moreno, R., Mayer, R. E., Spires, H. A., & Lester, J. C. (2001). The case for social agency in computer-based teaching: Do students learn more deeply when they interact with animated pedagogical agents? Cognition and Instruction, 19(2), 177–213.Google Scholar

Mousavi, S. Y., Low, R., & Sweller, J. (1995). Reducing cognitive load by mixing auditory and visual presentation modes. Journal of Educational Psychology, 87(2), 319–334.Google Scholar

Murdock, B. B. Jr. (1971). Four-channel effects in short-term memory. Psychonomic Science, 24(4), 197–198.Google Scholar

Mutlu-Bayraktar, D., Cosgun, V., & Altan, T. (2019). Cognitive load in multimedia learning environments: A systematic review. Computers & Education, 141, 103618.CrossRef Google Scholar

Oberauer, K., Lewandowsky, S., Awh, E., Brown, G. D. A., Conway, A., Cowan, N., … Ward, G. (2018). Benchmarks for models of short-term and working memory. Psychological Bulletin, 144(9), 885–958.Google Scholar

Ong, W. J. (1982). Orality and Literacy: The Technologizing of the Word. New York: Methuen.Google Scholar

Paivio, A. (1969). Mental imagery in associative learning and memory. Psychological Review, 76(3), 241–263.Google Scholar

Paivio, A. (2014). Intelligence, dual coding theory, and the brain. Intelligence, 47, 141–158.CrossRef Google Scholar

Park, S. (2015). The effects of social cue principles on cognitive load, situational interest, motivation, and achievement in pedagogical agent multimedia learning. Educational Technology & Society, 18(4), 211–229.Google Scholar

Penney, C. G. (1980). Order of report in bisensory verbal short-term memory. Canadian Journal of Psychology, 34(2), 190–195.CrossRef Google Scholar PubMed

Penney, C. G. (1989). Modality effects and the structure of short-term verbal memory. Memory & Cognition, 17(4), 398–422.Google Scholar

Peterson, L. R., & Peterson, M. J. (1959). Short-term retention of individual verbal items. Journal of Experimental Psychology, 58(3), 193–198.Google Scholar

Reinwein, J. (2012). Does the modality effect exist? and if so, which modality effect? Journal of Psycholinguistic Research, 41(1), 1–32.Google Scholar

Rollins, H. A., & Thibadeau, R. (1973). The effects of auditory shadowing on recognition of information received visually. Memory & Cognition, 1(2), 164–168.Google Scholar

Sanders, A. F., & Schroots, J. J. F. (1969). Cognitive categories and memory span. III. Effects of similarity on recall. Quarterly Journal of Experimental Psychology, 21(1), 21–28.Google Scholar

Scheiter, K., Schüler, A., Gerjets, P., Huk, T., & Hesse, F. W. (2014). Extending multimedia research: How do prerequisite knowledge and reading comprehension affect learning from text and pictures. Computers in Human Behavior, 31, 73–84.Google Scholar

Sweller, J., & Chandler, P. (1994). Why some material is difficult to learn. Cognition and Instruction, 12(3), 185–233.Google Scholar

Tabbers, H. K., Martens, R. L., & van Merriënboer, J. J. G. (2004). Multimedia instructions and cognitive load theory: Effects of modality and cueing. British Journal of Educational Psychology, 74(1), 71–81.Google Scholar

Tindall-Ford, S., Chandler, P., & Sweller, J. (1997). When two sensory modes are better than one. Journal of Experimental Psychology: Applied, 3(4), 257–287.Google Scholar

Wong, A., Leahy, W., Marcus, N., & Sweller, J. (2012). Cognitive load theory, the transient information effect and e-learning. Learning and Instruction, 22(6), 449–457.Google Scholar

Accessibility standard: WCAG 2.2 AAA

Why this information is here

This section outlines the accessibility features of this content - including support for screen readers, full keyboard navigation and high-contrast display options. This may not be relevant for you.

Accessibility Information

The HTML of this chapter complies with version 2.2 of the Web Content Accessibility Guidelines (WCAG), offering more comprehensive accessibility measures for a broad range of users and attains the highest (AAA) level of WCAG compliance, optimising the user experience by meeting the most extensive accessibility guidelines.

Content Navigation

Table of contents navigation
Allows you to navigate directly to chapters, sections, or non‐text items through a linked table of contents, reducing the need for extensive scrolling.

Index navigation
Provides an interactive index, letting you go straight to where a term or subject appears in the text without manual searching.

Reading Order & Textual Equivalents

Single logical reading order
You will encounter all content (including footnotes, captions, etc.) in a clear, sequential flow, making it easier to follow with assistive tools like screen readers.

Full alternative textual descriptions
You get more than just short alt text: you have comprehensive text equivalents, transcripts, captions, or audio descriptions for substantial non‐text content, which is especially helpful for complex visuals or multimedia.

Visualised data also available as non-graphical data
You can access graphs or charts in a text or tabular format, so you are not excluded if you cannot process visual displays.

Visual Accessibility

Use of high contrast between text and background colour
You benefit from high‐contrast text, which improves legibility if you have low vision or if you are reading in less‐than‐ideal lighting conditions.