Multimedia Learning (2009) — Ch. 1-3 (Richard E. Mayer)

This post covers Section I: Introduction to Multimedia Learning:

  • Chapter 1: The Promise of Multimedia Learning
  • Chapter 2: The Science of Instruction: Determining What Works in Multimedia Learning
  • Chapter 3: The Science of Learning: Determining How Multimedia Learning Works

[*Note: Much of this material was copied directly from Mayer’s book.]

Chapter 1: The Promise of Multimedia Learning

Multimedia learning refers to learning from words and pictures.

Multimedia instruction is the presentation of material using both words and pictures, with the intention of promoting learning. Words = verbal form (printed or spoken text). Pictures = pictorial form (static or dynamic graphics). This type of presentation is also called dual-mode, dual-format, dual-code, or dual-channel learning.

The case for multimedia learning: The rationale for multimedia presentations is that it takes advantage of the full capacity of humans for processing information (p.6). Furthermore, words and pictures can complement one another and…human understanding occurs when learners are able to mentally integrate corresponding pictorial and verbal representations…[and] are able to build meaningful connections between pictorial and verbal representations. In the process of trying to build connections between words and pictures, learners are able to create a deeper understanding than they could from words or pictures alone (cognitive theory of multimedia learning) (p.7).

Three views of multimedia messages (this book is concerned with the two that are in bold):

  • Delivery-Media View
  • Presentation-Modes View — assumes learners have separate information-processing channels for verbal and pictorial knowledge.
  • Sensory-Modality View two or more sensory systems in the learner are involved; assumes learners have separate information-processing channels for auditory and visual processing.

The dual-channel assumption: that humans process information in more than one channel.

Two approaches to multimedia design:

  • Technology-Centered (p.10-13)
    • begins w/ fxnal capabilities of MM; focus on cutting-edge advances in MM technology.
    • research focus often on determining which technology is most effective in presenting info.
    • this approach generally fails to lead to lasting improvements in education.
      • the driving force behind implementation was the power of the technology rather than an interest in promoting human cognition. The focus was on giving people access to the latest technology rather than on helping people to learn through the aid of technology.
  • Learner-Centered (p.13-15)
    • begins with an understanding of how the human mind works
    • focus is on using multimedia technology as an aid to human cognition
    • premise: multimedia designs that are consistent with the way the human mind works are more effective in fostering learning than those that are not.
    • Norman calls this “human-centered technology” and says technology can expand our cognitive capabilities.
    • Norman refers to tools that aid the mind as cognitive artifacts: “anything invented by humans for the purpose of improving thought or action counts as an artifact.”
      • Examples include language and arithmetic, paper and pencils, computer technology.
    • Norman: “technology…should complement human abilities, aid those activities for which we are poorly suited, and enhance and help develop those for which we are ideally suited.”

Landauer describes two major phases in the use of computer technology:

  • In the automation phase, computers are used to replace humans on certain tasks: robots in manufacturing; imaging devices such as CAT scans and MRIs; computer-based switching. The number of easy-to-automate tasks is dwindling.
  • Augmentation involves the use of computers to enhance human performance on various cognitively complex tasks. The major challenge in making the augmentation phase work involves a learner centered design of computer based technology.

There are three major views of multimedia learning. Multimedia learning as:

  • Response strengthening (p.15-16)
    • Drill and practice system
    • The response strengthening view is based on Thorndike’s 1911 classic research on how cats learn to pull a loop of string to get out of a puzzle box. Behaviors that are followed by satisfaction are more likely to occur in the future under the same circumstances; behavior that are followed by dissatisfaction are less likely to occur in the future under the same circumstances.
    • Critics have argued that the law of effect and the response strengthening view on which is based are not necessarily wrong, but rather are somewhat limited. They may apply to how laboratory animals learn to give a response, or even to carry out a procedure, but how can they account for more complex conceptual learning?
  • Information acquisition (p.16-17)
    • Information delivery system
    • Underlying metaphor is that of multimedia as a delivery system; multimedia is a vehicle for efficiently delivering information to the learner.  Related terms: empty vessel view, the  transmission view, the commodity view.
    • This view conflicts with the research base on how people learn complex material (Bransford et al., 1999). Humans focus on the meaning of presented material and interpret it in light of their prior knowledge.
  • Knowledge construction (p.17-19)
    • Cognitive aid
    • Multimedia learning is a sense-making activity in which the learner seeks to build a coherent mental representation from the presented material.
    • Unlike information, which is an objective commodity that can be moved from one mind to another, knowledge is personally constructed by the learner and cannot be delivered in exactly the same form from one mind to another.
    • The learner’s job is to make sense of the presented material, thus, the learner is an active sense-maker who experiences a multimedia presentation and tries to organize and integrate the  presented material into a coherent mental representation.
    • The teacher’s job is to assist the learner in this sense making process; thus, a teacher is a cognitive guide who provides needed guidance to support the learner’s cognitive processing.
    • The goal of multimedia presentations is not only to present information, but also to provide  guidance for how to process the presented information that is, for determining what to pay attention to, how to mentally organize it, and how to relate it to prior knowledge.
    • The underlying metaphor is that of multimedia as a helpful communicator; according to this metaphor, multimedia is a sense making guide, that is, an aid to knowledge construction.

Three kinds of multimedia learning outcomes: no learning; rote learning; meaningful learning.

Two major goals of learning:

  • Remembering is the ability to reproduce or recognize the presented material, and is assessed by retention tests. The most common retention tests are:
    • recall, in which players are asked to reproduce what was presented, and
    • recognition, in which learners are asked to select what was presented or judge whether a given item was presented.
    • The major issue in retention test involves quantity of learning, that is how much was remembered.
  • Understanding is the ability to construct a coherent mental representation from the presented material; it is reflected in the ability to use the presented material in novel situations, and is assessed by transfer tests.
    • The major issue in transfer tests involves the quality of learning,  that is, how well someone can use what they have learned.

The distinguishing pattern for rote learning outcomes is good retention and poor transfer. In this case, the learner has acquired fragmented knowledge or inert knowlege (factoids-isolated bits of information).

(p.20-21) Meaningful learning results from active learning. Research on learning shows that meaningful learning depends on the learner’s cognitive activity during learning rather than on the behavioral activity during learning. Well designed multimedia instructional messages can promote active cognitive processing in learners even when they seem to be behaviorally inactive.

Two goals of multimedia research:

  • The goal of basic research is to contribute to theory, such as the science of learning.
  • The goal of applied research is to contribute to practice, for example, the science of instruction.
  • Research that contributes to learning theory and to instructional practice  is the hallmark of what Stokes calls “use-inspired basic research” or what Mayer calls “basic research in applied situations.” (p.25)

Use-inspired basic research challenges learning theory to explain how learning works on authentic tasks and enriches instructional practice by helping us understand the conditions under which the principles can be expected to apply.

Chapter 2: The Science of Instruction: Determining What Works in Multimedia Learning

Science of instruction

  • Involves the creation of evidence-based principles for helping people learn
  • Goal is to contribute to evidence-based practice, i.e., instructional practices that are consistent with research evidence

A retention score is computed from learner’s recall protocol. (p.38)

Types of transfer questions include: (p.39)

  • redesign question
  • troubleshooting question
  • prediction question
  • conceptual question

An instructional method is a way of presenting a lesson, such as using spoken versus printed text along with an animation.

  • An instructional method does not change the content of the lesson — the covered material is the same. In short, what is presented stays the same under both instructional methods.
  • Similarly, an instructional method does not change the medium of the lesson (i.e., presented on a computer screen or on paper). In short,  the device used to present the material stays the same under both instructional methods.
  • [Surprising] Clark (2001): Instructional methods cause learning, but instructional media do not cause learning. The same instructional methods have the same effects on learning regardless of whether the medium is a desktop computers, nonimmersive virtual reality, or immersive virtual reality (Mayer & Moreno, 2002). (p. 53)

Chapter 3: The Science of Learning: Determining How Multimedia Learning Works

Learning is a change in knowledge attributable to experience (p.59). This definition has three parts:

  • learning is a change in the learner
  • what is changed is the learner’s knowledge
  • the cause of the change is the learner’s experience in a learning environment.

The change may involve reorganizing and integrating knowledge rather than simply adding new knowledge.

What is learned may involve five kinds of knowledge: facts, concepts, procedures, strategies, beliefs.

A cognitive theory of multimedia learning assumes that:

  • The human information processing system includes dual channels for visual/pictorial & auditory/verbal processing (i.e., visually represented material and auditorially represented material) (p.64)
    • There are two ways of conceptualizing the differences between the two channels one based on presentation modes, the other based on sensory modalities.
    • The presentation-mode approach focuses on whether the presented stimulus is verbal (spoken or printed words) or non-verbal ( illustrations, video, animation, or background sounds.) According to the presentation mode approach, one channel processes verbal material, and the other channel processes pictorial material and nonverbal sounds. This is most consistent with Paivio’s 1986, 2006 distinction between verbal and nonverbal systems.
    • The sensory modality approach focuses on whether learners initially  process the presented materials through their eyes (such as for illustrations, video, animation, or printed word) or their ears (such as for spoken words or background sounds.) One channel processes visually represented material and the other channel processes auditorially represented material.  This is most consistent with Baddeley’s 1992, 1999 distinction between the visuo-spatial sketchpad and the articulatory (or phonological) loop.
    • Whereas the presentation mode approach focuses on the format of the stimulus as presented (verbal or nonverbal), the sensory modalities approach focuses on the stimulus as represented in working memory (auditory or visual). The major difference concerning multimedia learning rests in the processing of printed words (i.e., on-screen text) and background sounds (p.65).
    • Relationship btw two channels: Although information enters the human information system via one channel, learners may also be able to convert the representation for processing in the other channel. Such cross channel representations of the same stimulus play an important role in Paivio’s 1986, 2006 dual-coding theory.
  • Each channel has limited capacity for processing. (p.66-67)
    • When an illustration or animation is presented, the learner is able to hold only a few images in working memory at any one time, reflecting portions of the presented material rather than an exact copy of the presented material. Baddeley’s 1992, 1999 theory of working memory and Sweller’s 1999, 2005 cognitive load theory.
    • The classic way to measure someone’s cognitive capacity is to give the person a memory span test to measure memory span for digits or digit span. The longest list you can recite of line drawings of simple objects is your memory span for pictures.  An average memory span is approximately 5 to 7 chunks.
    • Metacognitive strategies are techniques for allocating, monitoring, coordinating, and adjusting these limited cognitive resources. These strategies are at the heart of what Baddeley 1992 calls the central executive, the system that controls the allocation of cognitive resources, and play a central role in modern theories of intelligence.
  • Active learning entails carrying out appropriate cognitive processing during learning (active processing) in order to construct a coherent mental representation of their experiences. (p.67-70)
    • These active cognitive processes include paying attention, organizing incoming information, and integrating incoming information with other knowledge.
    • A mental model or knowledge structure represents the key parts of the presented material and their relations.
    • Some basic knowledge structures include process, comparison, generalization, enumeration, and classification.
      • Process structures can be represented as cause and effect chains and consist of explanations of how some system works.
      • Comparison structures can be represented as matrices and consist of comparisons among two or more elements along several dimensions.
      • Generalization structures can be represented as a branching tree and consist of a main idea with subordinate supporting details.
      • Enumeration structures can be represented as lists and consist of a collection of items.
      • Classification structures can be represented as hierarchies and consist of sets and subsets. Understanding a multimedia message often involves constructing one of these kinds of knowledge structures.
    • The presented material should have a coherent structure and the message should provide guidance to the learner on how to build the structure. Thus, multimedia design can be conceptualized as an attempt to assist learners in their model building efforts.

Three processes essential for active learning:

  1. Selecting relevant material — occurs when a learner pays attention to appropriate words and images in the presented material. This process involves bringing material from the outside into the working memory component of the cognitive system.
  2. Organizing selected material — involves building structural relations among the elements. This process takes place within the working memory component of the cognitive system.
  3. Integrating selected material with existing knowledge — involves building connections between incoming material and relevant portions of prior knowledge. This process involves activating knowledge in long-term memory  and bringing it into working memory.

Five cognitive processes necessary for meaningful learning:  (p.70-76)

  • Selecting relevant words from the presented text or narration
    • External presentation of spoken words (such as a computer-generated narration) -> sensory representation of sounds -> internal working memory representation of word sounds.
    • Input = spoken verbal message. Output = word sound base, a mental representation in the learner’s verbal working memory of selected words or phrases.
    • Involves paying attention to some of the words that are presented in the multimedia message as they pass through auditory sensory memory.
    • If the words are presented as speech, this process begins in the auditory channel. If the words are presented as on-screen text or printed text, this process begins in the visual channel.
  • Selecting relevant images from the presented illustrations
    • External presentation of pictures (an animation segment or an illustration) -> sensory representation of unanalyzed visual images -> internal representation in working memory (such as a visual image of part of the animation or illustration.)
    • Input = pictorial portion of a multimedia message that is held briefly in visual sensory memory. Output = visual image based, a mental representation in the learner’s working memory of selected images.
    • Involves paying attention to part of the animation, or illustrations presented in a MM msg.
    • The process begins in the visual channel, but it is possible to convert part of it to the auditory channel (such as by mentally narrating an ongoing animation.)
  • Organizing the selected words into a coherent visual representation
    • Words -> coherent representation, a knowledge structure called a verbal model.
    • Input = word sound base. Output = verbal model, a coherent, or structured, representation of the selected words or phrases in the learner’s working memory.
    • Learner builds connections among pieces of verbal knowledge.
    • This process most likely occurs in the auditory channel and is subject to capacity limitations.
  • Organizing selected images into a coherent visual representation
    • Image base from the incoming pictures of a segment of the multimedia message -> coherent representation, a knowledge structure called a pictorial model.
    • Input = visual image base, the pictures selected from the incoming pictorial message. Output = pictorial  model, a coherent, or structured, representation of the selected images in the learner’s working memory.
    • Learner builds connections among pieces of pictorial knowledge.
    • This process occurs in the visual channel.
  • Integrating the visual (image-based) and verbal (word-based) representations and prior knowledge
    • Two separate representations (a pictorial model and a verbal model) -> integrated representation in which corresponding elements and relations from one model are mapped onto the other.
    • Input = pictorial model and the verbal model the learner has constructed. Output = integrated model that is based on connecting the two representations. In addition, the pictorial and verbal models are connected with prior knowledge activated from long-term memory.
    • Involves building connections between corresponding portions of a pictorial and verbal models as well as knowledge from long-term memory.
    • Occurs in visual and verbal working memory, and involves the coordination between them.

Each of the five steps is likely to occur many times throughout a multimedia presentation. The steps are applied segment by segment, not on the entire message as a whole. These processes do not necessarily occur in linear order, so a learner might move from process to process in many different ways. Multimedia learning takes place in the learner’s information processing system, a system that contains separate channels for visual and verbal processing, a system with serious limitations on the capacity of each channel, and a system that requires coordinated cognitive processing in each channel in order for active learning to occur.

Processing of printed words [special case; see p.77]: Words are initially be processed through the eyes. Certain incoming words are selected and brought into working memory as part of the visual image base.  By mentally pronouncing the images of the printed word these words can then get transferred into the word sound base. Once the words are represented in the auditory/verbal channel they are processed like spoken words.  When verbal material must enter through the visual channel, the words must take a complex route through the system, and must also compete for attention with the illustration that is also procesed in the visual channel.

Three kinds of cognitive load are: (p.79-81)

  • Extraneous cognitive processing (Sweller: “extraneous cognitive load”)
    • Cognitive processing that does not serve the instructional goal and is caused by poor instructional design
    • If extraneous processing consumes all the learner’s available cognitive capacity, the result is no learning, which is reflected in poor retention and poor transfer performance.
  • Essential processing (Sweller: “intrinsic cognitive load”)
    • Cognitive processing that is required to represent the material in working memory and is determined by the complexity of the material
    • One way to help learners process complex material is to provide pre-training in the names and characteristics of the key elements.
    • Corresponds to the “selecting” arrows, which indicate that the learner is building a representation of the materials in working memory.
    • If learners engage mainly in essential cognitive processing during learning, the result will be rote learning, as reflected in good retention and poor transfer performance.
  • Generative processing (Sweller: “germane cognitive load”)
    • Cognitive processing during learning that is aimed at making sense of the essential material and that can be attributed to the learner’s level of motivation.
    • Corresponds to the “organizing” and “integrating” arrows, which indicate deeper processing.
    • Generative processing may be primed by creating an engaging learning environment in which the narrator uses a conversational style and polite wording.

The objective of multimedia design is to reduce extraneous cognitive processing, manage essential cognitive processing, and foster generative cognitive processing.

One Response to “Multimedia Learning (2009) — Ch. 1-3 (Richard E. Mayer)”
  1. Aaron Chen says:

    Say Hello to You!
    I am now an international student in SFSU studying instructional technology. I was born in Shanghai. Glad to read your articles.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: