Direct measurement of cognitive load in multimedia learning (Brunken, Plass & Leutner)

Brünken, R., Plass, J.L., & Leutner, D. (2003). Direct measurement of cognitive load in multimedia learning. Educational Psychologist, 38, 53-61.

Research on multimedia learning uses cognitive load as a theoretical rationale to explain differences in learning outcomes, yet often such findings lack supporting evidence in the form of measurement of cognitive load on the learners themselves. Furthermore, when measurement of cognitive load is conducted, it is usually done via indirect and subjective methods. In this article, the authors propose a direct approach based on a dual-task exercise. A dual-task approach has many advantages to previous methods, including the ability to gather temporal data on “loading,” to uncover the specific processes involved, and greater experimental rigor via a “within-subjects” design.

Instructional strategies that may reduce extraneous cognitive load and optimize germane cognitive load:

  • worked examples (Kalyuga, Chandler, Tuovinen, & Sweller, 2001)
  • goal-free activities (Sweller, 1999)
  • strategies of imagining (Cooper, Tindall-Ford, Chandler, & Sweller, 2001)
  • activities designed based on the completion effect (van Merrienboer, Schuurman, de Croock, & Paas, 2002) [read more]
  • activities designed based on the modality effect (Brunken &Leutner, 2001; Mayer & Moreno, 2003; Sweller, 1999)
  • activities designed based on the redundancy effect (Sweller, 1999).

“The same learning material can induce different amounts of memory load when different instructional strategies and designs are used for its presentation, because the different cognitive tasks required by these strategies and designs are likely to result in varying amounts of extraneous and germane load” (p.54).

Free cognitive resources = Cognitive capacity in working memory – Total cognitive load

Generative theory of multimedia learning (Mayer, 2001) – process theory that supplements CLT in describing the cognitive processes involved in multimedia learning. Based on the combination of two foundational assumptions:

  • dual-coding assumption (presentation mode) – verbal and pictorial material are processed and represented in separate but interconnected systems.
  • dual-channel assumption (sensory perception mode) – visual and auditory information are processed in visuospatial and phonological subsystems, respectively (Baddeley, 1986, 1999).

The principles that describe the effect of instructional design on cognitive load, organized into three broad categories:

  1. Multimedia effect, split-attention effect, and contiguity effect – reduce extraneous load by enhancing information integration and schema construction (Brünken, Steinbacher, Schnotz, & Leutner, 2001; Mayer, 2001)
  2. Individual differences effect and expertise reversal effect – related to the individual learner’s amount of available capacity (Plass, Chun, Mayer, & Leutner, 1998, in press; Kalyuga et al., 2003)
  3. Modality effect, redundancy principle, and coherence principle – concerned with optimizing the use of available capacity for information processing (Brünken & Leutner, 2001; Mayer, 2001; Mayer & Moreno, 1998).

***

Measurement of cognitive load

The authors have classified the various methods of measuring cognitive load along two dimensions: objectivity (subjective or objective) and causal relation (direct or indirect).

One of the most common methods is to collect performance outcome measures (objective, indirect). An example of a behavioral measure would involve eye-tracking analysis (Paas et al., 2003).

The use of neuroimaging techniques (PET scans and fMRI) to measure brain activity during task execution represents an objective, direct method. However, there are important practical limitations (only simple tasks, performed in non-authentic learning situations).

A popular method used in experimental psychology involves the dual-task paradigm. It is based on the assumption of limited cognitive resources that can be allocated flexibly to different requirements of task execution.

The dual-task approach can be used in two ways:

  1. A secondary task is added to a primary task with the intention of inducing memory load. The dependent variable of interest is the performance in the primary task, which should decrease in a dual-task condition compared to a single-task condition.
  2. A secondary task is used for the measurement of the memory load induced by a primary task. Here, the performance in the secondary task is the variable of interest. If different variants of a primary task induce different amounts of memory load, then the performance in the secondary task will vary accordingly.

Advantages of the dual-task approach, compared to previously discussed methods for measuring cognitive load:

  • Since primary and secondary tasks are attended to at the same time, it is possible to measure cognitive load at the very point in time when the load is induced in the learner. Subjective rating scales of mental effort, for example, can only be reasonably applied after the task execution.
  • By using different secondary tasks that are linked to different process steps of information processing, such as perception, preprocessing in one of the slave systems, or information integration (Baddeley, 1986), it is possible to identify in which of the process steps the cognitive load is imposed.
  • Measuring the load induced by different design variants of multimedia instruction for the same learner makes the load measurement independent from individual differences, such as abilities, interest, or prior knowledge, that are known
    to affect learning outcomes in between-subjects designs.

Some methodological and technical concerns when constructing a dual-task approach:

  • For the secondary task to be a sensitive measure it has to require the same cognitive resources as the primary task; otherwise, secondary task performance will be independent of primary task performance.
  • The performance measure for the secondary task has to be reliable and valid.
  • The secondary task has to be so simple that it does not suppress simultaneous learning processes; otherwise, the secondary task may affect the learning outcome of the primary task, the multimedia instruction, by requiring cognitive resources that are no longer available for learning (Marcus et al., 1996). Because primary task performance is not the variable of interest in this context, this might not be a problem as long as the learning process is not suppressed in general.
  • The secondary task has to be able to consume flexibly all of the available free cognitive capacity.

“Reaction time in a secondary monitoring task is a valid measure of cognitive load induced by the multimedia instruction serving as primary task” (Verwey & Veltman, 1996; Wickens, 1984). (p.57).

Experimental evidence:

  • Textbook-based: Marcus et al. (1996) used different forms of simultaneously presented secondary tasks for assessing cognitive load induced by different instructional designs of visual primary tasks. Britton, Glynn, Meyer, and Penland (1982) found statistically significant relations between secondary task performance and knowledge acquisition from differently structured texts.
  • Computer-based:  Reed et al. (1985) studied the effects of writing ability and mode of discourse on cognitive capacity engagement for different computer-based writing tasks with different levels of difficulty. Cognitive engagement of learners increased from the easiest task that induced a low level of cognitive load to the moderately difficult task with a medium level of load, but it actually decreased for the most difficult task that induced the highest levels of load [cognitive disengagement=reversal effect]. Chandler and Sweller (1996) reported findings on the usability of secondary tasks for computer-based learning materials for computer-aided design using a secondary task generated by a second computer that was presented simultaneously on a separate computer screen. Secondary task performance was related to split-attention and redundancy effects in the primary task as well as to element interactivity (i.e., intrinsic load) of the material.

The authors have studied the feasibility of the dual-task approach to measure cognitive load in multimedia learning using the modality effect as an example (Brünken et al., 2002).

Brief description: A visual secondary reaction time task (color of letter “A” at top of screen) was added to two different multimedia learning tasks. The primary task was a multimedia learning environment (human cardiovascular system or travel guide). Pages alternated btw visual-only format and audio-visual format to conform to an within-subjects design. Three experimental conditions: (1) secondary task alone; (2) visual-only learning material a primary task; (3) audiovisual learning material as primary task. Repeated measurements of reaction time were recorded at random intervals.

Results: Performance on secondary task was significantly faster in single-task condition compared to dual-task conditions, indicating that the secondary task is indeed a sensitive measure that requires the same cognitive resources as primary task. Also, reaction times were faster for the audiovisual primary task than for the visual-only primary task, as predicted by CLT (effect sizes were .82 and .75 — wow!). Effect pattern observed in all participants.

“The dual-task method of measuring cognitive load for multimedia learning represents a promising alternative to existing methods. The most important benefit of this method is that it is able to measure cognitive load without relying on subjective self-reported data, indirect performance measures, or behavioral or physiological measures that have only an indirect causal link to cognitive load and that may, therefore, be affected by other factors. Instead, it provides, similar to brain activation measures, a direct way to measure cognitive load. In comparison to other methods, such as neuroimaging techniques, heart rate measures, or eye tracking, the technical apparatus required for this method is minimal and easily implemented” (p.59).

Some important considerations:

  • The use of the method depends on the sensory modality of the information. The investigation of acoustical information, for example, would require the use of auditory secondary tasks.
  • The use of reaction time measures requires within-subjects designs.
  • The secondary task, even if it is designed as a very simple task requiring few resources, may still affect the learning outcomes in the primary task. Future studies should measure both the primary and secondary task performance simultaneously to validate differences in cognitive load and knowledge acquisition caused by different instructional designs within the same experiment.
  • Reed et al. (1985) reporting of a reversal effect indicates that the “additive component model of CLT” may need to be refined to “incorporate interactions such as those between intrinsic load, germane load, and learners’ level of experience” (p.59).
  • Another interesting issue concerns the relation between cognitive load and attentional processes (i.e., possible costs involved with switching btw different areas or instructional design strategies within the same learning environment).
Advertisements
Comments
One Response to “Direct measurement of cognitive load in multimedia learning (Brunken, Plass & Leutner)”
  1. Eunbae says:

    Hey, I came across your blog as I was searching for this article in Google. Happy to see it. I have a similar blog that help me survive in grad school mostly revolving around research interests. Good luck!

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: