Focus on Formative Feedback (Shute, 2008)

Shute, V. J. (2008). Focus on Formative Feedback. Review of Educational Research, 78(1), 153 -189. doi:10.3102/0034654307313795

In this comprehensive review of the current literature on feedback, Shute organizes decades of work into major areas of consensus and controversy. Ultimately, Shute calls for an expanded view of feedback whereby situational and individual characteristics of the instructional context and learner are considered along with the nature and quality of a feedback message. Investigations conducted under such a paradigm could eventually lead to the development of adaptive formative feedback systems for use within interactive learning media.

[*Note: Much of this material was copied directly from Shute’s article.]

The main goal of formative feedback: (1)  to enhance learning, performance, or both, engendering the formation of accurate, targeted conceptualizations and skills; and (2) shape the perception, cognition, or action of the learner (e.g., Moreno, 2004; Schimmel, 1983; Wager & Wager, 1985).

Formative feedback:

  • Information communicated to the learner that is intended to modify thinking or behavior to increase knowledge, skills, and understanding in some content area or general skill (e.g., problem solving).
  • FF should be nonevaluative, supportive, timely, and specific.
  • Comes in a variety of types (e.g., verification of response accuracy, explanation of the correct answer, hints, worked examples, response specific, goal directed)
  • Factors such as individual characteristics of the learner and aspects of the task can interact with FF’s effectiveness.

“Researchers who have tackled the tough task of performing meta-analyses on the feedback data use descriptors such as ‘inconsistent,’ ‘contradictory,’ and ‘highly variable’ to describe the body of feedback findings (Azevedo & Bernard, 1995; Kluger & DeNisi, 1996). Ten years later those descriptors still apply” (p. 156).

Feedback produces negative effects on learning when it:

  • Is construed as critical or controlling (Baron, 1993)
  • Thwarts efforts to improve performance (Fedor, Davis, Maslyn, & Mathieson, 2001)
  • Indicates student’s standing relative to peers (via scores or grades) and is coupled with with low levels of specificity (i.e., vagueness) (Butler, 1987; Kluger & DeNisi, 1998; McColskey & Leary, 1985; Wiliam, 2007; Williams, 1997)
  • When it interrupts a student who is actively engaged in problem solving (Corno & Snow, 1986)

Directive feedback is focused on what needs to be fixed or revised. Specific compared. Facilitative feedback provides comments and suggestions to help guide students in their own revision and conceptualization (Black & Wiliam, 1998).

Cognitive Mechanisms of Formative Feedback

Several mechanisms have been proposed. Formative feedback can:

  • Signal a gap between a current level of performance and some desired level of performance or goal. Resolving this gap can motivate higher levels of effort (Locke & Latham, 1990; Song & Keller, 2001).
  • Reduce uncertainty about how well (or poorly) the student is performing on a task (Ashford, 1986; Ashford, Blatt, & VandeWalle, 2003). Reducing uncertainty may lead to higher motivation and more efficient task strategies.
  • Reduce the cognitive load of a learner, especially novice or struggling students (e.g., Paas, Renkl, & Sweller, 2003; Sweller, Van Merriënboer, & Paas, 1998). (Also see: Sweller et al., 1998 – worked examples; Moreno, 2004 – explanatory feedback)
  • Provide information that may be useful for correcting inappropriate task strategies, procedural errors, or misconceptions (e.g., Ilgen et al., 1979; Mason & Bruning, 2001; Mory, 2004; Narciss & Huth, 2004). The
    corrective function effects appear to be especially powerful for feedback that is more specific (Baron, 1988; Goldstein, Emanuel, & Howell, 1968).

Feedback Specificity

-More specific feedback may be more effective than less specific feedback.

Feedback specificity – the level of information presented in feedback
messages (Goodman, Wood, & Hendrickx, 2004). Specific (or elaborated) feedback provides information beyond their accuracy; more directive than facilitative.

Feedback lacking in specificity may cause students to view it as useless, frustrating, or both (Williams, 1997). It can also lead to uncertainty
about how to respond to the feedback (Fedor, 1991) and may require greater
information-processing activity on the part of the learner.

p. 158: Phye and Sanders (1994) tested two types of feedback (i.e., general advice vs. specific feedback, the latter providing the learner with the correct answer). They found that the more specific feedback was clearly superior to general advice on a retention task. However, they found no significant differences between feedback types on a transfer task. They caution against assuming that procedures that enhance performance during acquisition (e.g., providing specific feedback) will necessarily enhance transfer to new tasks.

Feedback verification and elaboration (p. 158-9)

  • Verification – simple judgment of whether an answer is correct. Can be done explicitly (checkmark) or implicitly (outcome in a simulation).
  • Elaboration – the informational aspect of the message, providing relevant cues to guide the learner toward a correct answer; also may explain why the selected response is wrong, and may indicate what the correct answer should be. Elaboration can  (a) address the topic, (b) address the response, (c) discuss the particular error(s), (d) provide worked examples, or (e) give gentle guidance. One type of elaboration, response-specific feedback, appears to enhance student achievement, particularly learning efficiency, more than other types of feedback, such as simple verification or “answer until correct” (e.g., Corbett & Anderson, 2001; Gilman, 1969; Mory, 2004; Shute, Hansen, & Almond, 2007).

Feedback complexity (p. 159)

If feedback is too long or too complicated, many learners will simply not pay attention to it, rendering it useless. Lengthy feedback can also diffuse or dilute the message.

Feedback complexity – how much and what information should be included in the feedback messages.

(p. 159) Schimmel (1983):  meta-analysis (15 studies) on feedback as used in computer-based instruction and programmed (scripted) instruction. The amount of information (i.e., feedback complexity) was not significantly related to feedback effects; also, feedback effects were significantly larger in computer-based than in programmed instruction.

Sleeman, Kelly, Martinak, Ward, and Moore (1989) noted that few studies have systematically compared the effects of different styles of error-based feedback, and of those that have, the results are inconclusive (see p. 159-60).

“Inconclusive findings on feedback complexity suggest that there may be other mediating factors involved in the relationship between formative feedback and learning. For instance, instead of feedback complexity, a more salient facet of feedback may be the nature and quality of the content, such as providing information about learning goals and how to attain them” (p. 161).

Goal-directed feedback and motivation

Goal-directed feedback – includes information about progress toward a desired goal (or set of goals) rather than discrete responses (i.e., responses to individual tasks).

Motivation and engagement depends on a close match between a learner’s goals and her expectations that these goals can be met (Fisher & Ford, 1998; Ford, Smith, Weissbein, Gully, & Salas, 1998). If goals are set unattainably high, the learner will likely experience failure and become discouraged. When goals are set too low, success loses its power to promote further effort (Birney, Burdick, & Teevan, 1969).

Learner’s goal orientation (learning orientation or performance orientation) should be considered (Dempsey et al. 1993). Hoska (1993) showed how goal-orientation feedback can modify a learner’s view of intelligence.

Feedback as a cognitive support mechanism

Some research has shown that directive feedback (providing corrective information) may actually be more helpful than facilitative (providing guidance and cues), particularly for novice learners (e.g., Knoblauch & Brannon, 1981; Moreno, 2004). Thus, it may be useful to consider scaffolded feedback, which includes models, cues, prompts, hints, partial solutions, and direct instruction (Hartman, 2002).

Feedback Timing

Should feedback be delivered immediately or delayed? Immediate feedback may prevent errors from being encoded into memory, though delayed feedback may reduce proactive interference, thus allowing the initial error to be forgotten and the correct information to be encoded with no interference (see Kulhavy & Anderson, 1972) (p. 163-4).

Case for delayed feedback: Delayed feedback has often been shown to be as effective as immediate feedback. The interference-perseveration hypothesis (Kulhavy & Anderson, 1972) states that initial errors do not compete with to-be-learned correct responses because errors are likely to be forgotten and thus cannot interfere with retention. The delay-retention effect (DRE) is supported by Anderson and colleagues (e.g., Kulhavy & Anderson, 1972; Surber & Anderson, 1975), and not supported by others (e.g., Kippel, 1974; Newman, Williams, & Hiller, 1974; Phye & Baller, 1970).  Schroth (1992) reported that although delayed feedback slowed the rate of initial learning, it facilitated transfer after the delay.

Case for immediate feedback: The earlier corrective information is provided, the more likely it is that efficient retention will result (Phye & Andre, 1989). Immediate feedback has been shown to be effective for acquisition of verbal materials, procedural skills, and some motor skills (Anderson, Magill, & Sekiya, 2001; Brosvic & Cohen, 1988; Corbett & Anderson, 1989, 2001; Dihoff, Brosvic, Epstein, & Cook, 2003).

Azevedo and Bernard (1995) meta-analysis on feedback and computer-based instruction: Effect size of feedback versus no feedback relating to immediate outcomes was 0.80 (22 CBI studies). Delayed outcome conditions resulted in a mean weighted effect size of just 0.35 (9 studies).

Schroth (1992); Corbett and Anderson (2001): delayed feedback may be superior for promoting transfer of learning, especially for concept-formation tasks; immediate feedback may be more efficient, particularly in the short run and for procedural skills (i.e., programming and mathematics).

Schmidt, Young, Swinnen, and Shapiro (1989): Longer delays between
feedback episodes resulted in relatively poorer performance during acquisition but better retention compared with shorter delay conditions.

Mathan and Koedinger (2002): Feedback effectiveness depends not on the main effect of timing but on the nature of the task and the capability of the learner. There are possible interactions involving timing effects and optimal ways to match feedback (type and timing) to learning tasks and students’ individual needs or characteristics (e.g., Schimmel, 1988; Smith & Ragan, 1999). If the task is difficult, then immediate feedback is beneficial, but if the task is easy, then delayed feedback may be preferable (Clariana, 1999).

Immediate vs. delayed feedback

Immediate feedback may facilitate the decision or motivation to practice as well as provide the explicit association of outcomes to causes. However, it may also facilitate reliance on information that is not available during transfer and promote less careful or mindful behavior.

Delayed feedback may encourage learners’ engagement in active cognitive and metacognitive processing, thus engendering a sense of autonomy (and perhaps improved self-efficacy). However, delaying feedback for struggling and less motivated learners may prove to be frustrating and detrimental to their knowledge and skill acquisition.

Feedback and other variables:

  • Learner level:
    • Low-achieving students may benefit from immediate feedback, whereas high-achieving students may prefer or benefit from delayed feedback (Gaynor, 1981; Roper, 1977).
    • Clariana (1990): low-ability students benefit from receipt of correct response feedback more than from try again feedback
    • Hanna (1976): verification feedback produced the highest scores for high-ability students and elaborated feedback produced the highest scores for low-ability students. There were no significant differences between verification and elaborated feedback for middle-ability students.
  • Response Certitude
    • Kulhavy and Stock (1989): when students are certain their answer is correct, they will spend little time analyzing feedback, and when students are certain their answer is incorrect, they will spend more time reviewing feedback.
    • Mory (1994): although there were differences in the amount of feedback study time, there was no significant learning effect for feedback tailored to response certitude and correctness.
  • Goal Orientation
    • More specific feedback should be used for learners with either high-performance or low-learning goal orientations.
  • Normative feedback
    • McColskey and Leary (1985): low-achieving students should not receive normative feedback but should instead receive self-referenced feedback—focusing their attention on their own progress.

Toward a Framework for Formative Feedback

Kluger and DeNisi (1996) (p. 168-70)

  • Feedback intervention theory (FIT): FIs change the locus of a learner’s attention among three levels of control: (a) task learning processes (task details), (b) task motivation processes (focal task), and (c) metatask processes (self). FI cues affect performance by changing the locus of attention.
  • The lower in the hierarchy the FI-induced locus of attention is, the stronger is the benefit of an FI for performance, i.e., formative feedback that focuses the learner on aspects of the task promotes learning and achievement compared to FIs that draw attention to the self.
  • FIT consists of five interdependent and consecutive arguments: (a) behavior is regulated by comparisons of feedback to goals or standards, (b) goals or standards are organized hierarchically, (c) attention is limited and therefore only feedback–standard gaps that receive attention actively participate in behavior regulation, (d) attention is normally directed to a moderate level of the hierarchy; and (e) FIs change the locus of attention and therefore affect behavior.
  • The meta-analysis showed the following to be effective: (a) velocity FIs (i.e., “self-referenced” feedback that addresses a change from the learner’s prior performance) and (b) correct response FIs. Also, FI effects on performance of physical tasks are lower than FI effects on cognitive tasks.
  • Types that reduce FI effects:(a) discouraging FIs, (b) praise, (c) FIs threatening self-esteem, and (d) orally delivered FIs (from the instructor).
  • FIs that provide frequent messages enhance FI effects, and FI effects are stronger for memory tasks and weaker for more procedural tasks.
  • Computerized FIs yield stronger effects than noncomputerized FIs
  • FIs in the context of complex tasks yield weaker effects than for simpler tasks
  • FIs are more effective with a goal-setting intervention than in the absence of goal setting.
  • In more than one third of the 607 cases, FIs reduced performance.

Bangert-Drowns et al. (1991)

  • To direct behavior, a learner needs to be able to monitor physical changes brought about by the behavior. That is, learners change cognitive operations and thus activity by adapting it to new information and matching it with their own expectations about performance.
  • 5-stage learning cycle: current state -> search/retrieval -> response -> evaluation -> adjustment
  • Verification feedback (correct–incorrect) resulted in lower effect sizes compared to correct response feedback (i.e., providing the correct answer).
  • Feedback can promote learning if it is received mindfully. Conversely, feedback can inhibit learning if it encourages mindlessness, as when the answers are made available before learners begin their memory search,
    or if the feedback message does not match students’ cognitive needs (e.g., too easy, too complex, too vague).

Narciss and Huth (2004)

  • Adapting the content, function, and presentation format of the feedback message should be driven by considerations of the instructional goals and learner characteristics to maximize the informative value of the feedback.

Instruction and Learner factors interact with Feedback to influence learning:

  • Instruction: (a) the instructional objectives (e.g., learning goals or standards), (b) the learning tasks (e.g., knowledge items, cognitive operations, metacognitive skills), and (c) errors and obstacles (e.g., typical errors, incorrect strategies, sources of errors).
  • Learner: (a) learning objectives and goals; (b) prior knowledge, skills, and abilities (e.g., domain dependent, such as content knowledge, and domain independent, such as metacognitive skills); and (c) academic motivation (e.g., one’s need for academic achievement, academic self-efficacy, and metamotivational skills).
  • Feedback: (a) the content of the feedback (i.e., evaluative aspects, such as verification, and informative aspects, such as hints, cues, analogies, explanations, and worked-out examples), (b) the function of the feedback (i.e., cognitive, metacognitive, and motivational), and (c) the presentation of the feedback components (i.e., timing, schedule, and perhaps adaptivity considerations).
  • Specific steps for generating effective formative feedback include selecting and specifying learning objectives (concrete learning outcomes), identifying learning tasks, matching to learning outcomes, and after conducting cognitive task and error analyses, specifying information (i.e., formative feedback) that addresses specific, systematic errors or obstacles.

Mason and Bruning (2001)

  • Examined feedback for CBI systems. Considered type of feedback and level of elaboration in relation to student achievement level, task complexity, timing of feedback, and prior knowledge.
  • Immediate feedback for students with low achievement levels in the context of either simple (lower level) or complex (higher level) tasks is superior to delayed feedback, whereas delayed feedback is suggested for students with high achievement levels, especially for complex tasks.

Summary

(Azevedo & Bernard, 1995; Birenbaum & Tatsuoka, 1987; Cheng, Lin, Chen, & Heh, 2005; Cohen, 1985; Kulhavy, 1977; Sales, 1993; Sleeman et al., 1989)

In general, formative feedback:

  • should address the accuracy of a learner’s response to a problem or task
  • may touch on particular errors and misconceptions
  • permit the comparison of actual performance with some established standard of performance (Johnson & Johnson, 1993).

Feedback from a trustworthy source will be considered more seriously
than other feedback, which may be disregarded. This may explain why computer-based feedback is often better than human-delivered because perceived biases are eliminated (see Kluger & DeNisi, 1996).

Although hints can be facilitative, they can also be abused (e.g., Aleven & Koedinger, 2000; Shute, Woltz, & Regian, 1989). Consider using prompts and cues (i.e., more specific kinds of hints).

Exploit the potential of multimedia to avoid cognitive overload due to modality effects (e.g., Mayer & Moreno, 2002) and do not default to presenting feedback messages as text. Instead, consider alternative modes of presentation (e.g., acoustic, visual).

Effective and useful feedback depends on (a) motive (the student needs it), (b)
opportunity (the student receives it in time to use it), and (c) means (the student is able and willing to use it). Several meta-analyses found that feedback generally
improves learning, ranging from about .40 SD (Guzzo, Jette, & Katzell, 1985) to
.80 SD and higher (Azevedo & Bernard, 1995; Kluger & DeNisi, 1996)

More research is needed examining the interactions among task characteristics, instructional contexts, and student characteristics that potentially mediate feedback effects. One reason studies are so inconsistent may be a function of individual differences among motivational prerequisites (e.g., intrinsic
motivation, beliefs, need for academic achievement, academic self-efficacy, and
metacognitive skills).

Vygotsky (1987) noted that the study of psychology had been damaged by the separation of the intellectual from the motivational and emotional (or affective) aspects of thinking. Crafting and delivering formative feedback may help bridge these “aspects of thinking” and enhance learning (see Goleman, 1995; Mayer & Salovey, 1993, 1997; Picard et al., 2004 — emotional upsets can interfere with mental activities (e.g., anxious, angry, or depressed students do not learn). Thus, one intriguing area of future research is to systematically examine the relationship(s) between affective components in feedback and outcome performance. And although there have been inroads in the area, according to Picard et al. (2004), extending cognitive theory to explain and exploit the role of affect in learning is still in its infancy.

Cognitive task and error analyses may be used to match formative feedback components to (a) learning objectives, (b) skills needed for the mastery of the task, and (c) typical errors or incorrect strategies. However such expensive analyses and methods may not, in fact, be necessary to promote learning (e.g., Sleeman et al., 1989).

Future research may examine (a) additional learner characteristics and (b) links between different types of knowledge and feedback types. For instance, feedback to support fact learning (declarative knowledge) could reiterate definitions or provide the learner with a handy mnemonic technique; feedback to support conceptual knowledge could provide examples, counterexamples, and big pictures; and feedback to improve procedural knowledge could involve demonstrations, solution paths (complete or partial), and so forth. Ultimately, information about the learner, combined with information about desired outcomes, may inform the development of adaptive formative feedback. Various feedback types could be generated and incorporated into a program (or generated on the fly based on formative feedback models) and then accessed and delivered according to the characteristics of the learner in conjunction with the nature of the task and instructional goals.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: