The instructional effect of feedback in test-like events (Bangert-Drowns, et al., 1991)

Bangert-Drowns, R.L., Kulik, C.-C, Kulik, J.A., & Morgan, M. (1991). The instructional effect of feedback in test-like events. Review of Educational Research, 61(2), 213-238.

The authors performed a meta-analysis reviewing 58 effect sizes from 40 reports. Feedback effects were found to vary with control for presearch availability, type of feedback, use of pretests, and type of instruction and could be quite large under optimal conditions. Also, mediated intentional feedback for retrieval and application of specific knowledge appears to stimulate the correction of erroneous responses in situations where its mindful (Salomon & Globerson, 1987) reception is encouraged.[*Note: Much of this material was excerpted from the original article.]

“This principle, that the consequences of action have enduring effects on future performance, has survived the transition from a behavioristic to a cognitive learning paradigm (Shuell, 1980, 1986). Behavioral technology asserted that the consequences of a behavior in the presence of a discriminative stimulus could reinforce or suppress the future association of the behavior with the discriminative stimulus. This principle grounded the design of programmed instruction. Programmed instruction would prompt the learner to emit a target behavior and then reinforce that behavior by confirming it” (p. 214).

The cognitive view of learning equally emphasizes the consequences of behavior, but it sees their operation as more than the mere strengthening of automatic stimulus response associations. Behavioral consequences serve as sources of new information necessary for verification of retrieval accuracy, concept development, skill refinement, and metacognitive adaptation. In this view, feedback empowers active learners with strategically useful information, thus supporting self-regulation.

Any theory that depicts learning as a process of mutual influence between learners and their environments must involve feedback implicitly or explicitly because, without feedback, mutual influence is by definition impossible. Hence, the feedback construct appears often as an essential element of theories of learning and instruction.

External feedback is not helpful when students are capable and willing to produce their own feedback or when a task is so easy that performance information is unnecessary (Clark, Aster, & Hession, 1987; Corno & Snow, 1986; Snow & Lohman, 1984).

Ways to categorize feedback:

  • Intentional/Incidental.
    • Intentional fb concerns the appropriateness (quality, correctness, etc.) of relatively specific aspects of performance. Typifies direct or expository instruction.
    • Fb may also simply be an incidental consequence of natural interactions with the social and physical environment. Present in unstructured cooperative peer interactions (Webb, 1982), the use of computer tools, or unguided simulations (Kozma & Bangert-Drowns, 1987).
      • Highly notational symbol systems (Salomon, 1979) such as language, mathematical symbols, or graphs often used for intentional fb because their capacities for specificity and flexibility.
  • Delivery
    • Fb can be delivered via direct interpersonal action (between teacher and student or among students) or mediated through intervening agents (computer-based instruction).
  • Target
    • affective dimensions of learning: supporting intrinsic motivation (Csikszentmihalyi, 1978) or self-attributions (Dweck, 1975).
    • self-regulated learning by cuing self-monitoring and other metacognitive processes (Meichenbaum, 1985).
    • correctness of a task.
  • Content (Kulhavy & Stock, 1989)
    • load – the total amount of information given in the feedback from simple yes-no statements to fuller explanations,
    • form – the structural similarity between information as presented in feedback compared to the instructional presentation
    • type of information – whether feedback restated information from the original task, referred to information given elsewhere in the instruction, or actually provided new information
  • Operation (Kozma & Bangert-Drowns, 1987): error correction, presentation of prototypic responses, display of the consequences of responses, and explanation of the appropriateness of responses.

Some important outcomes of Kulhavy’s research:

  • response perseveration theory – a student’s response to test-like items tends to naturally perseverate. Therefore, fb’s most important instructional effect would be to correct erroneous responses not to strengthen correct responses, as behaviorists argued.
  • effectiveness of error correction depended on student expectation. When students discovered that a “sure” answer was actually incorrect, they were more likely to study the feedback than if they had originally been uncertain of their answer (Kulhavy & Stock, 1989; Kulhavy, Yekovich, & Dyer, 1976)
  • introduction of even a small delay between response and feedback helped to eliminate proactive interference and thus increased the impact of error-correcting feedback.
  • presearch availability – the availability of fb before students construct their own answers to questions. In studies where presearch availability is uncontrolled, students could simply copy feedback answers and, therefore, not study the material at all. Students in uncontrolled presearch availability conditions showed lower posttreatment achievement, lower error rates during instruction, and less time to complete the instruction than no-feedback control groups; these are symptoms of simple answer copying.

Schimmel (1983): the amount of information was nonsignificantly related to feedback effects. But feedback effects were significantly larger in computer-
based instruction than in programmed instruction [this could be do to presearch availability in PI but not CBI].

Kulik and Kulik (1988): In tasks that seemed to present greater cognitive
demands, such as more complex conceptual learning in applied situations or list
learning where the feedback gave only the correct response without repeating the
stimulus, immediate feedback was superior to delayed feedback.

Findings of previous researchers and reviewers can be organized and extended in a five-stage model describing the state of the learner receiving mediated intentional feedback for retrieval accuracy:

  1. Initial state: The student’s experience with the instruction and any previous question/feedback sequences will shape the student’s knowledge, interests, goals, and self-efficacy. When exposed to pretests or when given instruction in redundant, prompted, small units of information, students’ expectations for success may be so high that they are less mindful of feedback to their responses. The same would be true if students were bored or disinterested because the instruction was too simple or too structured.
  2. Activation of search and retrieval strategies: When feedback is made available before students construct their responses, it can in no way be used to evaluate responses. When students simply copy answers, this short-circuits any mindful reflection that could have been prompted by searching memory, responding, and evaluating the response. Search and retrieval strategies are activated by a question. It is presumed that information stored in a richer context of elaborations would be easier to locate in memory because there are more pathways providing access to the information (meta-analysis did not confirm).
  3. The learner responds to the question. The learner feels some degree of certainty about the response and thus has some expectation about what the feedback will indicate.
  4. The learner evaluates the response in light of information given in feedback. The nature of the evaluation may depend on the learner’s expectation about feedback and on the feedback’s actual nature. Kulhavy (1977) and Kulhavy and Stock (1989) argue that the degree of response certitude strongly influences the way feedback is used in response evaluation. If the learner was sure of the response and feedback confirmed its correctness, the retrieval pathway might be strengthened, but the knowledge would otherwise be unaltered. If the learner was sure of the response and feedback indicated its incorrectness, the learner could set out to understand the incongruity. If the learner was originally unsure about the response, feedback confirmation or disconfirmation would be less likely to stimulate deep reflection unless the learner was genuinely interested in acquiring the instructional content. (HOWEVER, Chanond, 1988; Kulhavy, Yekovich, & Dyer, 1976; Lhyle & Kulhavy, 1987, Experiments 1 & 2 — there was no clear relation between initial confidence, confirming or disconfirming feedback, and posttest performance.)
  5. Adjustments are made to relevant knowledge, self-efficacy, interests, and goals as a result of the response evaluation. These adjusted states, with subsequent experiences, determine the next initial state.

In summary, a learner comes to instruction in some initial state that is a configuration of interests, goals, degree of self-efficacy, and degree of prior relevant knowledge. Search and retrieval strategies are activated during or after instruction by questions. The learner constructs a response to the question. The learner then evaluates the response on the basis of subsequent feedback. After response evaluation, the learner makes adjustments to knowledge, level of motivation, and assessment of self-efficacy.

The construct of mindfulness can further help explain how feedback operates in
this five-stage framework. Mindful activities are exactly the opposite of automatic, overlearned responses. There are many ways in which feedback can promote or inhibit mindful learning.

  • Presearch availability is likely to diminish mindfulness by stopping the learner at Stage 1 and short-circuiting the response-evaluation-adjustment process.
  • Confirming or disconfirming feedback is likely to stimulate mindfulness, perhaps especially when a “sure” response is contradicted.
  • Feedback can additionally inform mindful retrieval correction and suggest additional elaboration when it supplies the correct answer and provides explanations.
  • When instruction is easy, rote, or redundant, the need for feedback is diminished, and it is likely to receive less mindful attention.

Different feedback treatments:

  • Type of feedback: Right/wrong feedback; fb that supplied the correct answer if the first attempt was incorrect; fb that required students to answer a question repeatedly until they got the correct answer; fb that offered extended explanations of why particular answers were correct or incorrect.
    • Right/wrong feedback has virtually no effect on achievement (ES = -0.08). When learners are guided to or given the correct answer, the average effect of feedback is 0.31 standard deviations.
    • Effective feedback should not only signal the correctness or incorrectness of an answer but also provide correction when necessary. (See Heald, 1970; Sassenrath & Gaverick, 1965.)
    • When feedback merely indicated that a response was correct or incorrect, it resulted in a lower effect than when the feedback in some way informed the learner of the correct answer.
  • Timing of feedback. Feedback was given either immediately after each response, immediately after an entire test was completed, or after a delay of a day or more.
  • Counts for grade. Did performance on the criterion measure count toward a course grade?
  • Instructional item type. Includes completion, multiple choice, short answer, or mixed types of items.
  • Error rate during instruction. Some researchers recorded the number of errors students made on their first attempts to answer the test-like items during instruction. These errors were recorded here as the percentage of items answered incorrectly on the first try.

Students receiving feedback were more likely than controls to correct their errors (in a post-test). Feedback and no-feedback groups were about equally likely to answer criterion questions correctly when they had answered the same item correctly during instruction.

The author’s meta-analysis examined instructional uses of feedback in four kinds of instructional settings: programmed instruction, computer-assisted instruction, texts with adjunct questions, and conventional testing. Such feedback would be classified as mediated intentional feedback directed at refining knowledge retrieval and application. Feedback research in these areas does not address the various forms of interpersonal feedback nor does it include the nonverbal, consequential feedback typical of self-initiated inquiry, simulations, and uses of instructional tools. However, the kinds of feedback included here are commonly found in classroom teaching and are of special importance to text-based and technology-based instruction (p. 230).

“…one is struck by the counterintuitive finding, previously observed by Kulhavy and others, that feedback does not always increase achievement and, in fact, is sometimes associated with decrements in achievement. A full third of the 58
findings were negative; four of these were significantly negative. Effect sizes ranged, however, from very low to very high in magnitude” (p. 232).

In test-like events, feedback’s primary importance is in correcting errors. the probability that an item on the dependent measure would be answered correctly when the same item was answered correctly during instruction was very high regardless of the presence or absence of feedback. However, when an item was
answered incorrectly during instruction, it was much more likely to be answered
correctly in the feedback condition than in the no-feedback condition. Also, a significantly large positive correlation between error rate and effect size again shows that, as students make more errors during instruction, feedback has a more important effect on later retrieval of correct information (p. 232).

Feedback in programmed and computer-based instruction was less effective than in text comprehension and conventional testing situations, even after accounting for presearch availability in many programmed instruction studies. It is probably fair to generally characterize PI and CBI as presenting redundant and heavily cued questions on very small fragments of information. Questions in text comprehension and testing situations tended to review larger and more complex units of content. Taken with the observation of pretest effects, it would seem that feedback is more important when the content is more complex and when the student is given fewer cues, organizers, and other instructional supports (p. 233).

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: