Design-based research in education (Bell, Hoadley, & Linn, 2004)

Bell, P., Hoadley, C. M., & Linn, M. C. (2004). Design-based research in education. In M. C. Linn, E. A. Davis, & P. Bell (Eds.), Internet environments for science education (pp. 73–84). Mahwah, NJ: Lawrence Erlbaum Associates.

The authors describe and discuss the utility of design narratives in DBR. They also point to various types of design principles that a research team must strive to capture, in order to convey to the larger community any design knowledge and refinements in learning theory that was gained through a series of design experiments.

Historical underpinnings/”research trunk”: “Design-based research studies continue a tradition begun over 100 years ago. At the laboratory school at the University of Chicago, Dewey (1896) pioneered a research model that employed the systematic study of teaching and learning associated with the enactment of complex educational interventions. As Dewey (1929) later explained in detail, he sought to make the act of teaching less ad hoc and mystical. Research traditions that follow in Dewey’s footsteps include design experiments (Brown, 1992; Collins, 1992), design studies (Bell, in press; Linn & Hsi, 2000), local sciences research (diSessa, 1991), teaching experiments (Steffe & Thompson, 2000), and design-based research in education (Design-Based Research Collective, 2003; Hoadley, 2002)” (p. 74).

“Design studies put the development of educational approaches in a reciprocal relation with research that involves theorizing about learning, cognition, and development. Design-based research thereby affords the opportunity to develop detailed design knowledge (e.g., design principles, curriculum design patterns) while simultaneously advancing theoretical knowledge of learning and cognition (diSessa, 1991). As researchers, we seek to make our research increasingly cumulative, cohesive, repeatable, and inspectable” (p. 74).

Associated research paradigms (and why they are incomplete by themselves, based on what we, as educational researchers, hope to accomplish): “The systemic nature of education combined with rapid, often unanticipated, changes in the context of teaching and learning calls for new methods to conduct interventionist research in naturalistic settings. We draw on established research paradigms in cognitive anthropology (e.g., Hutchins, 1995; Lave, 1987), cognitive psychology (e.g., Anderson & Schunn, 2000; Bjork, 1999; Kintsch, 1998), and action research (e.g., Masters, 1995) while also investigating new approaches for data collection and analysis as necessitated by the research. Controlled laboratory studies of learning and cognition in the cognitive psychology tradition can only partially inform the systemic and enduring issues in education. Detailed observational studies of “cognition-in-the-wild,” as found in the cognitive anthropology tradition, add to our understanding but fall short of embracing the designed nature of real-world contexts and typically lack the interventionist mandate associated with design-based research. Action research can be considered an established design-based research approach that weds research activities to the concerns of practice (Masters, 1995), although it has been stereotyped (fairly or unfairly) more as political action rather than research” (p. 75).

There are several reasons for adopting a design-based approach:

  1. Need for interventions to be localized based on context of use. “Student populations very greatly in terms of experience, fluency  with language, technology and science as well as cultural norms for communicating and learning. This diversity leads to many educational opportunities and challenges with regard to the promotion of innovation. It calls for creative customization of innovations and contradicts a one-size-fits-all approach to curriculum, instruction, and assessment. Educational designers and teachers can leverage off of the characteristics and practices of specific communities, although significant effort is typically involved. Still, design studies allow for the sensible local customization and appropriation of materials, tools, and approaches originally designed for use with other participants” (p. 76).
  2. Constantly changing state of scientific knowledge. “To respond to the rapid expansion of scientific knowledge, educational designers and teachers need to regularly refine curriculum for future scientists and nonscientists alike…Design studies allow one to explore new educational opportunities and the concomitant issues associated with advances in one’s understanding of the natural world” (p. 76).
  3. Constantly changing state of technology (and the need to contextualize and integrate it thoughtfully into the education enterprise.) “New technological possibilities outstrip the ability to research their implications for education. As the Internet reaches more more schools and families, the possibilities for engaging with information and for interpersonal interaction increase. Beyond proving the feasibility of new technologies, researchers need to understand how new forms of technology can be productively embedded into larger systems of human activity (curricula, intellectual investigations, group activities, disciplinary inquiry). Design-based approaches provide for such contextualization and integration of technology in educational practice” (p. 76).

“Clearly, the complexity of the settings, multiple interacting trajectories, pressing societal problems, and the new possibilities represented by emerging technologies justify design-based research. These studies use methods matched to the complexity of the endeavor. They help understand and distinguish real-world contexts of learning” (p. 76).

Design based research methods:

1) Partnerships: successful teams comprise teachers, technologists, educational researchers, and disciplinary experts.

2) Compelling comparisons: which are “two forms of the innovation are enacted under otherwise similar conditions…Compelling comparisons test hypotheses with strict methodological techniques such as controlled comparisons with random assignment, double-blind coding of outcomes and longitudinal studies. These techniques yield more when accompanied by rich  understanding of the context of research” (p. 77).

Design Narratives

“Design narratives tell the story of the events leading up to, during, and following the investigation. Design narratives help make sense of design studies by including compelling comparisons as well as more informal research” (p. 79).

“…because our interventions as researchers are culturally embodied the complexity of human nature may prevent us from adequately and completely describing our research context…A second, related idea is that educational research is often naturalistic and may be quasi-experimental, correlational, or descriptive. Researchers rarely control every variable, every aspect of experience in and around a classroom, much less the out-of-school experiences students and teachers bring to their classroom lives. Because researchers cannot precisely engineer learning contexts, replications vary many factors. There is an art to identifying relevant aspects of the research context and to communicating results that include potentially relevant factors” (p. 78).

“…the goal is to conduct research that leads to locally grounded theories and findings, and through application by experienced practitioners in other contexts, to uncover just how localized or generalizable the findings are. Design principles are one important way of providing these findings, but by themselves they often fail to help educational designers interpret others’ experience and apply it to new problems. Likewise, educational researchers need more information to understand, question, test, and refine these principles” (p. 78).

Features of design narratives:

-describe the tool that was designed,  “the learning context, the activities and practices offered to the users, and most important, the evolution of the context over time in response to the tools” (p. 79)

-critically evaluate the assessments and outcomes and reflect on whether or not the measures used in the research adequately capture the results.

-encourage deeper attention to cultural questions in education.

-include the important agents, events, causes, and results.

-describes the history and evolution of a design over time.

-“It is broader than a design rationale, which provides only the reasons for the current state of the design; a good design narrative should describe failed design elements as well as successful ones and should relate the warrants used for making changes to the design over time. By relating the design changes over time, a design narrative can help make explicit some of the implicit knowledge the designer or research partnerships used to understand and implement the intervention” (p 79).

-clarifies which elements of the program are intentional and which were accidental. Often narratives specify which elements were responses to local constraints (therefore may not apply in other settings). Also specifies trade-offs.

-elaborates on the origins of design principles.

Importance of reflection: may uncover important regularities such as locally applicable design principles or overarching findings that help isolate relevant factors in technologies. “Reflections help achieve better alignment between theories, interventions, and assessment (Cronback, 1975), thereby increasing the validity, especially consequential validity (Gray, 1997) (p. 80).

“The minimal descriptions of a single study often reprinted in journals rarely capture the theoretical reframing of customizable components of the innovation. Design narratives help report, honestly and credibly, a refinement in understanding rather than a replacement. In  much classroom research, to really understand what happened — or what could be made to happen — requires a story” (p. 80).

Representing design knowledge in design principles:

Necessary to present principles that can be applied to localized, specific aspects of learning, as well as various broader contexts. May be accomplished via the following types of principles:

1) General Cognitive Principles – describe cognitive principles. Emerges from research on memory and skill acquisition performed in labs. Ex: Three processes that underlie performance of all learners: a) recognizing and adding ideas, b) generating connections among ideas, and c) monitoring progress toward self-generated goals.

2) Metaprinciples – synthesized/processed from multiple research programs (ex: a) make science accessible, b) make thinking visible, c) help students learn from others, d) promote autonomy and lifelong learning.

3) Pragmatic Pedagogical Principles – each is associated with a metaprinciple; captures a single research program; typically applicable across instructional settings.

4) Specific Principles – synthesize results from specific design-based research studies and may or may not generalize beyond the setting from which they emerge. Offer starting points for new design partnerships as well as individual teachers.

Using design principles to improve instruction:

“Much in the way design principles or patterns in architecture do not fully determine the design of a house but rather can serve to guide the process in the hands of a skilled architect (Alexander et al., 1977). researchers view design principles as an intermediate step between scientific findings, which must be generalized and replicable, and local experiences or examples that come up in practice” (p. 83).

“How can design knowledge systematically inform educational design and teacher practice? How can such knowledge become integrated into the day-to-day practices of designers? Other fields such as architecture and engineering have evolved rich design-based traditions and practices. More work is needed to explore productive forms of design knowledge for the field of education” (p. 85).


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: