“One way of thinking about this pair of developments (learning the properties shared and not shared by different types of numbers) is as a gradual change from initially conceptualizing numbers in terms of characteristic features (salient properties of whole numbers that are not necessarily properties of other types of numbers) to later distinguishing between defining features (properties of all real numbers, in particular their magnitudes) and features that apply to some but not all classes of numbers. This change is analogous to the shift from characteristic to defining features in semantic development described by Keil and Batterman (1984).”

“One implication of the present theory of numerical development is that acquisition of knowledge about fractions emerges as a crucial process in numerical development, rather than being of secondary importance. Learning about fractions provides the first major opportunity for children to learn that a variety of salient and invariant properties of whole numbers are not definitional for numbers in general.”

If fractions are crucial for overall mathematical understanding, and if understanding magnitudes is crucial for understanding fractions, then (1) Understanding of fraction magnitudes should be strongly related to proficiency at fractions arithmetic; and (2) Understanding of fraction magnitudes should be strongly related to overall mathematical knowledge.

“Children could memorize fraction arithmetic algorithms without understanding the magnitudes of the fractions being manipulated. Indeed, many mathematics educators have lamented that this is exactly what most students do (e.g., Cramer, Post, & del Mas, 2002; Hiebert & Wearne, 1986; Mack, 1995; Sowder et al., 1998). However, rote memorization without understanding tends to be inaccurate over short time periods and becomes even more inaccurate over longer periods (Reyna & Brainerd, 1991). Thus, fraction arithmetic procedures seemed likely to be more accurately remembered by children who understand the magnitudes of the fractions used in the computation than by children who do not understand the fraction magnitudes. One reason is that accurate fraction magnitude representations make it possible to estimate the results of fraction arithmetic operations and to reject implausible solutions. This, in turn, might lead children to reject flawed arithmetic procedures that produced implausible solutions and to continue trying to learn a procedure that generates reasonable answers. Consistent with this perspective, Hecht (Hecht, 1998; Hecht, Close, & Santisi, 2003; Hecht & Vagi, 2010) has found repeatedly that mea- sures of conceptual understanding of fractions correlate positively with fraction addition skill. Another reason for the prediction is that knowledge of whole number arithmetic correlates positively with knowledge of whole number magnitudes (Booth & Siegler, 2008; Siegler & Ramani, 2009). If knowledge of magnitudes play the same role with fractions as it does with whole numbers, as envisioned by the present theory, similar relations between knowledge of numerical magnitudes and arithmetic would be expected with both types of numbers.”

Understanding of numerical magnitudes has been shown to be related to many other aspects of mathematical development, including counting (Ramani & Siegler, 2008; Whyte & Bull, 2008) arithmetic (Booth & Siegler, 2008; Geary, Hoard, Byrd-Craven, Nugent, & Numtee, 2007; Geary, Hoard, Nugent, & Byrd-Craven, 2008; Siegler & Ramani, 2009), memory for and categorization of numbers (Laski & Siegler, 2007; Opfer & Thompson, 2008; Thompson & Siegler, 2010), and mathematics achievement test scores (Booth & Siegler, 2006; Geary et al., 2007, 2008; Halberda, Mazzocco, & Feigenson, 2008; Siegler & Booth, 2004). Manipulations that improve numerical magnitude knowledge have been shown to be causally related to increased proficiency in arithmetic (Booth & Siegler, 2008; Siegler & Ramani, 2009) and counting (Ramani & Siegler, 2008; Whyte & Bull, 2008).

The number line task has several important advantages for measuring representations of numerical magnitudes:

• It can be used with any real number (large/small, positive/negative, integer/fraction, rational/irrational).
• It transparently reflects the ratio characteristics of the number system.
• The task is practiced infrequently compared to skills such as counting and arithmetic, so estimates reflect people’s sense of the magnitudes of the numbers rather than memorization of procedures.

Two other estimation tasks: numerosity estimation (‘‘There is 1 dot in this beaker and 1000 in this one; hold down the mouse until there are N dots in this empty beaker.’’) and measurement estimation (‘‘This short line is 1 zip long; this long line is 1000 zips long; draw a line N zips long.’’) (Booth & Siegler, 2006).

Individual differences in numerical magnitude representations have been found to be positively related to other individual differences in mathematical knowledge — arithmetic competence (Booth & Siegler, 2006; Gilmore, McCarthy, & Spelke, 2007; Halberda et al., 2008; Holloway & Ansari, 2008; Mundy & Gilmore, 2009; Schneider et al., 2008) and scores on standardized math achievement tests (Booth & Siegler, 2006; Halberda et al., 2008; Laski & Siegler, 2007; Siegler & Booth, 2004).

Link btw numerical magnitude representations and arithmetic: If learning answers to arithmetic problems is a meaningful process, accurate magnitude representations might indicate the implausibility of many answers and the plausibility of a few, producing more peaked distributions of activation around the correct answer, thus facilitating correct retrieval. Thus, experiences that improve numerical magnitude representations not only increase subsequent learning of correct answers to arithmetic problems but also lead to errors being closer to the correct answer on trials where children err (Booth & Siegler, 2008; Siegler & Ramani, 2009).

“Another important way in which development of knowledge of fractions magnitudes seems likely to differ from development of whole number magnitudes is in the role of strategies. Although strategic influences have been recognized in a wide range of problem solving and reasoning contexts (Siegler, 1996), reviews of the literature on whole number magnitude representations (e.g., Ansari, 2008; Dehaene, Dehaene-Lambertz & Cohen, 1998; Fias & Fischer, 2005; Hubbard et al., 2005) typically do not even mention strategies or strategy choices (for exceptions, see Geary et al., 2007, 2008). The implicit assumption is that people invariably use a particular representation of numerical magnitudes and that the research task is to determine the characteristics of that representation. Sometimes, the assumption is explicit, as when Dehaene (1997, p. 78) described logarithmic representations of whole number magnitudes as occurring ‘‘like a reflex’’ that cannot be inhibited. Consistent with the assumption that magnitude representation is an automatic, non-strategic process, number line estimation with whole numbers is no less accurate under time pressure than without time pressure (Siegler & Opfer, 2003).”

Study design:

• 6th and 8th graders (11- and 12-year-olds and 13- and 14-year- olds)
• Assessments of fraction magnitude knowledge: 0–1/ 0–5 number line estimation,  0–1 magnitude comparison, fraction arithmetic problems.
• Verbal reports of strategy use were obtained immediately after each number line estimation and arithmetic problem, to allow examination of relations between strategy use and speed and accuracy on each task.
• Obtained students’ mathematics achievement test scores, to examine their relation to the three measures of fraction magnitude knowledge and to fraction arithmetic proficiency.

We made six predictions:

1. Even after years of fractions instruction, fraction magnitude representations, whether measured by magnitude comparison or number line estimation, will be quite inaccurate in both 6th and 8th grade.
2. Despite information about fractions magnitudes being taught explicitly in 3rd and 4th grade (NCTM, 2007), this understanding should still be increasing between 6th and 8th grade, due to students learning about fraction magnitudes from solving problems involving proportions and percentages in those grades.
3. No logarithmic to linear transition should be present with fractions, because frequency of encountering fractions (and therefore knowledge of specific fractions) is correlated minimally if at all with fraction magnitudes, at least in the 0–1 range.
4. Students should use a variety of strategies to solve fraction number line estimation and arithmetic problems, and the quality of these strategies should be related to students’ accuracy and speed in solving problems, as with whole number arithmetic.
5. Individual differences in knowledge of fractions magnitudes should correlate highly with success at solving fraction arithmetic problems.
6. Individual differences in knowledge of fractions magnitudes should correlate highly with individual differences in overall mathematics achievement test scores.

Number line estimation task:

• participants were sequentially presented 10 number lines on a computer screen.
• 0-1 number line items: 1/19, 1/7, 1/4, 3/8, 1/2, 4/7, 2/3, 7/9, 5/6, and 12/13.
• 0-5 number line items: 1/19, 4/7, 7/5, 13/9, 8/3, 11/4, 10/3, 7/2, 17/4, and 9/2.
• One fraction was drawn from each tenth of the number line, and presentation order was random.
• Participants responded on each trial by moving the cursor to the desired position on the number line and clicking the mouse.
• Accuracy of number line estimation was indexed by percent absolute error (PAE). PAE = (|Child’s Answer – Correct Answer|) / Numerical Range. For example, if a child was asked to locate 5/2 on a 0–5 number line, and marked the location corresponding to 3/2, the PAE would be 20% ((|1.5 – 2.5|) / 5).

Magnitude comparison task:

• Participants were asked to compare to 3/5 a fraction shown on the computer screen: 3/8, 5/8, 2/9, 4/5, 4/7, 5/9, 8/9, or 2/3.
• If the fraction was smaller than 3/5, the participant was to press the ‘‘a’’ key; if the fraction was larger than 3/5, the participant was to press the ‘‘l’’ key.”

Fraction arithmetic assessment:

• Participants were presented 8 problems, 2 for each of the 4 arithmetic operations: 3/5 + 1/2, 3/5 + 2/5, 3/5 – 1/2, 3/5 – 2/5, 3/5 * 1/2, 3/5 * 2/5, 3/5 / 1/2, and 3/5 / 2/5.
• One of the two problems for each arithmetic operation had operands with equal denominators. Problems appeared one at a time on the computer screen, and participants typed their answers.

Classification of strategies:

• Strategies were classified on the basis of overt behavior and immediately retrospective self-reports.
• When overt behavior clearly indicated the child’s approach, that behavior was the basis of the strategy assessment; otherwise, the child’s self-report was used.
• The two main types of number line estimation strategies were numerical transformation strategies, in which participants transformed the presented fraction to a more convenient number, and number line segmentation strategies, in which participants generated subjective landmarks on the number line.
• Both types of strategies could be, and often were, used on a single trial; children could transform the fraction to a more convenient numerical form and segment the number line in a way that helped them locate the fraction.
• In coding numerical transformation strategies, we distinguished only between using a numerical transformation and not using one.
• The most common numerical transformations were rounding the fraction (‘‘5/9 is a bit more than ½’’), simplifying it (‘‘9/5 = 1 and 4/5, which is a little less than 2’’), or translating it into a different form (‘‘12/13 is about 90%). The reason for not distinguishing among these numerical transformations was that they overlapped and could not be reliably distinguished.
• The main number line segmentation strategies were division into halves; division into fifths or whole number units (e.g., placing marks on a 0–5 number line at the estimated positions of 1, 2, 3, and 4); division into units corresponding to the denominator (e. g., dividing a 0–1 number line into sevenths to locate 4/7); flawed approaches (e.g., on a 0–1 number line, reporting, ‘‘I put 3/7 near 0 because 3 rounds down’’); and none unknown (e.g., saying, ‘‘I don’t know’’).
• “The lengthy solution times on none unknown trials, roughly 10 s, suggested that children might have used some strategy, but neither self- reports nor overt behavior indicated what it was.”
• The mean of the median solution times across number lines and age groups was 9.5 s. On 0–1 number lines, the means of the medians of individual children’s solution times were 8.4 s for 6th graders and 7.3 s for 8th graders; on 0–5 number lines, they were 11.9 s for 6th graders and 10.3 s for 8th graders.
• The response times suggest that fractions number line estimation is far from automatic; rather, it appears to be a controlled, strategic process.”

“The present findings indicate that understanding of fraction magnitudes and fractions arithmetic are closely related. If learning fraction arithmetic algorithms reflected rote memorization, as has often been claimed (e.g., Cramer & Bezuk, 1991; Hiebert, 1986; Kerslake, 1986), there would be no reason to expect such a relation. However, the strong correlations between fractions arithmetic and all three measures of magnitude knowledge in both 6th and 8th grades indicate that conceptual and procedural knowledge of fractions are intertwined. (See Hecht (1998), Hecht, Close, and Santisi (2003), Hecht and Vagi (2010), and Schneider and Stern (2010) for similar findings.) One plausible interpretation of these results is that magnitude knowledge makes it easier to learn and remember fraction arithmetic algorithms. This might occur through children with good magnitude knowledge of fractions rejecting procedures that produce unreasonable answers, such as operating independently on numerator and denominator often does, and searching longer for procedures that produce reasonable answers. For example, children might reject the procedure that produces arithmetic errors of the form 3/5 – 1/2 = 2/3 if they recognized that subtracting a positive number can- not lead to an answer larger than the number being subtracted from. This could lead them to try other procedures and test whether they yielded plausible answers.”

“Another implication for understanding fraction arithmetic is methodological: The same strategy assessment techniques that have proved useful with whole number arithmetic also are useful for investigating fractions arithmetic. As with whole number arithmetic, individual children used a variety of fractions arithmetic strategies. Even on a single fractions arithmetic operation, strategy use varied with problem characteristics, notably with the equality or inequality of denominators. The quality of fraction arithmetic strategy use was related to both knowledge of numerical magnitudes and to overall mathematics achievement test scores. In whole number arithmetic, these strategy assessment techniques have provided a base for computer simulation models of arithmetic learning that accounted for numerous findings regarding variations in accuracy, solution times, and strategy use across problems; discovery of useful new strategies; individual differences in arithmetic proficiency; and changes in speed, accuracy, and strategy use with problem-solving experience (Shrager & Siegler, 1998; Siegler & Shipley, 1995). The prominence of strategy use in fractions arithmetic suggests that similar models might be applicable to that area, and that it might be possible to formulate a common model of development of whole number and fractions arithmetic.”

“A third implication for understanding fractions arithmetic is that fraction arithmetic errors often reflect confusion about the right strategy, together with a lack of constraints on the magnitudes of answers, rather than a consistent whole number bias or other systematic misunderstanding.”

“[whole number bias] has been said to lead to children treating numerators and denominators as independent whole numbers and operating on them independently, for example by subtracting numerator from numerator and denominator from denominator (Carpenter et al., 1981; Gelman, 1991; Kilpatrick, Swafford, & Findell, 2001). However, the present findings revealed greater variability in fraction procedures than implied by this attribution of errors to a systematic misconception.”

“Roughly half of arithmetic errors stemmed from applying whole number algorithms independently to numerators and denominators, but a similar percentage reflected using parts of algorithms that would have been correct for a different fraction arithmetic operation or trying other erroneous procedures. The inconsistency of strategies even within a single arithmetic operation was striking; 40% of children correctly solved one of the pair of problems for a single arithmetic operation and erred on the other. This variability suggests that the whole number bias is only part of the problem in understanding fractions arithmetic. Rather than reflecting a systematic misconception, fractions arithmetic knowledge seems piecemeal; understanding of whole numbers is one source of ideas about how to solve fractions arithmetic problems, but other types of numerical knowledge are also incorporated. It also is unclear whether children who use erroneous fractions arithmetic procedures believe that those procedures are correct. They might well be skeptical about their correctness but have learned that saying ‘‘I don’t know’’ is not an acceptable alternative in school mathematics. This issue could be addressed through studies that examine children’s confidence in their fraction arithmetic answers.”

“In the United States, instruction in fractions emphasizes part-whole interpretations far more than other interpretations of fractions (Ni & Zhou, 2005; Sophian, 2007; Thompson & Saldanha, 2003). For example, students are taught to interpret 1/5 as one of five slices of pizza, but less often to think of 1/5 as one fifth of the distance from zero to one on a number line (Moseley, Okamoto, & Ishida, 2007). This is quite different than the approach to teaching fractions in Japan, China, and other countries where students understand fractions better. Indeed, many teachers in the US can only explain fractions in terms of the part-whole interpretation, unlike teachers in China and Japan who also emphasize num- ber line and other interpretations (Ma, 1999; Moseley et al., 2007). Part-whole interpretations have the advantages of concreteness and accessibility. When numerators and denominators are small and positive and the numerator is less than the denominator, it is easy to think about N parts of a whole that includes M parts. For example, children have little difficulty understanding that if a pizza is cut into four pieces, then each piece is 1/4 of the pizza (Mix, Levine, & Huttenlocher, 1999). However, the part-whole interpretation of fractions also has some serious limitations. Negative fractions cannot be represented in this way, it is very difficult to imagine fractions with large numerators and denominators (e.g., 734/878), and improper fractions can be confusing within the part-whole interpretation, as illustrated by one learner’s reaction to being presented 4/3, ‘‘You cannot have four parts of an object that is divided into three parts’’ (Mack, 1993). Moreover, there is nothing in the operation of dividing an object into N parts that says that the size of the parts must be equal; many students fail to understand that the parts must be equal for fractions to have any consistent meaning (Sophian, 2007).”

“emphasizing that fractions are measurements of quantity might improve learning about fractions. Indeed, a common feature of instructional studies that have yielded especially promising results in teaching rational numbers, such as work by Robbie Case and his associates, is that they emphasize that fractions are measures of quantity (e.g., Cramer et al., 2002; Fujimura, 2001; Keijzer & Terwel, 2003; Moss & Case, 1999; Rittle-Johnson & Koedinger, 2002, 2009). The present integrated theory of numerical development helps to explain the prevalence of this common feature of successful instruction: If magnitudes are central to understanding fractions as well as whole numbers, then instruction that emphasizes magnitude understanding is more likely to succeed than instruction that does not emphasize magnitude understanding.”

The authors tested the validity of eye-movement data as a means for investigating children’s use of the number line when solving number line estimation tasks, and as a measure of children’s developing number sense. In a cross-sectional design with children from Grades 1 to 3, they assessed (a) the accuracy of manual solutions of number line estimation tasks, (b) the accuracy of the positions fixated by gaze while solving a second set of number line estimation tasks, and (c) the accuracy of responses to mental addition tasks.

Research questions:

• Is grade-related increase in children’s estimation competence only reflected by manual answers or also by eye-tracking data?
• Are individual differences in the accuracies of the estimated positions and in the accuracies of their eye movements during the estimation process correlated?
• Are the accuracy of the estimated positions and the accuracy of the positions fixated by gaze during solution production correlated with children’s addition competence?
• To what extent does the criterion validity of the eye-movement measure increase with age?
• Does eye-movement data indicate that older children increasingly use the midpoint of the number line as an orientation point?”

Number sense: “ability to quickly understand, approximate, and manipulate numerical quantities” (Dehaene, 2001, p. 16).

“The “mental number line” is regarded as the core neurocognitive system underlying number sense (Fias, Lammertyn, Reynvoet, Dupont, & Orban, 2003; Hubbard, Piazza, Pinel, & Dehaene, 2005; Pinel, Dehaene, Riviere, & LeBihan, 2001), which in turn underlies a variety of behavioral competencies, like estimating, computing, and efficiently using notational systems to solve mathematical problems (Berch, 2005; Jordan, Kaplan, Locuniak, & Ramineni, 2007).” Represents the magnitudes of numbers in an analogous form.

“Arabic numerals, like 5 and 7, do not allow for any direct inference as to which of them is the one with the higher value. The same is true for number words, like 5 and 7. In contrast, when numerical magnitudes are represented by positions on a number line, one immediately grasps which of them is the higher value number. Therefore, Case and Okamoto (1996) suggested that children use the mental number line “to build models of the conceptual systems that their culture has evolved for measuring such dimensions as time, space. . . They use it to make sense of any direct instruction that they may receive regarding the particular systems that their culture has evolved for arranging numbers into groups and for conducting numerical computations” (pp. 8–9).”

Measuring number sense:

• See Berch, 2005; Jordan, Kaplan, Oláh, & Locuniak, 2006.
• Siegler and Opfer (2003) “asked children to estimate the positions of given numbers on an external number line where only the starting and the end points were labeled. They interpreted the patterns of estimates as indicative of children’s representation of magnitudes on their internal number line. These answer patterns relate not only to children’s competence in performing other estimation tasks, such as numerosity estimation and computational estimation, but even to children’s addition competence and general math achievement (Booth & Siegler, 2006, in press; Siegler & Booth, 2004).”
• The number line estimation task is a “practical and powerful tool applicable across a wide range of age groups. It has been hypothesized to reflect children’s mental representation of numbers more directly than alternative assessments of number sense do.”

“Despite its potential benefits, the number line estimation task has a major drawback. Although it is easy to measure such products of children’s estimation processes as accuracy, solution times and estimate patterns, it is hard to investigate the processes themselves that children employ to construct their solutions.”

Solution strategies:

• [Grade 1] Counting up strategy, counting down strategy, in whole units or decades from the beginning of the number line (Petitto, 1990; Newman & Berger, 1984).
• [Older children] Start counting up from the midpoint.

“Eye-movement data can be collected with high temporal and spatial resolution (e.g., several-hundred measures per second with a spatial precision of 0.01◦ ) notwithstanding the fact that the reliability of the resulting data can suffer from technically caused measurement error and task-irrelevant fixations. Compared to accuracy and speed measures, eye-tracking data potentially provide more direct evidence of the process of problem solving. Moreover, these data are more objective than self-reports or behavioral observations of strategy use.”

“Children’s increasing ability to focus their attention on the task-relevant features of a problem situation are likely to reduce the number of task-irrelevant eye movements and, thus, to increase the validity of our eye-movement measure over the three grade levels.”

“By means of eye-tracking data, Rehder and Hoffman (2005) demonstrated that in adults, increasing competence in object categorization goes along with an increasing tendency to focus attention on task-relevant characteristics of a problem situation. To guide attention to task-relevant characteristics of a problem situation and to ignore task-irrelevant features is an important part of mathematical competence (The Cognition and Technology Group at Vanderbilt, 1992). Therefore, eye movements might reflect individual differences in mathematical competence.” [could also guide the design of scaffolding in a math game...]

“Two studies (Green, Lemaire, & Dufau, 2007; Verschaffel, De Corte, Gielen, & Struyf, 1994) have demonstrated that eye movements validly reflect various strategies chosen by elementary school children and adults to solve mental addition problems.”

“the external number line, as a diagram, is an analogous and more holistic diagrammatic knowledge representation (Larkin & Simon, 1987). The validity of eye movements as an indicator of children’s competence concerning this type of external knowledge representation has not been investigated.”

Methodology:

• Children had to solve 30 trials of the number line estimation task.
• Following Rittle-Johnson, Siegler, and Alibali’s (2001) example, we coded an answer as correct if it was within an error margin of ±10% of the number line around the actual position of the stimulus on the line, and as incorrect in all other cases.
• The stimuli were selected by a pseudo-random algorithm from the natural numbers between 0 and 100.
• For the eye-tracking version of the number line estimation task, children were instructed to actively search for and focus their gaze on the correct position for each number. After 4000 ms, a marker appeared. The children were asked to decide as fast as possible whether the marker position was correct or not and to give their answer by clicking a respective button. Button clicks and reaction times were recorded automatically.
• No feedback was given to the children during the experimental trials. The order of the two number line estimation tasks was counter-balanced at each grade level.
• In addition to individual fixation accuracies, we derived the frequency distribution of all fixations over the number line for each grade level by computing the position of each fixation on the number line and by rounding to the nearest whole value. We counted per individual and task how often each of the 101 positions of whole numbers on the line had been fixated. The resulting value for each position was averaged per grade level. To enhance readability, these values were multiplied by 10,000.
• Fixation accuracy measure is based on the assumption that not only the last fixation but all fixations during a trial indicate children’s knowledge.

Some results

• Addition accuracy increases most strongly from 15 to 91% and has the highest proportion of explained variance, as indicated by 2 values.
• The effect sizes also show that grade-related increases in knowledge are more clearly reflected by estimation accuracy than by fixation accuracy. Of the three variables, fixation accuracy shows the least change and the smallest, albeit still high, proportion of explained variance.
• Significant change in estimation accuracy and fixation accuracy occurs only between Grades 1 and 2, but not between Grades 2 and 3. Differences in addition accuracy are highly significant between each of the grade levels.
• Exploratory comparisons of (a) the last fixation in each trial with (b) all previous fixations in that trial confirm our expectations. The average percentage of correct fixations in the sample is 42.1% for only the last fixations and 34.0% for all fixations. Both accuracies lie well above chance level (i.e. 20%) and, thus, indicate knowledge.
• The accuracies for both types of fixations correlate across individuals with r = .53, p = .017, for first graders, r = .73, p < .001, for second graders, and r = .70, p < .001, for third graders, suggesting that the two groups of fixations measure similar aspects of children’s knowledge.
• Guided by our second research question, we computed the correlations between estimation accu- racy and fixation accuracy. These are r = .28, p = .212, for first graders, r = .66, p = .002, for second graders, and r = .63, p = .001, for third graders.
•  Children in all three grades fixate positions near the starting point of the line, near the end of the line, and near the number 50 in the middle of the line more frequently than any other position. respectively.
• Children’s fixations (a) validly reflect grade-related competence increases, (b) are closely related, in Grades 2 and 3, to manual solutions of estimation tasks, (c) are related, in Grade 2, to addition com- petence, (d) are very systematically distributed over the number line, and (e) replicate Petitto’s (1990) findings with respect to the use of the midpoint strategy and the counting-up strategy by students in Grades 1–3.

“The fact that grade level explains more than one third of the interindividual variance of children’s fixation accuracies establishes that eye movements reflect children’s increasing knowledge about natural numbers, their interrelations, and ways of their spatial representation. This understanding of natural numbers lies at the very core of children’s number sense and is a prerequisite for their future acquisition of more advanced mathematical concepts (Dehaene, 1997).”

“As explained by Verschaffel et al. (1994) and Green et al. (2007), identifying trial-by-trial strategy use by means of eye-tracking data can considerably improve research on the development of mathematical strategies. For example, parallel use of a verbal and a nonverbal measure of strategy use allows researchers to investigate the interaction of explicit and implicit knowledge during strategy development (Siegler & Stern, 1998). However, distinguishing strategy use on a trial-by-trial basis by means of eye-tracking may be as difficult as it is desirable. In most eye-tracking studies the data are aggregated over trials – and often over persons – because ‘eye tracking data is never perfect. The system may lose track of the pupil or the corneal reflection, or the observation may be simply incorrect (e.g., beyond the screen limits even though the subject is clearly looking at the screen)’ (Aaltonen, Hyrskykari, & Räihä, 1998, p. 135). “Eye-movement data are inherently noisy” (Hornof & Halverson, 2002, p. 593), due to task-irrelevant fixations (e.g., when the individual is distracted by sounds from the surroundings). Individuals may make data-distorting head-movements (children more so than adults) which might even require a re-calibration of the scanning system. Finally, individuals may use peripheral vision to detect information from regions on the screen that they do not fixate directly. All these influences increase measurement error, thus, making it harder to find effects on the trial level than on the level of aggregated data (Rayner, 1998; Verschaffel et al., 1994).”

“Eye movements allow for a direct investigation of how children orient themselves in problem situations and how they direct their attention to specific features. Although previous theories tended to conceptualize mathematical problem solving as an abstract symbol- manipulation process, more recent approaches emphasize the interaction with problem situations as a highly important part of mathematical competencies (Collins, Greeno, & Resnick, 2001). For example, advocates of the situated-cognition view have argued that the ability to pick up action-relevant information from the environment is the most important foundation of competent problem solving and knowledge transfer (Greeno, 1994; Greeno, Moore, & Smith, 1993). Eye movements offer a means of investigating the dynamic and selective search for information in problem situations. Such search invokes higher-level cognitive processes, including what one knows about, and expects of, a situation (Rehder & Hoffman, 2005). More advanced analysis techniques for eye-tracking data, such as fixation density maps (Ouerhani, von Wartburg, Hügli, & Müri, 2004), may become useful for deepening our understanding of these phenomena.”

References:

Berch, D. B. (2005). Making sense of number sense: Implications for children with mathematical disabilities. Journal of Learning Disabilities, 38(4), 333–339.

Booth, J. L., & Siegler, R. S. (2006). Developmental and individual differences in pure numerical estimation. Developmental Psychology, 41, 189–201.

Booth, J. L., & Siegler, R. S. (2008). Numerical magnitude estimations influence arithmetic learning. Child Development, 79(4), 1016–1031.

Butterworth, B. (2005). Developmental dyscalculia. In J. I. D. Campbell (Ed.), Handbook of mathematical cognition (pp. 455–467). New York: Psychology Press. Campbell, J. I. D. (Ed.). (2005). Handbook of mathematical cognition. New York: Psychology Press.

Case, R., & Okamoto, Y. (1996). The role of central conceptual structures in the development of children’s thought. Monographs of the Society for Research in Child Development, 61(1 2), 1–295.

Collins, A., Greeno, J. G., & Resnick, L. B. (2001). Educational learning theory. In N. J. Smelser & P. B. Baltes (Eds.), International encyclopedia of the social & behavioral sciences (pp. 4276–4279). Oxford: Elsevier Science.

Dehaene, S. (1997). The number sense: How the mind creates mathematics. New York: Oxford University Press.

Dehaene, S. (2001). Précis of “The number sense”. Mind and Language, 16, 16–32.

Dehaene, S., Bossini, S., & Giraux, P. (1993). The mental representation of parity and number magnitude. Journal of Experimental Psychology: General, 122(3), 371–396.

Fias, W., & Fischer, M. H. (2005). Spatial representations of numbers. In J. I. D. Campbell (Ed.), Handbook of mathematical cognition (pp. 43–54). New York: Psychology Press.

Fias, W., Lammertyn, J., Reynvoet, B., Dupont, P., & Orban, G. A. (2003). Parietal representation of symbolic and non-symbolic magnitude. Journal of Cognitive Neuroscience, 15(1), 47–56.

Geary, D. C., & Hoard, M. K. (2005). Learning disabilities in arithmetic and mathematics: Theoretical and empirical perspectives. In J. I. D. Campbell (Ed.), Handbook of mathematical cognition (pp. 253–267). New York: Psychology Press.

Green, H. J., Lemaire, P., & Dufau, S. (2007). Eye movement correlates of younger and older adults’ strategies for complex addition. Acta Psychologica, 125, 257–278.

Greeno, J. G. (1994). Gibson’s affordances. Psychological Review, 101(2), 336–342.

Greeno, J. G., Moore, J. L., & Smith, D. R. (1993). Transfer of situated learning. In D. K. Detterman & R. J. Sternberg (Eds.), Transfer on trial: Intelligence, cognition, and instruction. Norwood, NJ: Ablex.

Hornof, A. J., & Halverson, T. (2002). Cleaning up systematic error in eye tracking data by using required fixation locations. Behavioral Research Methods, Instruments, and Computers, 34(4), 592–604.

Hubbard, E. M., Piazza, M., Pinel, P., & Dehaene, S. (2005). Interactions between number and space in parietal cortex. Nature Reviews Neuroscience, 6(6), 435–448.

Jordan, N. C., Kaplan, D., Locuniak, M. N., & Ramineni, C. (2007). Predicting first-grade math achievement from developmental number sense trajectories. Learning Disabilities Research & Practice, 22(1), 36–46.

Jordan, N. C., Kaplan, D., Oláh, L. N., & Locuniak, M. N. (2006). Number sense growth in kindergarten: A longitudinal investigation of children at risk for mathematical difficulties. Child Development, 77, 153–175.

Larkin, J. H., & Simon, H. A. (1987). Why a diagram is (sometimes) worth ten thousand words. Cognitive Science, 11, 65–99.

Newman, R. S., & Berger, C. F. (1984). Children’s numerical estimation: Flexibility in the use of counting. Journal of Educational Psychology, 76(1), 55–64.

Ouerhani, N., von Wartburg, R., Hügli, H., & Müri, R. M. (2004). Empirical validation of the salience-based model of visual attention. Electronic Letters on Computer Vision and Image Analysis, 3, 13–24.

Petitto, A. L. (1990). Development of numberline and measurement concepts. Cognition and Instruction, 7(1), 55–78.

Pierce, C. A., Block, R. A., & Aguinis, H. (2004). Cautionary note on reporting eta-squared values from multifactor anova designs. Educational and Psychological Measurement, 64(6), 916–924.

Pinel, P., Dehaene, S., Riviere, D., & LeBihan, D. (2001). Modulation if parietal activation by semantic distance in a number comparison task. NeuroImage, 14, 1013–1026.

Rayner, K. (1998). Eye movements in reading and information processing: 20 years of research. Psychological Bulletin, 124(3), 372–422.

Rehder, B., & Hoffman, A. B. (2005). Eyetracking and selective attention in category learning. Cognitive Psychology, 51, 1–41.

Rittle-Johnson, B., Siegler, R. S., & Alibali, M. W. (2001). Developing conceptual understanding and procedural skill in mathematics: An iterative process. Journal of Educational Psychology, 93(2), 346–362.

Siegler, R. S., & Booth, J. L. (2004). Development of numerical estimation in young children. Child Development, 75(2), 428–444.

Siegler, R. S., & Opfer, J. E. (2003). The development of numerical estimation: Evidence for multiple representations. Psychological Science, 14, 237–243.

Siegler, R. S., & Stern, E. (1998). Conscious and unconscious strategy discoveries: A microgenetic analysis. Journal of Experimental Psychology: General, 127(4), 377–397.

The Cognition and Technology Group at Vanderbilt. (1992). The Jasper series as an example of anchored instruction: Theory, program description and assessment data. Educational Psychologist, 27, 291–315.

Verschaffel, L., De Corte, E., Gielen, I., & Struyf, E. (1994). Clever rearrangement strategies in children’s mental arithmetic: A confrontation of eye-movement data and verbal protocols. In J. E. H. V. Luit (Ed.), Research on learning and instruction of mathematics in kindergarten and primary school (pp. 153–180). Doetinchem, The Netherlands: Graviant.

Verschaffel, L., De Corte, E., & Pauwels, A. (1992). Solving compare problems: An eye movement test of Lewis and Mayer’s consistency hypothesis. Journal of Educational Psychology, 84(1), 85–94.

Whiteside, J. A. (1974). Eye movements of children, adults, and elderly persons during inspection of dot patterns. Journal of Experimental Child Psychology, 18(2), 313–332.”

The authors used eye tracking data to investigate the development of numerical magnitude understanding in primary school children (grades 1-3). Children were presented with two number line estimation tasks; one was restricted to behavioral measures, whereas the other included the recording of eye movement data. Gaze data indicates a quantitative increase as well as a qualitative change in children’s implicit knowledge about numerical magnitudes that precedes the overt (behavioural) demonstration of explicit numerical knowledge. Thus, eye movements may reveal more about the presence of implicit precursors of later explicit knowledge in the numerical domain than more traditional approaches, suggesting further exploration of eye tracking as a potential early assessment tool of individual achievement levels in numerical processing.

“The idea that cognitive growth can be characterized as a transition from implicit forms of knowledge to more explicit knowledge provides developmental researchers with a fascinating yet challenging starting point for translating the Vygotskyan ideas into the domain of knowledge acquisition.”

“Vygotsky’s concept of what he called a child’s zone of proximal development emphasizes the need to also consider children’s learning potential instead of merely focussing on what a child has explicitly mastered at any given point in his or her development. An important question is therefore how to assess children’s learning potential, that is, how to determine their level of knowledge that is not yet explicitly expressed (Brown & Ferrara, 1985).”

Dienes and Perner (1999): fully explicit knowledge – when the person is conscious of that knowledge and is in a position to entertain second-order thoughts about it. Eye movements can be used as sensitive indicators of implicit knowledge that precede more explicit forms of knowledge.

Susan Goldin-Meadow et al. conducted a number of influential developmental studies that aim at identifying children’s implicit or ‘budding knowledge’ (Vygotsky, 1978). “Focussing on the gestures that accompany children’s problem solving processes, Goldin-Meadow et al. were able to demonstrate subtle mismatches between children’s verbal and nonverbal behaviours (Alibali & Goldin- Meadow, 1993; Goldin-Meadow, Alibali, & Church, 1993). According to the authors, gesture–speech mismatches can be indicative of a child’s specific readiness-to-learn by revealing forms of knowledge that are there at an implicit level, but not yet explicitly available to the child (Goldin-Meadow, 2000; Goldin- Meadow & Sandhofer, 1999).”

Siegler et al.: primary schoolers’ representational patterns change from a logarithmic to a more adequate linear model of numerical magnitude between the first and the second grade. This transition is considered a crucial step that enables children to improve their achievement in a wide range of mathematical task domains.

“The question is whether the behaviourally established quantitative and qualitative changes can also be demonstrated using eye movement parameters, and, more importantly, whether eye movement measures that we assume to be able to tap into implicit knowledge add to what we already know about the development of numerical magnitude representation in children.”

Some results:

• “While in first graders, the eyes seem to roam a wide range of numerical positions on the number line, in third graders, approximately two-thirds of all fixations fall within the immediate vicinity of the respective correct positions. The distributional pattern of second graders’ fixations lies somewhere between the two extremes.”
• “In-depth analyses of the fits of number line fixations to the logarithmic and linear models determined in the behavioural setting revealed a somewhat different picture for the eye movement data compared with the behavioural data. Although first graders’ estimates were fitted substantially better by the logarithmic model than by the linear model in the behavioural task, this does not hold true for their eye movements. Comparable to the second and third graders, for the eye-tracking data the linear model provides a better fit in first graders as well.”
• “However, tests of the residuals between the models and the gaze data revealed that in first graders the differences in the fit of the linear compared to the logarithmic functions are not significant. In second and third graders the linear functions provide the significantly better models.”
• “Eye movement data demonstrate that even when, on first sight, children still appear to be stuck on a more immature level their implicit knowledge might already have undergone some subtle yet detectable changes. The eye movement data demonstrate that even in cases where no evidence of appropriate insight into a certain numerical magnitude can be found in children’s overt behavioural responses, their eye movements might still show manifestations of knowledge at work on a more implicit level. Thus, for instance, our analyses of children’s eye movements in error trials indicate that third graders seem to understand more about certain numerical magnitudes than first or second graders even when they do not actually appear to be more knowledgeable than their younger peers on the level of overt behaviour. The eye movement data reveal that compared with children from lower grade levels, older children shift their gaze significantly more often and also for a significantly longer time to the respective correct positions on the number line in trials where their explicit responses are consistently false.”

“While first graders’ overt behaviour suggests that their representation of numerical magnitude still follows an immature logarithmic model, a change towards more mature representational patterns might already be on the way, as revealed by measures that are capable to tap this emerging knowledge. There might, thus, be a transitional phase where both representational patterns exist in parallel. Such an idea is well in line with Siegler’s (1996) Overlapping the Wave model of change that proposes that during development ‘multiple ways of thinking coexist for prolonged periods’ (p. 89).”

References to read:

Clements, W., & Perner, J. (1994). Implicit understanding of belief. Cognitive Development, 9, 377–397.

Clements, W. A., Rustin, Ch. L., & McCallum, S. (2000). Promoting the transition from implicit to explicit understanding: A training study of false belief. Developmental Science, 3, 81–92.

Alibali, M., Flevares, L., & Goldin-Meadow, S. (1997). Assessing knowledge conveyed in gesture: Do teachers have the upper hand? Journal of Educational Psychology, 89, 183–193.

Alibali, M. W., Bassok, M., Solomon, K. O., Syc, S. E., & Goldin-Meadow, S. (1999). Illuminating mental representations through speech and gesture. Psychological Science, 10, 327–333. Alibali, M. W., & Goldin-Meadow, S. (1993). Gesture–speech mismatch and mechanisms of learning: What the hands reveal about a child’s state of mind. Cognitive Psychology, 25, 468–523.

Dienes, Z., & Perner, J. (1999). A theory of implicit and explicit knowledge. Behavioral and Brain Sciences, 22, 735–755. Garber, P., Alibali, M. W., & Goldin-Meadow, S. (1998). Knowledge conveyed in gesture is not tied to the hands. Child Development, 69, 75–84.

Goldin-Meadow, S. (2000). Beyond words: The importance of gesture to researchers and learners. Child Development, 71, 231–239. Goldin-Meadow, S., Alibali, M. W., & Church, R. B. (1993). Transitions in concept acquisition: Using the hand to read the mind. Psychological Review, 100, 279–297.

Goldin-Meadow, S., & Sandhofer, C. M. (1999). Gesture conveys substantive information about a child’s thoughts to ordinary listeners. Developmental Science, 2, 67–74.

Perner, J., & Dienes, Z. (1999). Deconstructing RTK: How to explicate a theory of implicit knowledge. Behavioural and Brain Sciences, 22, 790–808.

Karmiloff-Smith, A. (1992). Beyond modularity: A developmental perspective on cognitive science. Cambridge, MA: MIT Press.

[Karmiloff-Smith’s (1992) notion of representational re-description as a process by which ‘implicit information in the mind subsequently becomes explicit knowledge to the mind’ (p. 18).]

Garnham, W. A., & Perner, J. (2001). Actions really do speak louder than words—but only implicitly: Young children’s understanding of false belief in action. British Journal of Developmental Psychology, 19, 413–432.

Garnham, W. A., & Ruffman, T. (2001). Doesn’t see, doesn’t know: Is anticipatory looking really related to understanding of belief? Developmental Science, 4, 94–100.

Laski, E. V., & Siegler, R. S. (2007). Is 27 a big number? Correlational and causal connections among numerical categorization, number line estimation, and numerical magnitude comparison. Child Development, 76, 1723–1743.

Perry, M., Church, R. B., & Goldin-Meadow, S. (1988). Transitional knowledge in the acquisition of concepts. Cognitive Development, 3, 359–400.

Ruffman, T., Garnham, W., Import, A., & Connolly, D. (2001). Does eye gaze indicate knowledge of false belief: Charting transitions in knowledge. Journal of Experimental Child Psychology, 80, 201–224.

Play is situated within the game but play doesn’t come from the game itself but from the way the players interact w the game in order to play it. Meaningful play emerges from the interaction btw players and the system of the game, as well as from the context in which the game is played.

Playing means making choices and taking actions. The designer should try and make the choices as meaningful as possible.

Two notions of meaningful play:
1) descriptive – meaningful play resides in relationship btw action and outcome;
2) evaluative – when those relationships are both discernible and integrated into the larger context of the game.

Players must be able to discern whether or not a choice lead them closer or farther from winning. Must know the consequences of one’s actions — this requires the player knowing the meaning of each action. Actions have consequences that are woven throughout the game (a great example of this is Go).

Chapter 7
Lusory attitude – the peculiar state of mind of game players

“A game is a system in which players engage in an artificial conflict, defined by rules, that results in a quantifiable outcome” (p. 80).

Chapter 8
What can digital technology do? Traits of most digital games:
Immediate but narrow interactivity
Information manipulation. Karen Sideman points out that with digital games, part of the play is discovering the rules (contrast w board games).
Automated complex systems. Innerworkings hidden (what Dunnigan calls the ‘black box syndrome’ of computer games)
Networked communication

Chapter 9
The Magic Circle – defined as a special place in time and space created by a game. Separate from, but still references the real world.

Lusory attitude – allows players to “adopt rules which require one to employ worse rather than better means for reaching an end” (cited from Bernard Suits).

“From somewhere in the gap between action and outcome, in the friction between frustrated desire and the seductive goal of a game, bubbles up the unique enjoyment of game play” (p. 98).

One should design structures that can create and support the magic circle, as well as qualities that affect the lusory attitude and the possibility of meaningful play.

At the heart of games are rules, the space of games framed as formal systems.

Chapter 11
“Rules are the logical underbelly beneath the experiential surface of any games” (p. 120). The formal system of a game, the game considered as a set of rules, is not the experience of the game. When looking at games from the point of view of rules, we are less concerned w player experience than w the rules constituting the experience. Rules are the ‘formal identity’ of a game” (p. 121).

Characteristics of game rules
limit player action
explicit and unambiguous
shared by all players
fixed (do not change)
binding
repeatable & portable

Chapter 12
Three levels of rules:
Operational – rules of play, essentially
Constituative – underlying formal structures; logical & mathematical in nature
Implicit – unwritten rules. Concern etiquette, good sportsmanship, etc.

The identity of a game emerges from the interaction btw operational and constituative sets of rules.

All three levels work in concert to generate the formal meaning of a game.

Elegant rules allow the player to focus on the experience of play rather than the logic of the rules.

Chapter 13
In digital games, the game rules regulate game logic, player action and outcome, scoring system, structural arrangement of the game space, etc. — the formal structure of the game.

Operational rules for digital games include the use of input devices, certain external or representational event that impact player interactivity and formal game events.

Learning games, if designed well, may serve as effective learning tools supporting knowledge construction (e.g. Kiili, 2007; Lainema & Makkonen, 2003; Gee, 2003; Amory, 2001; Prensky, 2001). In this article, the authors empirically examine how elements in the game specifically support the knowledge construction processes in players. They use a math game in which a player teaches a ‘teachable agent’ (TA) character that participates in competitions with other player-taught TAs. Background on Conceptual Change

Conceptual change is required in the acquisition of the concept of a fraction because it requires radical changes in the pre-existing concept of a natural number (Stafylidou & Vosniadou, 2004). Motivational, cognitive, and metacognitive processes are also involved.
“Most learning theories rely on the assumption that concepts change through an enrichment of prior knowledge (Vosniadou, 2007). Conceptual change differs from these learning theories, because it cannot be achieved through additive mechanisms involving only the enrichment of pre-existing knowledge. In fact, the conceptual change approach emerged from an ef- fort to explain the radical reorganization of conceptual knowledge and ac- quire an understanding of difficult concepts (Vosniadou, 2007). Conceptual change is required in situations when the new infonnation to be learned conflicts with a leamer’s naive domain-specific theories that have been constructed on the basis of everyday experiences” (p. 400-1).

Numerous models of conceptual change exist (e.g. Merenluoto & Lehtinen, 2004; Vosniadou, 1994, 1999; Duit, 1999). Features: knowledge structure coherence (Ozdemir & Clark, 2007); motivational and socia-cultural aspects of conceptual change; creation of cognitive conflicts (Limon, 2001).

Cognitive conflict – when learner is dissatisfied with her existing conception of phenomenon. A model describing the dynamics of motivational, cognitive, and metacognitive processes in conceptual change is provided by Merenluoto and Lehtinen (2004).

Merenluoto’s and Lehtinen’s (2004) model

In this model, the learner’s cognitive, metacognitive, and motivational sensitivity (i.e., the extent to which the learner is aware of and interested in the novel cognitive aspects of the phenomenon) to the task influences how the task is perceived.

The model distinguishes three possible learning paths:

• The experience of conflict (leading to radical conceptual change) – requires that the learner has sufficient prior knowledge, sensitivity to novel features, tolerance of ambiguity (a metacognitive skill). With high tolerance, a learner feels that the experienced conflict is solvable; in contrast, low tolerance may decrease sensitivity or lead to a loss of trust, resulting in low certainty and avoidance behavior

• The illusion of understanding (leading to an enrichment of naive models or the construction of synthetic models) -  conflict is unnoticed because of over-confidence. Self-efficacy and high motivation may increase a learner’s tendency to take this path. The learner recognizes some familiar elements in the new phenomenon, but her prior knowledge is not adequate for paying attention to the novel aspects of the phenomenon. High motivation may lead to perception of the conflict and result in more radical conceptual change later on.

• Having no relevant perception – (no cognitive change). The learner misses the conflict because of her broad cognitive distance to the phenomenon to be learned (possibly due to cognitive overload; Sweller, van Merrienboer, & Paas, 1998). This may lead to avoidance behavior or routine activity unrelated to the cognitive demands of the task. “Because any attempts to create cognitive conflicts are doomed in this path, game elements facilitating conceptual change cannot be designed for these learners. The only way to support these learners is to provide them with the information that is needed to understand the phenomenon and so be able to perceive the cognitive conflicts” (p. 402-3).

Reflective Thinking and Cognitive Conflict 

“Basically, the perception of cognitive conflict can be seen as being a starting point for reflection. According to Boud, Keogh and Walker (1985), reflection is a human activity in which people recapture their experience, think about it, mull it over, and evaluate it. Xie, Ke and Sharma (2008) point out that researchers seem to agree that the degree of reflection is a function of how much a learner’s cognitive structures are used or changed” (p. 403). (For more on reflection of learning refs, see (e.g. Kiili, 2007; Garris, Ahlers & Driskell, 2002)

The process of reflection can be facilitated by providing ‘cognitive feedback‘ to the player (Butler & Winne 1995). Cognitive feedback aims to stimulate players to reflect on their experiences, problem-solving strategies, and create solutions in order to further develop their mental models (Merrienboer & Kirschner 2007). Thus, cognitive feedback can also be understood as being a trigger of cognitive conflict. Social dialogue (which can be game world- or real world-mediated) needs to be part of the design (Foko & Amory, 2008).

Reflection is also linked to time (Schon, 1983). Reflection-in-action - reflective actions during playing; reflecting-on-action – reflective actions performed after the playing session. For example, the authors observed that ”After competitions and playing breaks these players tended to correct mistakes that they had made earlier. This indicates that the players could use competition as a reflection tool (reflection-in-action) and could reflect on their playing behavior also during the breaks in playing (reflection-on-action). We cannot be sure what triggered players to reflect on their playing behavior during breaks, but we assume that discussions with their classmates about the game may have played a major role” (p. 413).

“As Mayer (2004) has pointed out, guided discovery learning is much more effective than pure discovery learning. Guidance, structure and focused goals cannot be ignored when trying to promote appropriate cognitive processing. Triggers, clear goals and guidance are vital at the least to learners with low metacognitive abilities” (p. 417).

In this paper, the authors describe a semi-automated method for improving a cognitive model called Learning Factors Analysis that combines a statistical model, human expertise and a combinatorial search.

A cognitive model – set of production rules or skills encoded in intelligent tutors to model how students solve problems. (Production = skill = rule)

A good cognitive model:

• captures the fine knowledge components in a curriculum
• provides tailored feedback and hints
• selects problems with difficulty level and learning pace matched to individual students
• improves student learning.

The authors investigate whether or not middle school students’ reflected appraisals of competence, support, and importance were linked to their own corresponding self-perceptions. More specifically, do adolescents’ self perceptions mediate the relations between their perceptions of parents’, teachers’, and peers’ views, behavior, and academic perceptions?

There is an extensive body of research on links between self-perceptions and school achievement (see Eccles et al., 1998), i.e., when students deemed it important to do well in math/science, felt capable of doing well, and reported engaging in behaviors linked to doing well, they would in fact demonstrate successful math/science performance. As individual self-perceptions represent the most proximal influences on academic success, the authors hypothesized that:

• the relations between adolescents’ perceptions of what others thought about the importance of math/science on academic outcomes would occur indirectly through students’ own beliefs about importance and scholastic behavior.
• the effects of perceived support from others and adolescents’ perceptions of others’ beliefs about their competence in math/science would also occur indirectly through students’ own perceived competence and scholastic behavior in math/science.

Theoretical basis: Cooley’s (1902) and Mead’s (1934) symbolic interactionist theories (the self is created through the internalization of others’ beliefs about oneself (see Harter, 1998, 1999); Eccles’s (1993) expectancy-value theory (both children’s expectations for doing well in a particular domain and the value they place on doing well in that domain affect their subsequent academic choices and performance).

The role of parents and teachers as socializers of achievement beliefs is also central to expectancy-value theory. Children’s perceptions of socializers’ beliefs, expectations, and attitudes predict their own self-concept (Eccles, 1984; Eccles et al., 1985; Meece, Parsons, Kaczala, Goff, & Futterman, 1982). However, the role of adolescents’ perceptions of others’ beliefs and behavior in predicting their own academic self-perceptions and performance  is relatively unexplored.

The authors proposed a model in which adolescents’ reflected appraisals of what parents, teachers, and peers think about them with respect to math/science would predict their own academic self-perceptions which would in turn predict academic performance.

Achievement in Latino Youth
On average, economically disadvantaged, African American, and Latino adolescents typically fare worse academically (Byrnes, 2003; Henderson, 1997; Patterson, Kupersmidt, & Vaden, 1990; Pungello, Kupersmidt, Burchinal, & Patterson, 1996). Graham’s (1994) seminal review revealed a striking paucity of theoretically grounded research on the processes underlying achievement in African American youth (see all McLoyd, 1998).

The authors expected that Latino students would demonstrate lower mean levels of reflected appraisals, self-perceptions, and math/science performance than would European American students. A novel contribution of this study is examining whether Latino students believe that others think they are less capable at math/science, that others think math/science is less important for them, and that others provide less support for them to do well in math/science — these unfavorable reflected appraisals should predict lower self-perceptions, which should in turn predict lower math/science performance.

These findings could thus shed light on why Latino students, as a group, typically perform less well academically than do European American students.

The effects of SES were controlled for in the current study. “As many scholars have argued, the confounding of socioeconomic factors with ethnic background has stymied progress of the knowledge of achievement processes (Graham, 1994; Stevenson, Chen, & Uttal, 1990). Parental education level is closely linked to children’s math/science achievement (see Byrnes, 2003; Peng et al., 1995). We thus controlled for SES in the form of parental education level in the current study.”

Measures

• Modified scales from the Self-Perception Profile for Adolescents (SPP-A; Harter, 1988) and the Social Support Scale for Older Children and Adolescents (Harter & Robinson, 1988) were used.
• To minimize response bias, both positively and negatively worded items were included in all scales. Students circled their responses to each item.
• Mean-level ratings (obtained when the student reported on at least 75% of the items) were used for all observed scales.

Reflected Appraisals
Adolescents’ perceptions of significant others’ beliefs regarding how important it was for them to do well in math/science were assessed with a 7-item measure adapted from the How Important These Things Are scale of the SPP-A. Items from the Academic subscale were modified to assess adolescents’ perceptions of others’ beliefs and to inquire specifically about math/science schoolwork. Participants responded separately to each item for mothers, fathers, teachers, and classmates.

Sample items included “My mother (father, teacher, classmates) think(s) that doing well on tests in math/science is very important” and “My mother feels that getting good grades in math/science is not that big of a deal” (reverse coded).

Support for schoolwork.
Students’ perceived support for math/science schoolwork was assessed with a 10-item scale adapted from the Social Support Scale for Older Children and Adolescents (Harter & Robinson, 1988). Five items from both the Approval Support and the Instrumental
Support subscales were modified to reflect support specifically tailored to math/science.

Sample items included “My mother (father, teacher, classmates) praise(s) me for my schoolwork in math/science” (approval support) and “My mother does not teach me about the things I need to know in math/science” (instrumental support, reverse coded). We created average ratings of perceived support from mothers, fathers, teachers, and classmates using all 10 of the support items for each social partner.

Beliefs about the target student’s competence.
Adolescents’ assessments of significant others’ beliefs about their academic competence were measured with a five-item scale adapted from the What I Am Like subscale of the SPP-A (Harter, 1988). Sample items included “My mother (father, teacher, classmates) believe(s) that I am smart for my age in math/science” and “My mother thinks I am pretty slow at finishing my work in math/ science” (reverse coded).

Self-Perceptions
Importance of schoolwork. Adolescents’ own perceptions of the importance of math/science schoolwork were assessed using modifications of the Academic subscale items from the How Important These Things Are scale of the SPP-A (Harter, 1988). Five items tapping the importance of doing well on achievement tests, assignments, and class tests were modified to inquire specifically about math/science subjects. Sample items included “I think that it is important for me to do well in math/science” and “I feel that getting good grades in math/science is not that big of a deal” (reverse coded).

Scholastic behavior. Seven items assessed the extent to which students engaged in a range of behaviors devoted to specific coursework in math/science. Sample items included “I almost always complete my homework on time in math/science” and “I don’t really put as much time and energy into doing my math/science schoolwork as I should” (reverse coded).

Perceived competence. Adolescents’ perceived academic competence was assessed using modifications of the five Academic subscale items from the What I Am Like scale of the SPP-A (Harter, 1988). Sample items for this scale included “I am smart for my age in math/science” and “I am pretty slow at finishing work in math/science” (reverse coded).

Academic Performance
Current performance. School records of students’ current marking period grades assessed current performance. Letter grades for both math and science classes were converted to a 13-point scale (i.e., A
+ = 12, F  = 0).

Prior achievement. Because students’ course grades from the previous grade in school were unavailable, we used school records of students’ achievement test scores from the previous spring (one year before this study) as a measure of prior achievement (percentile ranks from math and science subtests of the Iowa Test of Basic Skills, ITBS).

Parental Educational Level
Students’ reports of their mothers’ highest educational level were used as an index of SES in the current study. Students checked the highest level of education completed by their mothers on a 10-point scale ranging from 1
no formal education to 10
graduate degree. Obtained scores from students ranged from 1 to 10.

Results

Correlations

“Correlational analyses revealed that adolescents’ perceptions of mothers’, fathers’, and teachers’ beliefs and behavior, but not of classmates’ belief and behavior, were most strongly related.”

• Students’ perceptions of how important adults thought it was to do well were positively related to their own perceived importance of math/science (rs = .33–.37) and to the effort (behavior) they put forth to succeed in these courses (rs = .31–.35), as well as to their perceived competence in math/science, to a lesser extent (rs =  .15–.27).
• Perceptions of classmates’ importance beliefs were significantly related only to students’ perceived importance and behavior (rs = .23 -.13, respectively), not to their perceived math/science competence.
• Reflected appraisals of importance for adults, but not for classmates, were also positively related to students’ grades in math/science (rs = .16 –.21).
• Moderate correlations for support from mothers, fathers, and teachers with students’ perceived competence and scholastic behavior in schoolwork (rs = .22–.44).
• Perceived support from classmates was positively correlated with students’ scholastic behavior (r = .21) but not with their perceived competence in math/science.
• Perceived support from others and students’ perceived importance of math/science was positively correlated (rs = .20 –.51).
• Perceived support was also positively correlated with math/science performance (rs =  .13–.35).
• Students’ reflected appraisals of competence for mothers, fathers, teachers, and classmates were significantly correlated with their own perceived competence ratings (rs = .44–.62).
• A similar pattern of results was found for their academic behavior, although the magnitude of these correlations was slightly smaller (rs  .32–.40).
• Students’ reflected appraisals of others’ competence beliefs were positively related to their self-rated importance of doing well in these subjects (rs  .18–.32).
• Students’ reflected appraisals of competence were also positively correlated with their math/science grades (rs  .29 –.37).
• The more important students thought it was to do well in math/science, the more time and effort they spent to succeed in math/science coursework, and the more competent they felt in these classes, the higher were their math/science grades (rs = .26 –.47).

Structural Equation Models
The test of the model as initially specified it yielded acceptable goodness-of-fit statistics. However, inspection of the modification indices and standardized path
coefficients suggested that five new pathways would significantly improve the fit of the model. Each added pathway statistically improved the fit of the model (as indicated by the modification index for that path) but also made both logical and conceptual sense given the underlying theory of the model.

These new pathways indicated that students’ self-perceptions of both (a) the importance of math/science schoolwork and (b) their competence in these courses predicted their scholastic behavior (time and effort invested in doing well), (c) students’ perceived support for math/science predicted their own importance ratings for these courses, (d) students’ perceptions of adults’ beliefs about their competence in math/science predicted their own importance ratings for these courses, and (e) students’ perceptions of support from teachers had a direct effect on their performance in math/science above and beyond the indirect effect of the latent reflected-appraisal support construct. In addition, because it appeared that the effects of students’ valuing of math/science on academic outcomes occurred indirectly through the amount of time and energy they invested in these courses, we deleted the direct path between self-perceived importance and academic performance in the model.

Four main functions of learning games
- prepare for future learning
- for specific learning goals (content & skills)
- for practice of existing skills
- development of 21st century skills

Games offer many assessment opportunities
1) Embedded assessment: user logs, event logs, biometrics

2) Possible variables to measure through embedded assessments:
- General trait – spatial ability, verbal ability, executive fxn
- General state – prior knowledge, learning strats, goal orientation, self-regulation
- Situation-specific state – engagement, emotion, cognitive load, situational interest
- Learning Outcomes – skills, knowledge, comprehension, transfer

What is required, though, is the design of thoughtful game mechanics.

If designed well, game mechanics for learning *may* provide details on learning process and outcome and they *may* reveal insights into learner variables.

Game mechanics for learning need to be specifically designed to:
- Engage the player in meaningful learning activities
- Elicit relevant behaviors that can be observed through the user log and interpreted to reveal learning process, outcome, and learner variables
- Be fun and engaging

Learning Mechanics & Assessment Mechanics need to be translated into the Game Mechanics

Learning Mechanics
- Patterns of behavior or building blocks of learner interactivity
- May be a single action or a set of interrelated actions which form the essential learning activity that is repeated throughout a game
- Learning Mechanic: “Apply rules to solve problems” -> Game Mechanics Variant: “Implode”

-Learning Mechanic “Arrange concepts to solve Problems” -> Game Mechanics Variant: “FlightControl” or  “Toobz”

-Learning Mechanic: “Select Sets to solve Problems” -> Game Mechanics Variant: “Osmosis”

Criteria for translating Learning Mechanics into Game Mechanics
- LM need to be grounded in learning sciences/learning theory
- Describe meaningful interaction with specific subject
- Based on theoretical model of interactivity
- Provide different but equally appropriate solutions to problems so that you have a choice

G4LI building a library of Learning Mechanics that is connected to game mechanic examples. Innovation is to start with how we learn and then pick game mechanics.

Requirements for selecting Game Mechanics based on Learning Mechanic
- GM must not introduce excessive amounts of extraneous cognitive load (e.g., narrative, resource management)
- GM must not reduce amounts of germane cognitive load/mental effort by too much
- GM must not introduce unnecessary confounds (fine motor skills, content knowledge or skills, etc.)

Assessment Mechanics
- Patterns of behavior or building blocks of diagnostic interactivity
- May be a single action or a set of interrelated actions which form the essential diagnostic activity that is repeated throughout a game

Assessment Mechanic: Apply Rules to solve Problems
- Learner selects among different rules and shows where/how they apply

Evidence-Centered Design:
- Student/competency Model – what should be assessed? What competencies am I interested in?
- Evidence Model – what kind of evidence would I accept/what behaviors reveal these constructs?
- Task model – what tasks should elicit these behaviors?

Criteria for Assessment Mechanics
- Based on Evidence Model
- Describe Aspects of Task Model
- Test Theoretical Concerns
- Create repeated exposures to similar problems to allow for multiple observations of the behavior of interest
- Make explicit the steps learners used for problem solving
- Assessment character may or may not be obvious to learner

G4LI Library of Assessment Mechanics that have Game Mechanics as examples
Ex LM: Learners apply rules to solve problems; Learner chooses how different items are to be arranged in space and time in order to solve a problem; Learner selects different items that belong to each other in time or space.

Requirements:
- reduce extraneous cognitive load
- germane cognitive load/mental effort
- fine motor skills
- content knowledge or skills
- emotional response

Summary:
- Design of GM benefits from separation of LM, AM
- Learning Theory, ECD provide foundation
- Multiple Game Mechanics (in multiple game genres) possible for each Learning/Assessment Mechanic
- Need to consider additional confounds or constraints they introduce

Log file analysis

DataCrypt for Data Collection
- Log file open standard for game research
- Tags: Common Core Standards, skills
- User actions
- Game Events
- Biometric Data
- Cloud-based data storage
- Analysis tools for Data Viz and Data Mining

The authors present a way to use both past and current student data is generate live hints for students who are completing programming exercises during a national programming online tutorial and competition (the NCSS Challenge). These hints highlight note sections or practice questions that are relevant to the user’s mistake (‘post-failure’) or offer pre-emptive tips to prevent future mistakes (‘pre-emptive’). Clustering, association rules and numerical analysis was used to find common patterns affecting the learners’ performance that we could use as a basis for providing hints. During its live operation in 2009, student data was mined each week to update the system as it was being used. The hinting system were evaluated through a large-scale experiment with participants of the 2009 NCSS Challenge. Users who were provided with hints achieved higher average marks than those who were not and stayed engaged for longer with the site.

“While the use of data mining to aid the diagnosis of students’ behaviour and ability is common, relatively little work has been done in using data mining to support student problem solving. One system that aims to do this is the Logic-ITA [6], where the system takes into account past associations of student mistakes to provide on-the-fly, proactive feedback to the students.”

Pre-emptive hints are available to weak students before they submit any code for a question. Postfailure hints are provided after a student’s submission failed. [also possible in Numbaland]

Description of competition for data analysis purposes: 16,814 submissions were gathered from 712 separate users. There were 25 questions in total (5 per week) that were available to the students, usually in increasing order of difficulty. Students could submit
several times for the same question until successful. All these attempts were recorded, along with the mark eventually obtained by students in each question.

Creating student clusters:

• Used the K-Means clustering algorithm used in Tada-Ed [9].
• For each student ID, collated the following attributes for each of the 25 questions: whether the student attempted the question (nominal), whether the student eventually passed the question (nominal), and the marks gained for the question (numeric [0 or 5-10]). Also computed the average numbers of passed and failed questions, and the average number of submissions before the student passed a question.
• Clustering with these attributes produced three distinct groups: “strong”, “medium” and “weak” students.
• The effectiveness of clustering with these pre-processed attributes indicated that clustering was a viable technique for discriminating between students.

Creating question clusters:

• Goals were to find questions that were similar to each other and to group questions by difficulty.
• Used the K-means algorithm. Similarity-based clusters were extracted using the question metadata (topic tags).
• Found 5 clusters, as each of the 5 weeks of the Challenge introduced new topics.
• Difficulty-based clusters of questions were extracted based on the number of students who passed each question and the percentage of students who passed it that attempted it.
• Found three clusters: “easy”, “medium” and “hard”.

Mining associations in topics:

• Goal was to find association rules that indicated which topics should be mastered before another question was attempted, so that the hints could suggest topics that students should review before moving onto a more complex one.
• Mined sequences of tags that students failed on; ordered the students’ results chronologically, and kept an ordered sequence of the tags for each question they made an incorrect submission to. Used these sequences to generate association rules.
• Lowered the support and confidence to 20%, and used cosine, possibly a more appropriate evaluation metric for educational data [10].
• Postprocessed the rules generated by the aPriori algorithm [11] to discard rules with a cosine of less than 0.65 [10] and rules with topics out of the order in which they appeared in the notes. Only retained rules that had two topics in the antecedent and one in the consequent. Finally, manually extracted the rules in which the three topics involved were related to one another to remove trivial rules.
• Ended up with 83 rules. Ex: students who struggle with basic arithmetic in Python and comparison operators also struggle with how to loop over a set of values; those who struggle with converting to integers and while loops also struggle with stopping after a number of iterations.

Numerical Analysis

• Used to find frequencies and averages for certain aspects of the data. An important measure was to have an idea of the “give-up point,” i.e., the number of wrong submissions a student made before he or she stopped attempting it.
• For each question, the total number of submissions made by students who never passed/the number of students who attempted but did not pass. We then computed the mean of the averages, which was found to be 3.7. This was used in the final system as the point at which students were presented with post-failure hints; a student would only receive such hints after making their fourth incorrect submission to a question.

Evaluation

• Learning. Hinted group’s mean score was 4.02 (sd = 2.78), while the control group had a mean score of 3.18 (sd = 2.71). This was a difference of 0.84, i.e. an increase of 26.4%, with a significance of p < 0.0006 using an Approximate Randomisation test [12]. We used this because the students’ marks were not normally distributed, making a t-test inappropriate.
• Engagement. There were consistently more users in the hinted group who made submissions, meaning the hinted group of users had an overall higher level of participation over the five weeks of the Challenge.
• User experience. 67% of students found the topics “relevant” or “somewhat relevant” and 90% of them found the questions “relevant” or “somewhat relevant.” Therefore, it is clear that as far as the users were concerned, the methods for choosing topics to present were effective. In addition, 71% of students stated they would like more hints.

“These results show that the use of data mining to provide hints as part of the system loop is extremely effective, and can be used to build intelligent systems with much less of the time and cost expenses associated with traditional ITSs.”

References

[6] Merceron, A. and K. Yacef, Educational Data Mining: a Case Study, in proceedings of Artificial Intelligence in Education (AIED2005), C.-K. Looi, G. McCalla, B. Bredeweg, and J. Breuker, Editors. 2005, IOS Press: Amsterdam, The Netherlands. p. 467-474.

[9] Merceron, A. and K. Yacef, TADA-Ed for Educational Data Mining. Interactive Multimedia Electronic Journal of Computer-Enhanced Learning, 2005. Volume 7, Number 1: p. http://imej.wfu.edu/articles/2005/1/03/index.asp.

[10] Merceron, A. and K. Yacef, Interestingness Measures for Association Rules in Educational Data, in proceedings of International Conference on Educational Data Mining. 2008: Montreal, Canada.

[11] Agrawal, R. and R. Srikant, Fast Algorithms for Mining Association Rules, in proceedings of VLDB. 1994: Santiago, Chile.

[12] Chinchor, N., Statistical significance of MUC-6 results, in proceedings of Fourth Message Understanding Conference (MUC-4). 1992. p. 390-395.

The authors conducted an analysis of ITS data from 1,392 students over two school years, comparing two conditions: a ‘static condition’ in which student data comprised only practice items without intervention; and a ‘dynamic condition’ in which data included information related to whether or not the student sought help when they encountered difficulties. The main goal of this analysis was to investigate whether or not ‘dynamic assessment’ was an efficient and accurate way to assess student learning (based on a year-end state test).

“Dynamic assessment (DA, or sometimes called dynamic testing, Grigorenko & Sternberg, 1998) has been advocated as an interactive approach to conducting assessments to students in the learning systems as it can differentiate student proficiency at the finer grained level. Different from traditional assessment, DA uses the amount and nature of the assistance that students receive which is normally not available in traditional practice test situations as a way to judge the extent of student knowledge limitations.”

Grigorenko and Sternberg (1998) reviewed relevant literature on this topic and expressed enthusiasm for the idea.

Sternberg & Grigorenko (2001, 2002) argued that dynamic tests not only serve to enhance students’ learning of cognitive skills, but also provide more accurate measures of ability to learn than traditional static tests.

Bryant, Brown & Campione, 1983; Campione & Brown, 1985 – took a graduated prompting procedure to compare traditional testing paradigms against a dynamic testing paradigm. In the dynamic testing paradigm, learners are offered increasingly more explicit prewritten hints in response to incorrect responses. They found that student learning gains were not as well correlated (R = 0.45) with static ability score as with their “dynamic testing” (R = 0.60) score.

*However, although DA has been shown to be effective predicting student performance, it generally takes longer for students to finish a test using the DA approach than using a traditional test.

A computer-based intelligent tutoring system called ASSISTments was used. In this instance ASSISTments presented math problems to students 13 to 16 years old who were in middle or high school. “The hypothesis is that ASSISTments can do a better job of assessing student knowledge limitations than practice tests or other online testing approaches by using the DA approach based on the data collected online.”

The authors compared the same student’s work in two different conditions, ruling out the subject effect. The student’s end of year state accountability test score was used as the measure of student achievement.

• Simulated static assessment condition (A’): 40 minutes of student work selected from existing log data on only main items. The data for condition A’ included student response data during the first 40 minutes of work on only main problems; all responses and other actions during the DA portion were ignored.

• Dynamic assessment condition (B’): 40 minutes of work selected from existing log data on both main items and the scaffolding steps and hints. Data for condition B’ included all the responses for main questions and scaffoldings, as well as hint requests.

Metrics for dynamic testing that measures student accuracy, speed, attempts, and help-seeking behaviors. Condition A’ used only the first one as predictor to simulate paper practice tests by scoring students either correct or incorrect on each main problem while condition B’ used all the metrics.

• Main_Percent_Correct – students’ percent correct on main questions; often referred to as the “static metric”.
• Main_Count – the number of main items students completed. This measures students’ attendance and how on-task they are. Also reflects students’ knowledge since better students have a higher potential to finish more items in the same amount of time. This is especially true for condition B’ where students’ work on scaffolding also counted as part of the 40 minute work. In condition A’, low performing kids could go through many items but give wrong answers since their time consumed during the tutoring session is disregarded.
• Scaffold_Percent_Correct – students’ percent correct on scaffolding questions. In addition to original items, students’ performance on scaffolding questions was also a reasonable reflection of their knowledge. For instance, two students who get the same original item wrong may, in fact, have different knowledge levels and this may be reflected in that one may do better on scaffolding questions than the other.
• Avg_Hint_Request – the average number of hint requests per question.
• Avg_Attempt – the average number of attempts students made for each question.
• Avg_Question_Time – on average, how long it takes for a student to answer a question, whether original or scaffolding, measured in seconds.

Stepwise linear regression was used to predict student state test scores. For all the models, the dependent variable is the state test score; in terms of the independent variable, for condition A’, it was Main_Percent_Correct; while for condition B’, it was a collection of metrics: Main_Percent_Correct, Main_Count, Scaffold_Percent_Correct, Avg_Hint_Request, Avg_Attempt, Avg_Question_Time.

Results

• More attempts or more hints on a question correlate with a lower estimated score.
• The dynamic assessment condition did a significantly better job at predicting state test scores than the control static condition.
• Dynamic assessment is more efficient than just giving practice test items. DA can assess student math performance just as accurately as a traditional practice test, even when controlling for testing time.

References

Brown, A. L., Bryant, N.R., & Campione, J. C. (1983). Preschool children’s learning and transfer of matrices problems: Potential for improvement. Paper presented at the Society for Research in Child Development meetings, Detroit.

Campione, J.C., Brown, A.L., & Bryant, N.R. (1985). Individual differences in learning and memory. In R.J. Sternberg (Ed.). Human abilities: An information-processing approach, 103–126. New York: W.H. Freeman.

Campione, J.C.& Brown, A.L. (1985). Dynamic assessment: One approach and some initial data. Technical Report No. 361. Cambridge, MA: Illinois University, Urbana. Center for the Study of Reading. ED269735

Feng, M., Heffernan, N.T., & Koedinger, K.R. (2009). Addressing the assessment challenge in an online system that tutors as it assesses. User Modeling and User-Adapted Interaction: The Journal of Personalization Research. 19(3), 2009.

Feng, M., Heffernan, N., Beck, J, & Koedinger, K. (2008). Can we predict which groups of questions students will learn from? In Beck & Baker (Eds.). Proceedings of the 1st International Conference on Education Data Mining. Montreal, 2008.

Feng, M., Heffernan, N. T., & Koedinger, K. R. (2006). Addressing the testing challenge with a web based E-assessment system that tutors as it assesses. Proceedings of the 15th Annual World Wide Web Conference. ACM Press: New York.

Fuchs, L.S., Compton, D.L., Fuchs, D., Hollenbeck, K.N., Craddock, C.F., & Hamlett, C.L (2008). Dynamic assessment of algebraic learning in predicting third graders’ development of mathematical problem solving. Journal of Educational Psychology, 100(4), 829-250.

Fuchs, D., Fuchs, L.S., Compton, D.L., Bouton, B., Caffrey, E., & Hill, L. (2007). Dynamic assessment as responsiveness to intervention. Teaching Exceptional Children, 39 (5), 58-63.

Grigorenko, E. L. and Sternberg, R. J. (1998). Dynamic testing. Psychological Bulletin, 124, 75–111.

Sternburg, R.J., & Grigorenko, E.L. (2001). All testing is dynamic testing. Issues in Education, 7, 137-170.

Sternburg, R.J., & Grigorenko, E.L. (2002). Dynamic testing: The nature and measurement of learning potential. Cambridge, England: Cambridge University Press.