A validation of eye movements as a measure of elementary school children’s developing number sense (Schneider

Schneider, M., Heine, A., Thaler, V., Torbeyns, J., De Smedt, B., Verschaffel, L., Jacobs, A. M., et al. (2008). A validation of eye movements as a measure of elementary school children’s developing number sense. Cognitive Development, 23(3), 409–422.

The authors tested the validity of eye-movement data as a means for investigating children’s use of the number line when solving number line estimation tasks, and as a measure of children’s developing number sense. In a cross-sectional design with children from Grades 1 to 3, they assessed (a) the accuracy of manual solutions of number line estimation tasks, (b) the accuracy of the positions fixated by gaze while solving a second set of number line estimation tasks, and (c) the accuracy of responses to mental addition tasks.

Research questions:

  • Is grade-related increase in children’s estimation competence only reflected by manual answers or also by eye-tracking data?
  • Are individual differences in the accuracies of the estimated positions and in the accuracies of their eye movements during the estimation process correlated?
  • Are the accuracy of the estimated positions and the accuracy of the positions fixated by gaze during solution production correlated with children’s addition competence?
  • To what extent does the criterion validity of the eye-movement measure increase with age?
  • Does eye-movement data indicate that older children increasingly use the midpoint of the number line as an orientation point?”

Number sense: “ability to quickly understand, approximate, and manipulate numerical quantities” (Dehaene, 2001, p. 16).

“The “mental number line” is regarded as the core neurocognitive system underlying number sense (Fias, Lammertyn, Reynvoet, Dupont, & Orban, 2003; Hubbard, Piazza, Pinel, & Dehaene, 2005; Pinel, Dehaene, Riviere, & LeBihan, 2001), which in turn underlies a variety of behavioral competencies, like estimating, computing, and efficiently using notational systems to solve mathematical problems (Berch, 2005; Jordan, Kaplan, Locuniak, & Ramineni, 2007).” Represents the magnitudes of numbers in an analogous form.

“Arabic numerals, like 5 and 7, do not allow for any direct inference as to which of them is the one with the higher value. The same is true for number words, like 5 and 7. In contrast, when numerical magnitudes are represented by positions on a number line, one immediately grasps which of them is the higher value number. Therefore, Case and Okamoto (1996) suggested that children use the mental number line “to build models of the conceptual systems that their culture has evolved for measuring such dimensions as time, space. . . They use it to make sense of any direct instruction that they may receive regarding the particular systems that their culture has evolved for arranging numbers into groups and for conducting numerical computations” (pp. 8–9).”

Measuring number sense:

  • See Berch, 2005; Jordan, Kaplan, Oláh, & Locuniak, 2006.
  • Siegler and Opfer (2003) “asked children to estimate the positions of given numbers on an external number line where only the starting and the end points were labeled. They interpreted the patterns of estimates as indicative of children’s representation of magnitudes on their internal number line. These answer patterns relate not only to children’s competence in performing other estimation tasks, such as numerosity estimation and computational estimation, but even to children’s addition competence and general math achievement (Booth & Siegler, 2006, in press; Siegler & Booth, 2004).”
  • The number line estimation task is a “practical and powerful tool applicable across a wide range of age groups. It has been hypothesized to reflect children’s mental representation of numbers more directly than alternative assessments of number sense do.”

“Despite its potential benefits, the number line estimation task has a major drawback. Although it is easy to measure such products of children’s estimation processes as accuracy, solution times and estimate patterns, it is hard to investigate the processes themselves that children employ to construct their solutions.”

Solution strategies:

  • [Grade 1] Counting up strategy, counting down strategy, in whole units or decades from the beginning of the number line (Petitto, 1990; Newman & Berger, 1984).
  • [Older children] Start counting up from the midpoint.

“Eye-movement data can be collected with high temporal and spatial resolution (e.g., several-hundred measures per second with a spatial precision of 0.01◦ ) notwithstanding the fact that the reliability of the resulting data can suffer from technically caused measurement error and task-irrelevant fixations. Compared to accuracy and speed measures, eye-tracking data potentially provide more direct evidence of the process of problem solving. Moreover, these data are more objective than self-reports or behavioral observations of strategy use.”

“Children’s increasing ability to focus their attention on the task-relevant features of a problem situation are likely to reduce the number of task-irrelevant eye movements and, thus, to increase the validity of our eye-movement measure over the three grade levels.”

“By means of eye-tracking data, Rehder and Hoffman (2005) demonstrated that in adults, increasing competence in object categorization goes along with an increasing tendency to focus attention on task-relevant characteristics of a problem situation. To guide attention to task-relevant characteristics of a problem situation and to ignore task-irrelevant features is an important part of mathematical competence (The Cognition and Technology Group at Vanderbilt, 1992). Therefore, eye movements might reflect individual differences in mathematical competence.” [could also guide the design of scaffolding in a math game…]

“Two studies (Green, Lemaire, & Dufau, 2007; Verschaffel, De Corte, Gielen, & Struyf, 1994) have demonstrated that eye movements validly reflect various strategies chosen by elementary school children and adults to solve mental addition problems.”

“the external number line, as a diagram, is an analogous and more holistic diagrammatic knowledge representation (Larkin & Simon, 1987). The validity of eye movements as an indicator of children’s competence concerning this type of external knowledge representation has not been investigated.”


  • Children had to solve 30 trials of the number line estimation task.
  • Following Rittle-Johnson, Siegler, and Alibali’s (2001) example, we coded an answer as correct if it was within an error margin of ±10% of the number line around the actual position of the stimulus on the line, and as incorrect in all other cases.
  • The stimuli were selected by a pseudo-random algorithm from the natural numbers between 0 and 100.
  • For the eye-tracking version of the number line estimation task, children were instructed to actively search for and focus their gaze on the correct position for each number. After 4000 ms, a marker appeared. The children were asked to decide as fast as possible whether the marker position was correct or not and to give their answer by clicking a respective button. Button clicks and reaction times were recorded automatically.
  • No feedback was given to the children during the experimental trials. The order of the two number line estimation tasks was counter-balanced at each grade level.
  • In addition to individual fixation accuracies, we derived the frequency distribution of all fixations over the number line for each grade level by computing the position of each fixation on the number line and by rounding to the nearest whole value. We counted per individual and task how often each of the 101 positions of whole numbers on the line had been fixated. The resulting value for each position was averaged per grade level. To enhance readability, these values were multiplied by 10,000.
  • Fixation accuracy measure is based on the assumption that not only the last fixation but all fixations during a trial indicate children’s knowledge.

Some results

  • Addition accuracy increases most strongly from 15 to 91% and has the highest proportion of explained variance, as indicated by 2 values.
  • The effect sizes also show that grade-related increases in knowledge are more clearly reflected by estimation accuracy than by fixation accuracy. Of the three variables, fixation accuracy shows the least change and the smallest, albeit still high, proportion of explained variance.
  • Significant change in estimation accuracy and fixation accuracy occurs only between Grades 1 and 2, but not between Grades 2 and 3. Differences in addition accuracy are highly significant between each of the grade levels.
  • Exploratory comparisons of (a) the last fixation in each trial with (b) all previous fixations in that trial confirm our expectations. The average percentage of correct fixations in the sample is 42.1% for only the last fixations and 34.0% for all fixations. Both accuracies lie well above chance level (i.e. 20%) and, thus, indicate knowledge.
  • The accuracies for both types of fixations correlate across individuals with r = .53, p = .017, for first graders, r = .73, p < .001, for second graders, and r = .70, p < .001, for third graders, suggesting that the two groups of fixations measure similar aspects of children’s knowledge.
  • Guided by our second research question, we computed the correlations between estimation accu- racy and fixation accuracy. These are r = .28, p = .212, for first graders, r = .66, p = .002, for second graders, and r = .63, p = .001, for third graders.
  •  Children in all three grades fixate positions near the starting point of the line, near the end of the line, and near the number 50 in the middle of the line more frequently than any other position. respectively.
  • Children’s fixations (a) validly reflect grade-related competence increases, (b) are closely related, in Grades 2 and 3, to manual solutions of estimation tasks, (c) are related, in Grade 2, to addition com- petence, (d) are very systematically distributed over the number line, and (e) replicate Petitto’s (1990) findings with respect to the use of the midpoint strategy and the counting-up strategy by students in Grades 1–3.

“The fact that grade level explains more than one third of the interindividual variance of children’s fixation accuracies establishes that eye movements reflect children’s increasing knowledge about natural numbers, their interrelations, and ways of their spatial representation. This understanding of natural numbers lies at the very core of children’s number sense and is a prerequisite for their future acquisition of more advanced mathematical concepts (Dehaene, 1997).”

“As explained by Verschaffel et al. (1994) and Green et al. (2007), identifying trial-by-trial strategy use by means of eye-tracking data can considerably improve research on the development of mathematical strategies. For example, parallel use of a verbal and a nonverbal measure of strategy use allows researchers to investigate the interaction of explicit and implicit knowledge during strategy development (Siegler & Stern, 1998). However, distinguishing strategy use on a trial-by-trial basis by means of eye-tracking may be as difficult as it is desirable. In most eye-tracking studies the data are aggregated over trials – and often over persons – because ‘eye tracking data is never perfect. The system may lose track of the pupil or the corneal reflection, or the observation may be simply incorrect (e.g., beyond the screen limits even though the subject is clearly looking at the screen)’ (Aaltonen, Hyrskykari, & Räihä, 1998, p. 135). “Eye-movement data are inherently noisy” (Hornof & Halverson, 2002, p. 593), due to task-irrelevant fixations (e.g., when the individual is distracted by sounds from the surroundings). Individuals may make data-distorting head-movements (children more so than adults) which might even require a re-calibration of the scanning system. Finally, individuals may use peripheral vision to detect information from regions on the screen that they do not fixate directly. All these influences increase measurement error, thus, making it harder to find effects on the trial level than on the level of aggregated data (Rayner, 1998; Verschaffel et al., 1994).”

“Eye movements allow for a direct investigation of how children orient themselves in problem situations and how they direct their attention to specific features. Although previous theories tended to conceptualize mathematical problem solving as an abstract symbol- manipulation process, more recent approaches emphasize the interaction with problem situations as a highly important part of mathematical competencies (Collins, Greeno, & Resnick, 2001). For example, advocates of the situated-cognition view have argued that the ability to pick up action-relevant information from the environment is the most important foundation of competent problem solving and knowledge transfer (Greeno, 1994; Greeno, Moore, & Smith, 1993). Eye movements offer a means of investigating the dynamic and selective search for information in problem situations. Such search invokes higher-level cognitive processes, including what one knows about, and expects of, a situation (Rehder & Hoffman, 2005). More advanced analysis techniques for eye-tracking data, such as fixation density maps (Ouerhani, von Wartburg, Hügli, & Müri, 2004), may become useful for deepening our understanding of these phenomena.”


Berch, D. B. (2005). Making sense of number sense: Implications for children with mathematical disabilities. Journal of Learning Disabilities, 38(4), 333–339.

Booth, J. L., & Siegler, R. S. (2006). Developmental and individual differences in pure numerical estimation. Developmental Psychology, 41, 189–201.

Booth, J. L., & Siegler, R. S. (2008). Numerical magnitude estimations influence arithmetic learning. Child Development, 79(4), 1016–1031.

Butterworth, B. (2005). Developmental dyscalculia. In J. I. D. Campbell (Ed.), Handbook of mathematical cognition (pp. 455–467). New York: Psychology Press. Campbell, J. I. D. (Ed.). (2005). Handbook of mathematical cognition. New York: Psychology Press.

Case, R., & Okamoto, Y. (1996). The role of central conceptual structures in the development of children’s thought. Monographs of the Society for Research in Child Development, 61(1 2), 1–295.

Collins, A., Greeno, J. G., & Resnick, L. B. (2001). Educational learning theory. In N. J. Smelser & P. B. Baltes (Eds.), International encyclopedia of the social & behavioral sciences (pp. 4276–4279). Oxford: Elsevier Science.

Dehaene, S. (1997). The number sense: How the mind creates mathematics. New York: Oxford University Press.

Dehaene, S. (2001). Précis of “The number sense”. Mind and Language, 16, 16–32.

Dehaene, S., Bossini, S., & Giraux, P. (1993). The mental representation of parity and number magnitude. Journal of Experimental Psychology: General, 122(3), 371–396.

Fias, W., & Fischer, M. H. (2005). Spatial representations of numbers. In J. I. D. Campbell (Ed.), Handbook of mathematical cognition (pp. 43–54). New York: Psychology Press.

Fias, W., Lammertyn, J., Reynvoet, B., Dupont, P., & Orban, G. A. (2003). Parietal representation of symbolic and non-symbolic magnitude. Journal of Cognitive Neuroscience, 15(1), 47–56.

Geary, D. C., & Hoard, M. K. (2005). Learning disabilities in arithmetic and mathematics: Theoretical and empirical perspectives. In J. I. D. Campbell (Ed.), Handbook of mathematical cognition (pp. 253–267). New York: Psychology Press.

Green, H. J., Lemaire, P., & Dufau, S. (2007). Eye movement correlates of younger and older adults’ strategies for complex addition. Acta Psychologica, 125, 257–278.

Greeno, J. G. (1994). Gibson’s affordances. Psychological Review, 101(2), 336–342.

Greeno, J. G., Moore, J. L., & Smith, D. R. (1993). Transfer of situated learning. In D. K. Detterman & R. J. Sternberg (Eds.), Transfer on trial: Intelligence, cognition, and instruction. Norwood, NJ: Ablex.

Hornof, A. J., & Halverson, T. (2002). Cleaning up systematic error in eye tracking data by using required fixation locations. Behavioral Research Methods, Instruments, and Computers, 34(4), 592–604.

Hubbard, E. M., Piazza, M., Pinel, P., & Dehaene, S. (2005). Interactions between number and space in parietal cortex. Nature Reviews Neuroscience, 6(6), 435–448.

Jordan, N. C., Kaplan, D., Locuniak, M. N., & Ramineni, C. (2007). Predicting first-grade math achievement from developmental number sense trajectories. Learning Disabilities Research & Practice, 22(1), 36–46.

Jordan, N. C., Kaplan, D., Oláh, L. N., & Locuniak, M. N. (2006). Number sense growth in kindergarten: A longitudinal investigation of children at risk for mathematical difficulties. Child Development, 77, 153–175.

Larkin, J. H., & Simon, H. A. (1987). Why a diagram is (sometimes) worth ten thousand words. Cognitive Science, 11, 65–99.

Newman, R. S., & Berger, C. F. (1984). Children’s numerical estimation: Flexibility in the use of counting. Journal of Educational Psychology, 76(1), 55–64.

Ouerhani, N., von Wartburg, R., Hügli, H., & Müri, R. M. (2004). Empirical validation of the salience-based model of visual attention. Electronic Letters on Computer Vision and Image Analysis, 3, 13–24.

Petitto, A. L. (1990). Development of numberline and measurement concepts. Cognition and Instruction, 7(1), 55–78.

Pierce, C. A., Block, R. A., & Aguinis, H. (2004). Cautionary note on reporting eta-squared values from multifactor anova designs. Educational and Psychological Measurement, 64(6), 916–924.

Pinel, P., Dehaene, S., Riviere, D., & LeBihan, D. (2001). Modulation if parietal activation by semantic distance in a number comparison task. NeuroImage, 14, 1013–1026.

Rayner, K. (1998). Eye movements in reading and information processing: 20 years of research. Psychological Bulletin, 124(3), 372–422.

Rehder, B., & Hoffman, A. B. (2005). Eyetracking and selective attention in category learning. Cognitive Psychology, 51, 1–41.

Rittle-Johnson, B., Siegler, R. S., & Alibali, M. W. (2001). Developing conceptual understanding and procedural skill in mathematics: An iterative process. Journal of Educational Psychology, 93(2), 346–362.

Siegler, R. S., & Booth, J. L. (2004). Development of numerical estimation in young children. Child Development, 75(2), 428–444.

Siegler, R. S., & Opfer, J. E. (2003). The development of numerical estimation: Evidence for multiple representations. Psychological Science, 14, 237–243.

Siegler, R. S., & Stern, E. (1998). Conscious and unconscious strategy discoveries: A microgenetic analysis. Journal of Experimental Psychology: General, 127(4), 377–397.

The Cognition and Technology Group at Vanderbilt. (1992). The Jasper series as an example of anchored instruction: Theory, program description and assessment data. Educational Psychologist, 27, 291–315.

Verschaffel, L., De Corte, E., Gielen, I., & Struyf, E. (1994). Clever rearrangement strategies in children’s mental arithmetic: A confrontation of eye-movement data and verbal protocols. In J. E. H. V. Luit (Ed.), Research on learning and instruction of mathematics in kindergarten and primary school (pp. 153–180). Doetinchem, The Netherlands: Graviant.

Verschaffel, L., De Corte, E., & Pauwels, A. (1992). Solving compare problems: An eye movement test of Lewis and Mayer’s consistency hypothesis. Journal of Educational Psychology, 84(1), 85–94.

Whiteside, J. A. (1974). Eye movements of children, adults, and elderly persons during inspection of dot patterns. Journal of Experimental Child Psychology, 18(2), 313–332.”


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: