This NEMP Probe study was associated with the course, TL835 Issues in Numeracy, in the Master of Teaching and Learning of the Christchurch College of Education. In 2005 five students enrolled in the course were required to collaboratively and individually analyse NEMP data for a research assignment within this course. The students were fourth year primary teacher education students enrolled in the full-time BTchLn(Hons) programme at the Christchurch College of Education. In this report, these five students are collectively referred to as the 'research team' and individually as researchers, and the author is the course lecturer. This report discusses some background issues to the study, outlines the analysis design and summarises key aspects of the analysis of the data from the research team. Further information about the process of incorporating experience with NEMP data within coursework is reported elsewhere.
Methodology
NEMP Items selected
Five task items from the 2001 Year 4 NEMP were selected for analysis, in consultation with USEE. These were chosen to provide a focus on numeracy or number-related activity, a range of contextualised and de-contextualised tasks, and a mix of written and video data (all from Crooks and Flockton, 2002).
Three independent tasks were selected and provided written data. The tasks were:
- Addition Examples: (p. 14)
- Speedo: (p. 17)
- Money A: (p. 37)
Two one-to-one tasks were selected, providing video data.
- 36 and 29: (p. 20)
- Number Line Y4: (p. 23)
Addition Examples was a Trend Task and Number Line was an Access Task. These five tasks plus the survey data were available for each child.
Sample
The national sample for mathematics of 1440 year 4 children was divided into three groups of 480 (Crooks and Flockton, 2002, p. 5). Each group of 480 were given different tasks, so in order to track tasks for the same children, tasks from one of these groups were selected. The data sample provided by NEMP was 47 and the final sample analysed by the research team was 40 (representing 8.5% of the sample group of 480). The extra data sets from seven children were used for trial analysis before the researchers each took responsibility for the data sets of their eight children.
Changes made to the Method
On viewing the video data, the Number Line Y4 task was eliminated as it was not always possible to read the fraction on each card in order to identify where each card was placed. This illustrated how the interviewer in situ was better placed than the video viewer to accurately record the children's actions. Some of this task data was also compromised due to inconsistency of interviewing. Consequently, the research team decided to omit the Number Line Y4 task from this study.
Analysis of Data
The process of data analysis centred around immersion in the data, category generation, refining category analysis and peer review. These aspects were considered important course experiences for the researchers.
The research team familiarised themselves with the marking schedules for each task supplied by NEMP. They examined the task, became familiar with the marking approach, and generated further analysis categories guided by the course lecturer. Further categories focused on errors, and associated possible misconceptions or perturbations in mathematical activity. The research team refined the analysis protocol for each of the three written tasks, outlined in the next section. Each researcher used the analysis protocols for their sample of eight children, recording results in a grid for each task. This was followed by peer review of the analysis in order to check for consistency and to add further clarity to the analysis. Peer review involved a member of the research team analysing another's data set and recording their results. This process of comparing analysis decisions and discussing similarities and differences provided opportunities for clarifying and 'sharpening' the analysis protocols, resulting in greater consistency between researchers.
One of the constraints of analysing children's written work is that researchers are unable to use direct observation or ask children to self report their strategies. Children's strategies can only be inferred from the written record and, while inferences are influenced by research, interpretations can still be somewhat speculative. When known strategies could not be inferred or peer review revealed multiple interpretations, then the research team resolved to categorise these items as unknown.
For the video task (36 and 29), the researchers generated a transcript of what was said by the child and the interviewer, adding supplementary information observed on the video. The transcripts were analysed using the NEMP marking protocol and peer reviewed for one child. Finally, the research team put together a profile for each of their eight children based on the survey data and the analysis of the four task items.