Research on the EFFECTIVENESS OF MATHEMATICS SPECIALISTS

The research component is led by Patricia Campbell at the University of Maryland. Its goal is to evaluate the effectiveness of Mathematics Specialists through a treatment-control design as it investigates the potential relationship between four outcomes:

• Mathematical content and pedagogical knowledge and the mathematical beliefs of the Mathematics Specialists;
• Leadership and support practices of the Mathematics Specialists;
• Nature of teachers’ classroom mathematics instruction as interpreted by the Mathematics Specialists, the degree of engagement of teachers with the Mathematics Specialists, and the mathematical beliefs of the teachers; and
• Mathematics achievement of students in the schools served by the Mathematics Specialists. 

The data sources for measuring these outcomes will include:

• Paper and pencil assessments of the mathematical content knowledge and mathematical pedagogical knowledge of Mathematics Specialists;
• Paper and pencil assessments of the mathematical beliefs of Mathematics Specialists and teachers;
• Virginia ’s standardized statewide test for mathematics (SOL);
• Modification of the Project IMPACT mathematics interview assessment ( Campbell , 1995);
• Hours of attendance in mathematics professional development by teachers and Mathematics Specialists; and,
• Specialists’ activity and reflection logs as entered by the Specialists into a Palm Pilot data collection system.

The research will explore a number of core questions:

• Does professional development and practice change Specialists’ mathematical content knowledge, pedagogical knowledge, and beliefs?
• What types of Specialist behaviors and interactions with individual teachers impact student achievement and teacher change?
• Does the impact of the Specialist change over time?
• Do Specialists influence teachers’ mathematics beliefs and pedagogical approaches?

In order to measure potential change in Specialists’, teachers’, and students’ understanding, as well as the relationship between these variables over time, this project will use repeated measures within a stratified randomized design.  As described in Section IIIA., twelve triples of schools will be identified, with the schools within each triple having a similar prior tradition of mathematics achievement, serving a similar population of students demographically, and operating within the same school district.  Within each triple, schools will be randomly assigned to one of three categories: Treatment 1 (Cohort 1 Mathematics Specialist placed in fall 2005), Treatment 2 (Cohort 2 Mathematics Specialist placed in fall 2007), and Control (No Mathematics Specialist).  This analysis will access students’ mathematics achievement scores over time, with a teacher having different classes of students across years and these classes having different collections of students across years.  For all students in one classroom in one year, the analysis will treat their scores on the SOL as repeated measures of achievement that yield a teacher’s classroom score for that year.  Then these classroom scores from year to year are repeated measures of the effect of the teachers’ instruction, with the potential concurrent repeated effect of teachers’ engagement with the Mathematics Specialist and the expertise of the Mathematics Specialist over time.  Recognizing the limited scope of achievement being measured in the SOL assessment, this project will also randomly select students from each triple of schools and administer a grade-specific mathematics interview assessment to determine sampled students’ conceptual understanding and reasoning.  While these interview data will not be entered in the quantitative analysis, they will address the validity of the project’s assessment of student achievement.

Because a teacher has both student achievement and mathematics beliefs scores for several years and because there is a control group, this analysis essentially can control for a teacher’s teaching “talent” as it evaluates the impact of the Mathematics Specialists and their offerings of professional development.  This analysis will reflect any increase in teachers’ professional development through either workshops or engagement with Mathematics Specialists over time.  Because the data also measures the mathematical content and pedagogical knowledge and beliefs of the Mathematics Specialists, as well as their level of activity and sophistication of reflection, this analysis can examine the impact of expertise and longevity of the Mathematics Specialists on students and teachers.   This technique is a multi-level model with cross-classification that incorporates within-classroom and within-teacher variance as well as school-level (Mathematics Specialist) variance.  Statistics will be computed using LISREL.

Because schools and teachers are not randomly assigned populations of students for instruction, and because not all teachers have identical prior professional backgrounds, this analysis will collect student demographic data and teacher certification status data to serve as control variables.