The research involved a several types of data collection and analysis. These included examination of district and state data on principals, schools, and students; interviews with district officials; and review of district expenditure reports. To estimate the effect of the pipeline effort, researchers compared changes in outcomes in pipeline district schools with changes in outcomes in similar schools located in non-pipeline districts in the same state. 

The researchers had four objectives for this study:

1.  Describe what policies, procedures, and practices  districts changed through principal pipelines. Then describe the effect of the initiative on participants.

2.  Estimate the impact of the Principal Pipeline Initiative on student achievement, principal retention, and other outcomes. 

3.  Explore how the outcomes happened.  

4.  Relate these findings to the cost of principal pipelines. 

To achieve these objectives, researchers analyzed the following data:

• District data on principals, schools and students. Participating districts provided data on principals, aspiring principals, schools, and students, including demographic information, data on principal placement, principal exposure to pipeline components, and student-level achievement and other outcome data.
• State data on principals and schools. This included data on student achievement, student average demographics, and other student outcomes at each school.
• Survey data. Policy Studies Associates administered surveys to novice principals in 2013, 2014, and 2015 as part of its implementation evaluation efforts.
• Expenditure reports from PPI districts and technical assistance providers from August 2011 through December 2015. 
• District proposals, budgets, and progress reports. 
• Interviews with district personnel. 
• Data to account for the value of the time that district personnel spent on principal pipeline activities.

To estimate the effect of the pipeline effort, researchers compared changes in outcomes in pipeline district schools with changes in outcomes in similar schools located in non-pipeline districts in the same state. Outcomes as of school year (SY) 2010–2011 served as the basis of comparison or baseline. The principal pipeline effect is the degree to which the changes in outcomes from baseline differed between schools in pipeline districts and the comparison schools.  

Comparison schools were selected not just from one or a few districts but from across the entire state. This provided  a bigger pool of comparison schools with newly placed principals and limited the chance that initiatives in any one non-PPI district would influence the findings. Researchers considered that other districts in the state were engaged in at least some of pipeline activities and may have had initiatives related to specific activities but were not addressing all major pipeline components in a strategic way. The main effect emphasized in this report focused on schools that received a newly placed principal in the school year 2012–2013 or later—a time when the pipeline efforts were in full swing. The researchers also considered whether there was an effect of the pipeline on all schools in pipeline districts.