The findings presented in this report are based on data collected by independent research teams at Vanderbilt University and Mathematica and include interviews and surveys. This report focuses on the experiences of the six districts in implementing the Principal Supervisor Initiative (PSI) during the first three years, from the 2014–2015 to 2016–2017 school years. 

Data for the report come from site visits and surveys conducted during the 2015–2016 and 2016–2017 school years . Researchers began data collection after the districts had already begun to implement the initiative. 

Interviews.  The Vanderbilt team visited each of the six PSI districts twice: once in the summer or fall 2015 and again in fall 2016. The team members conducted the first site visit about a year after districts had begun the initiative. Thus, they do not have baseline data collected before the start of the PSI, although data collection began early enough that participants often could draw comparisons between their current work and work before the start of the initiative. Each site visit lasted two to four days. During each site visit, the study team conducted semi-structured interviews with central office staff, principal supervisors, and principals. The study team developed protocols for these interviews based on a review of existing research on principal supervision and designed the protocols to cover each of the components of the PSI. The team also collected artifacts, such as training agendas and supervisor job descriptions. The team  recorded and transcribed all interviews for analysis. In total, the research team conducted 219 interviews across the six PSI districts 

• Central office staff. The study team interviewed two central office staff members in each district during each site visit. Typically, they  interviewed the district’s project director for the PSI (the main point of contact for the initiative) and the supervisor of the principal supervisors or someone else in the central office who was closely involved in the initiative.  
• Principal supervisors. Researchers interviewed up to six principal supervisors in each PSI district during each site visit. In districts with six or fewer supervisors (Cleveland, Des Moines, and Minneapolis), they  interviewed all supervisors. In districts with more than six supervisors (Baltimore, Broward, and Long Beach), they selected a sample of six to interview. In these districts, they  first stratified supervisors according to the type of schools supervised (for example, by grade level) to ensure they included supervisors from all types of schools, and then randomly sampled supervisors from these strata. For the second round of site visits, they  re-interviewed previously sampled supervisors. If a previously sampled supervisor was no longer employed in the position, they  randomly selected a replacement from the available pool of supervisors.
• Principals. In each district, researchers selected a random sample of 10 principals, stratified by supervisor, to ensure that as many supervisors as possible were represented. They  excluded principals who were new to the district in the year of the interview or were principals of special schools (for example, alternative night schools). In addition, they gathered information on principals’ school grade levels and school performance. Researchers  used this information to ensure that the sample of principals included a range of school grade and performance levels. In the second site visit, they re-interviewed 5 of the original principal participants in each district to capture their perspectives over time. In addition, they  selected a sample of 5 new principals, following the procedures described earlier.

Surveys. Researchers conducted two rounds of principal and supervisor surveys for this report. They administered the first round of surveys from November 2015 to February 2016 and the second round from March to June 2017. During both rounds, they sent online surveys to all principals and supervisors in the six PSI districts. They developed the surveys based on preliminary analysis of interview data, a review of current supervisor competency standards, and prior research on principal supervisors. They reviewed, piloted, and revised the survey items to ensure their validity. Response rates for both principals and supervisors were very high across districts.

Analysis. Researchers  coded the interview data using an iterative coding process, focusing on each component of the PSI as well as how each component unfolded and was experienced in each district. The data analyses focused on districts’ accomplishments over the early years of the initiative, the challenges districts experienced along the way, and districts’ planned approaches for moving forward. They used reports and documents, such as training agendas, to supplement the interview and survey data. Descriptive survey results are integrated with the qualitative data. They report unweighted survey responses, giving equal weight to the perceptions and experiences of respondents across the initiative.