This report presents findings from a 2018 survey of principal supervisors in the six Principal Supervisor Initiative districts and 48 other large urban districts. In total, 343 principal supervisors responded to the survey, including 50 in the six initiative districts (a 96 percent response rate) and 293 in the 48 other urban districts surveyed (a 64 percent response rate). The districts were all members of the Council of the Great City Schools, a coalition of the nation’s largest urban public school systems.

Researchers adapted the survey instrument from those used in previous years of the study of the Principal Supervisor Initiative–known by its initials, PSI–(Goldring et al. 2018 and 2020) and an earlier study of principal supervision by the Council of the Great City Schools (Casserly et al. 2013).  The researchers reviewed, piloted, and revised the original initiative surveys to ensure their face validity and clarity.

The survey asked supervisors about the following topics:

• Their professional background and demographics
• Their span of control (the number of principals they oversaw) and how these principals were assigned to them
• How they allocated their time
• Their practices and topics of discussion with principals
• Their professional development experiences
• Their perceptions of the central office
• Their perceptions of their own evaluations
• Their feedback concerning the supports supervisors need to be effective

The study team sent the web-based surveys to all principal supervisors (staff directly overseeing one or more principals) in 63 participating Council of the Great City Schools member districts, including the 6 PSI districts. Seven other districts that were members of the council did not provide a list of their principal supervisors to the study team and were not included in the survey. Researchers administered the survey between April and September 2018.

In 4 of the original 63 districts that received surveys, no supervisors responded. Researchers also excluded five other districts from the analysis sample. Two of these districts—District of Columbia Public Schools and Tulsa Public Schools—were engaged in separate but concurrent principal supervision changes with support from The Wallace Foundation. The other three—Charlotte-Mecklenburg Schools, Denver Public Schools, and Hillsborough County Public Schools—participated in the Principal Pipeline Initiative, a separate Wallace Foundation effort that also had implications for the principal supervisor role (the other three pipeline initiative districts were not members of the Council of the Great City Schools). Excluding these districts allowed the researchers to compare supervisors in PSI districts with supervisors in urban districts that were not participating in intensive, targeted work that included some of the PSI’s approaches and components. Although the other urban districts in the sample may also have made changes to their principal supervisor role during the same period, these districts likely implemented such changes on their own, without external support.

The final sample for this analysis included 343 confirmed principal supervisors across 54 districts (a 67 percent response rate). This included 50 supervisors in the 6 PSI districts (a 96 percent response rate) and 293 supervisors from 48 other urban districts (a 64 percent response rate). The lower response rates in the other urban districts means that the responses in those districts might not be fully representative of the views of all the principal supervisors in those districts. This could lead to overstating or understanding  the differences between supervisors in PSI and other urban districts. However, the researchers are unable to determine the size or direction of any possible bias.

Principal supervisors in PSI and other urban districts had similar professional backgrounds. The average principal supervisor in the PSI districts had spent about five years in the role, compared with six years for supervisors in other urban districts. In both PSI and other urban districts, most principal supervisors worked as principals immediately before becoming supervisors. However, PSI districts were more likely than other urban districts to hire their supervisors from principal positions in the district.

The report compares  the survey responses of principal supervisors in PSI and other urban districts. All analyses give equal weight to each principal supervisor, rather than averaging their responses and giving equal weight to each district. Researchers assessed the statistical significance of differences in means between PSI and other urban districts using unpaired t-tests. In general, in the discussion of the findings, the report focuses on differences between PSI and other urban districts that are both substantively meaningful and statistically significant at the 5 percent level. Researchers assess whether a difference is substantively meaningful based on context from their analysis of districts’ implementation of the PSI across the four years of the initiative (Goldring et al. 2018 and 2020). In discussing differences that are substantively meaningful but not statistically significant, researchers note the lack of statistical significance in the text.