In early February, the Office of Evaluation, Research, and Accountability (ERA) opened the District-Wide Survey to students, teachers, principals, parents and guardians, and, for the first time, school-based support staff. We receive more than 130,000 District-Wide Survey responses connected to both District and Charter schools every year. Those responses help to inform school staff and District leadership about what’s already working and what needs to improve in areas related to climate, instruction, leadership, professional capacity, and parent/guardian community ties. But why did we decide on those five areas? Where do our questions come from? And what, exactly, do we do with the data?

1. The birth of today’s District-Wide survey

In 2014, with support from a grant from the Institute of Education Sciences, ORE partnered with staff from the University of Pennsylvania’s Graduate School of Education to improve the District’s survey program. Drawing on work from the University of Chicago Consortium on School Research, we collaborated with school stakeholders to write questions aligned to five key “constructs,” or topics. Each one shares a general line of inquiry, summarized below.

• Climate: Does the school environment feel safe, supportive, and conducive to learning?
• Instruction: Are students and teachers deeply engaged in challenging work?
• Leadership: Do school leaders successfully communicate and implement their school mission, and do they work effectively with the community?
• Professional capacity: Do teachers and principals feel supported professionally in ways that are appropriate to their specific needs and roles?
• Parent/guardian community ties: How are schools engaging families, and do parents and guardians feel well-informed and welcome?

Each construct includes a number of individual questions. ORE periodically refines the survey to ensure that each question and construct provides reliable, valid information. We assessed and improved the survey in 2014 and 2016: this report contains lots more technical details about the survey questions and constructs.

2. We ask about a lot of things–but we don’t ask everybody everything

Students, parents/guardians, teachers, school-based staff, and principals all have different perspectives, needs, and areas of expertise, and while there are areas of overlap, we only include relevant questions on each survey. We don’t ask parents about teacher morale, for example, just as we don’t ask students about staffing, school budgets, or their principal’s level of autonomy. We’ve posted a complete list of questions for each respondent group here.

We do occasionally add or remove questions, mostly to keep the survey relevant or to improve areas of inquiry related to the five established constructs. In 2018-19, for example, we added school leadership questions to the parent survey based on feedback from respondents, and in 2019-20 we added clarifying questions about student safety.

3. What do we do with the data?

Because the surveys are confidential, we never follow up with any individual person about their survey responses. (Parents seeking to address concerns or resolve specific issues should visit FACE’s Concern Resolution website.) In addition, students, parents/guardians, teachers, school-based staff, and principals don’t have to answer any questions they don’t want to, and we don’t use the data to evaluate families or report on individual responses. Ever.

We do, however, follow up with various offices to help them understand the trends and patterns in the data and use it to plan next steps. For example, each year we meet with the Office of Food Services to look at how students have responded to questions about school lunch. We also encourage principals to use survey results to identify priority areas they’ll focus on in their annual school plans. Our website includes a series of vignettes that describe how other District offices and community organizations have used the data to improve services or refine their efforts–check it out here.

School-level survey results are especially useful because schools are the primary point of contact between the District and most families, school staff, and community members. We develop and release various materials to support principals (like this response rate tips and strategies sheet and this DWS data guide for District leaders), and we also produce short school-specific summary reports that provide the public with an overview of survey results for each school.

4. Response rates: Everyone’s perspective is important!

We often refer to the response rates for each respondent group, mostly in the context of encouraging students, parents and guardians, teachers, school-based staff, and principals to take the survey. We calculate response rates by counting how many people from each group completed the survey, then dividing that total by the number of people who were eligible to take it. In 2017-18, for example, 80,101 students took the survey. To calculate the 2017-18 District-wide student response rate, we divided 80,101 by the total number of students enrolled in District and Charter schools on May 31 of that year to arrive at a response rate of 54%. We also calculate response rates for each individual school, as these can vary substantially from the overall District rate. In 2018-19, for example, school-level student, teacher, and principal response rates ranged from 0-100%, and parent/guardian response rates ranged from 0-82%.

Response rates are important because, in general, higher response rates mean more accurate results. We carefully plan the timing and duration of the administration windows for each of the surveys to optimize participant access. Increasing response rates across all schools in the District will lead to higher-quality results that reflect the diversity of perspectives we’re aiming to capture.

5. How can I help, and where can I learn more?

If you are a student, parent or guardian, teacher, or principal (or, in 2019-20, if you work in a full-time school-based support role), the best way you can support the District-Wide Survey is by taking it yourself! This year, the survey is open from February 3 – May 8. You can also visit our website to read answers to frequently asked questions, review the survey items, explore survey data, and more. If you can’t find the answer to one of your questions on our site, you can also e-mail the survey team directly: schoolsurveys@philasd.org.