ORE Planned Research Agenda, 2022-23

Our planned 2022-23 research and evaluation projects are listed here by topic. If a project spans multiple topics, we have organized it by its primary topic.

Expand each project description to learn more about the questions we’re asking.

Assessments

Descriptive study of participation on District-wide assessments administered in 2021-22 (includes College and Career assessments)

This is a knowledge-building study. The questions we’re asking include:

  • Which tests were administered by the Office of Assessment in 2021-22 school year?
  • What were the participation rates for these tests?

View the published brief:
A Summary of District-Wide Assessments Administered to SDP Students During the 2021-22 School Year

Descriptive study of 2022-23 PSSA results

This is a knowledge-building study. The questions we’re asking include:

  • What percentage of students scored proficient or advanced on the PSSAs this year, and how does it compare to prior years?
  • What percentage of students scored proficient or advanced on the PSSAs this year, and how does it compare to prior years, by grade level?

View the published brief:
PSSA Performance Trends in SDP: 2018-19 to 2022-23

Descriptive study of 2022-23 Keystone results

This is a knowledge-building study. The questions we’re asking include:

  • What percentage of students scored proficient or advanced on the Keystone exams this year, and how does it compare to prior years?
  • What percentage of students scored proficient or advanced on the Keystone exams this year, and how does it compare to prior years, by grade level?

View the published slide deck here:
Keystone Performance Trends in SDP: 2018-19 to 2022-23

Descriptive study of Star end-of-year participation and performance

This is a knowledge-building study of a major district initiative.  The questions we’re asking include:

  • What were the participation patterns in the three Star assessment windows throughout the year?
  • What percentage of students performed At/Above Benchmark in the three Star assessment windows?

View the published report:
End of Year Analysis of Participation and Performance on the Star Assessments at the School District of Philadelphia: 2022-23

Study of the relationship between student performance on Star and PSSA assessments

This is a knowledge-building study. The questions we’re asking include:

  • What is the correlation between Star and 2021-22 PSSA performance
  • What is the classification accuracy of Star when predicting 2021-22 PSSA performance?

View the published report:
Analysis of the Relationship between 2021-22 Star Assessment and PSSA Assessment Performance

Study of the relationship between student performance on Star and Keystone assessments

This is a knowledge-building study. The questions we’re asking include:

  • What is the correlation between Star and 2021-22 Keystone performance?
  • What is the classification accuracy of Star when predicting 2021-22 Keystone performance?

View the published report:
Analysis of the Relationship between 2021-22 Star Assessment and Keystone Assessment Performance

English Learners

Descriptive study of the diversity of SDP English Learners transitioning to high school

This is a knowledge-building study. The question we’re asking is:

  • What is the diversity of home languages spoken, countries and regions of birth, and duration of EL status across the population of English Learners in the years leading up to their transition to high school?

This is a Philadelphia Education Research Consortium (PERC) project. PERC is currently a partnership between Research for Action and ORE.

View the published report: 
Exploring the Diversity and Supports for School District of Philadelphia English Learners as they Prepare to Transition to High School

Descriptive study of performance on the Star computer-adaptive tests by ACCESS levels

This is a knowledge-building study. The questions we’re asking include:

  • Who took the ACCESS English proficiency test?
  • What were the patterns in ACCESS and Star performance for tested SDP students?
  • What did Star performance look like by ACCESS level?

View the published report:
English Learner Performance on the 2021-22 ACCESS, Star Reading, and Star Math Assessments

Evaluation of Quality Teaching for English Learners (QTEL) Summer Institute, professional development, and coaching

This evaluation is required by a grant. The questions we’re asking include:

  • To what extent were participants satisfied with the QTEL Institute, and did they perceive participation as being useful and beneficial to their practice?
  • How did participant knowledge of best practices and attitudes about teaching ELs change as a result of the QTEL Institute?
  • Did participant perceptions of their knowledge of best practices and attitudes about teaching ELs differ based on participant subgroup?

View the published report:
Quality Teaching for English Learners (QTEL): 2022-23 Implementation Report

Family Engagement

Final evaluation report of Academic Parent Teacher Teams (APTT)

This evaluation is required by a grant. The questions we’re asking include:

  • To what extent did families participate in, and were satisfied with, APTT meetings?
  • How complete was implementation of the APTT model at schools where ORE conducted observations?
Read the APTT final evaluation report published in September 2022

Health and Nutrition

Descriptive information from Eat Right Philly Alternate School Health Index (SHI) survey of health and PE teachers

This study is required by a grant. The questions we’re asking include:

  • What common challenges do schools face in implementing health and wellness activities?
  • To what extent do school responses on the SHI indicate alignment with the District Wellness Policy?
  • In which areas do schools need the most support in meeting guidelines?

This report was published internally for use by the program office.

Descriptive information from 2021-22 District-Wide Survey food insecurity data

This is a knowledge-building study.  The questions we’re asking include:

  • To what extent do SDP students and families report experiencing food insecurity?
  • To what extend do SDP principals report that food insecurity is a challenge to student learning?

View the published brief:
Household Food Insecurity in the School District of Philadelphia: An Analysis of District-Wide Survey Results, 2021-22

Study of 2021 Youth Risk Behavior Survey (YRBS) mental health report (Philadelphia sample)

This study is required by a grant. The questions we’re asking include:

  • What are the year-to-year trends in suicidal ideation among Philadelphia youth, overall and across student subgroups?
  • Are experiences of Adverse Childhood Events (ACEs) associated with suicidal thoughts and attempts?
  • What protective factors are related to lower rates of suicidal behavior?

This project was removed from the 2022-23 agenda.

Study of 2019-20 and 2021-22 school breakfast models, participation rates, and responses to District-Wide Survey questions

This is a knowledge-building study.  The question we’re asking is:

  • How did school breakfast models and breakfast participation rates change from 2019-2020 to 2021-22?

The plan for this study was modified:

  • Find meal participation rates (including breakfast rates) on our Open Data website
  • We are working with a Temple researcher on an NIH-funded five-year study to identify approaches that are working best for schools and to work with Food Services to strengthen school meal implementation. Learn more here.

High School Graduation and Beyond

Descriptive study of graduation pathways and student Keystone performance

This is a knowledge-building study.  The question we’re asking is:

  • What percent of high school students are on track for each graduation pathway based on 2022-23 and earlier Keystones performance? (Tentative)

This is a Philadelphia Education Research Consortium (PERC) project. PERC is currently a partnership between Research for Action and ORE.

View the published report:
Changing the Finish Line: Implications of new graduation requirements in the School District of Philadelphia

Descriptive information from the 2021-22 Senior Exit Survey

This is a knowledge-building study.  The questions we’re asking include:

  • What post-secondary education plans do students have within one year of graduation?
  • Did disruptions due to the COVID-19 pandemic impact students’ post-secondary education plans?
  • What support systems did students have to help them navigate career and post-secondary education options?

View the published slide deck:
2021-22 Senior Exit Survey: District-Level Report

Descriptive information from the 2021-22 Career & Technical Education (CTE) follow-up survey

This is a knowledge-building study.  The question we’re asking is:

  • What do recent CTE graduates report about their post-graduation experiences, and the extent to which they are related to their CTE pathway?

View the published slide deck:
Results from the Spring 2022 CTE Graduate Follow-up Survey

Evaluation of Power Skills Academy

This evaluation is a grant requirement.  The questions we’re asking include:

  • To what extent was the Power Skills Academy implemented as intended?
  • To what extent did the program support high school students in meeting the PA Department of Education’s new graduation requirements?
  • What were the participation and completion rates for the program?

This project was removed from the 2022-23 agenda.

Remediation and Enrichment

Evaluation of Science Olympiad

This evaluation is a grant requirement.  The questions we’re asking include:

  • To what extent did students participate in the program?
  • What implementation barriers and facilitators were identified by coaches?
  • What feedback did students have about the program and their attitudes about STEM?

This report was published internally for use by the program office.

School Climate

Descriptive study of monthly student attendance trends

This is a knowledge-building study.  The question we’re asking is:

  • What were the trends in 90%+ monthly student attendance in 2021-22 and 2022-23?

View the published brief (2021-22):
Student Attendance Patterns in Philadelphia, 2017-18 to 2021-22

System Trends

Descriptive information from the October 2022 enrollment snapshot

This is a knowledge-building study.  The questions we’re asking include:

  • What are the enrollment numbers for different public school sectors in Philadelphia in 2022-23 school year?
  • What are the year-over-year and long-term trends in enrollment?

View the published brief:
Philadelphia Public School Enrollment, 2022-23

Descriptive information from the 2021-22 School Progress Report on Education and Equity (SPREE)

This is a knowledge-building study.  The questions we’re asking include:

  • What is SPREE and what metrics does it include?
  • What were the patterns of school performance in 2021-22 SPREE reports?

View the Research Roundup webinar and slide deck:

The School Progress Report on Education and Equity (SPREE)

Descriptive information from 2021-22 School Selection (final fall 2022 enrollment)

This is a knowledge-building study of a major district initiative.  The questions we’re asking include:

  • What are the enrollment patterns at criteria-based high schools over the last four years; overall and by student groups?
  • What were the enrollment patterns at the schools that used zip code priorities in 2021-22?

View the published report:
School Selection: Enrollment of First-Time 9th Graders at District High Schools, 2019-20 through 2022-23

Descriptive information from 2022-23 School Selection

This is a knowledge-building study of a major district initiative.  The question we’re asking is:

  • What were the patterns in student qualifications, application submission, admission offers received, and student acceptances of admission offers?

View the published report:
A Summary of the 2022-23 School Selection Process: Qualifications, Applications, Admission Offers, and Student Responses

Study of relationships between School Selection admission criteria and 9th grade On-Track rates

This is a knowledge-building study.  The questions we’re asking include:

  • To what degree do data points used in past cycles of School Selection predict 9th grade On-Track status at criteria-based schools?
  • Are there specific data points, combinations of data points, and/or qualification thresholds that best predict student success at criteria-based schools, or subsets of criteria-based schools?

This project was removed from the 2022-23 agenda.

Study of graduation rate trends over time

This is a knowledge-building study.  The questions we’re asking include:

  • What are the graduation rates for the 2018-19 9th grade cohort for different public school sectors in Philadelphia?
  • What are the longitudinal trends in graduation rates?

View the published brief:
2021-22 Four-Year High School Graduation Rates in Philadelphia

Study of Education for Children and Youth Experiencing Homelessness (ECYEH) grant funded activities during 2021-22 school year

This study is required by a grant. The questions we’re asking include:

  • During the 2021-22 school year, did ECYEH approaches result in the identification of school-age students as experiencing homelessness at some point during the year?
  • How many students did ECYEH support in terms of the following services: enrollment assistance, uniforms, school supplies, and transportation assistance?
  • Was ECYEH able to implement the TEEN program for students in addition to professional development opportunities for educators, school personnel, and others?
  • How do students experiencing homeless compare to other District students on measures of attendance and achievement?

View the published brief:
Education of Children and Youth Experiencing Homelessness (Analysis of 2021-22 Data)

2023-24 and Beyond

Here are some projects we are looking forward to completing in future years!

Study of the relationship between student grades and Keystone performance

This is a knowledge-building study.  The questions we’re interested in include:

  • What is the relationship between Keystone performance and student grades for Algebra I, Biology, and Literature?
  • Does how soon after course completion the Keystone test is taken have an effect on the strength of the relationship?

Study of the relationship between Star computer-adaptive tests (CATs) and Star curriculum-based measures (CBMs)

This is a knowledge-building study.  The questions we’re interested in include:

  • What is the relationship between Star CATs and Star CBMs?
  • How accurately do the CBMs predict performance on the Star CATs?