ORE Planned Research Agenda, 2023-24
Our planned 2023-24 research and evaluation projects are listed here by topic. If a project spans multiple topics, we have organized it by its primary topic.
Expand each project description to learn more about the questions we’re asking.
Descriptive study of participation on District-wide assessments administered in 2022-23 (includes College and Career assessments)
This is a knowledge-building study. The questions we’re asking include:
- Which tests were administered by the Office of Assessment in 2022-23 school year?
- What were the participation rates for these tests?
Descriptive study of 2023-24 PSSA results
This is a knowledge-building study. The questions we’re asking include:
- What percentage of students scored proficient or advanced on the PSSAs this year, and how does it compare to prior years?
- What percentage of students scored proficient or advanced on the PSSAs this year, and how does it compare to prior years, by grade level?
Descriptive study of 2023-24 Keystone results
This is a knowledge-building study. The questions we’re asking include:
- What percentage of students scored proficient or advanced on the Keystone exams this year, and how does it compare to prior years?
- What percentage of students scored proficient or advanced on the Keystone exams this year, and how does it compare to prior years, by grade level?
Descriptive study of Star end-of-year participation, performance, and growth
This is a knowledge-building study. The questions we’re asking include:
- What were the participation rates for the Star tests throughout the 2023-24 school year?
a) Did participation rates differ by grade bands?
b) Did participation rates differ by race/ethnicity?
- What were the patterns in distribution of students across performance levels throughout the 2023-24 school year?
a) How did performance levels on Star tests differ by grade bands?
b) How did performance levels on Star tests differ by race/ethnicity?
- Did students at each grade level demonstrate growth from fall to spring in the 2023-24 school year?
- How did performance this year compare to last year?
Study of the relationship between Star computer-adaptive tests and Star curriculum-based measures
This is a knowledge-building study. The questions we’re asking include:
- What is the relationship between Star CAT and Star CBM performance?
- How accurately do the Star CBMs predict performance on the Star CAT?
Study of the relationship between student performance on Star and PSSA assessments
This is a knowledge-building study. The questions we’re asking include:
- What is the correlation between Star and 2022-23 PSSA performance
- What is the classification accuracy of Star when predicting 2022-23 PSSA performance?
Relationship Between Star Assessments and PSSA Performance, 2022-23
Study of the relationship between student performance on Star and Keystone assessments
This is a knowledge-building study. The questions we’re asking include:
- What is the correlation between Star and 2022-23 Keystone performance
- What is the classification accuracy of Star when predicting 2022-23 Keystone performance?
Description of year-over-year PSSA performance among the cohort of 7th grade students who completed the PSSAs in 2022-23
This is a knowledge-building study. The questions we’re asking include:
- What was the change in the cohort’s PSSA performance level distribution from 2018-19 to 2022-23?
- What percentage of students maintained, improved, or had decreased performance year-over-year ?
Description of 2022-23 PreK Gold Assessment results
This is a knowledge-building study. The questions we’re asking include:
- Do PreK 4 children in SDP programs who meet or exceed expectations on the Teaching Strategies GOLD Literacy and Math domains also perform well on the Star and KEI assessments in kindergarten?
Curriculum and Instruction
Evaluation of the implementation of standards-aligned core instructional materials for math
This evaluation is related to strategic action 3.2 in the Accelerate Philly strategic plan. The questions we’re asking include:
- Year 1: Do teachers, school leaders, and central office support staff believe they have enough materials, training, and support to implement Illustrative Math with integrity/fidelity (as intended)?
Illustrative Math Implementation Study 2023-24: Focus Group Findings
Study of the enrollment mobility of School District of Philadelphia English Learners
This is a knowledge-building study. The questions we’re asking include:
- What are the within-year and between-year mobility patterns of students in high school (public and charter) from SY2021-2022 to SY 2023-2024?
(a) Are English Learner students more mobile than non-English Learner students? Do EL students have distinct within-year or between-year mobility trends?
(b) What are the differences, if any, in Long-Term English Learner (LTEL) students’ mobility patterns, compared to English Learner students?
- Do EL students have distinct mobility trends and reasons for changing schools as indicated by their exit codes?
This is a Philadelphia Education Research Consortium (PERC) project. PERC is currently a partnership between Research for Action and ORE.
Evaluation of Quality Teaching for English Learners (QTEL) professional development, and coaching
This evaluation is required by a grant. The questions we’re asking include:
Part 1 – QTEL Summer Institute:
- To what extent were participants satisfied with the QTEL Summer Institute, and did they perceive their participation as being useful and beneficial to their practice?
- How did participant knowledge of quality instruction for ELs change after participation in the QTEL Institute?
- To what extent do participants agree with statements about their ability to teach ELs and their attitudes and beliefs related to teaching ELs after the QTEL Institute?
Part 2 – Implementation of the QTEL model and related changes in practices:
- How frequently did QTEL participants report using QTEL practices during the school year?
- What did participants report as the primary challenges to implementing the QTEL model?
- To what extent did participants report implementation of the QTEL model changing their approach to teaching ELs?
- In what ways did respondents find that implementing the QTEL model benefited students?
Part 3 – Participant attitudes and beliefs after QTEL implementation:
- To what extent do participants agree with statements about their ability to teach ELs and their attitudes and beliefs related to teaching ELs after the program year?
Implementation and outcomes of the Together is Better program: 2022-2023
This evaluation is required by a grant. The questions we’re asking include:
- To what extent were teachers satisfied with the support provided by SupportEd?
- To what extent did teachers collaborate and engage in practices that result in quality instruction in support of ELs?
- What are participants’ beliefs about teaching EL students and their school’s approach to teaching EL students?
- What did participants perceive to be the benefits of and challenges to co-teaching, and what supports are needed for co-teaching in general?
Case studies of 19 SDP schools receiving SNAP-Ed-funded programming that identify barriers and facilitators to sustainable Policy, Systems, and Environment changes in schools
This study is grant-funded.
- How do SNAP-Ed implementers and schools understand sustainability and SNAP-Ed’s PSE initiatives?
- What factors facilitate implementing sustainable PSE initiatives?
- How do SNAP-Ed implementers and schools understand their roles in the partnership?
High School Graduation and Beyond
Analysis of student experiences that are predictive of passing the Biology, English, and Algebra Keystone exams
This is a knowledge-building study. The question we’re asking is:
- What data predicts that an SDP student will score Proficient or Advanced on each of the three Keystones?
Descriptive study of pathways of 2022-23 graduates
This is a knowledge-building study. The questions we’re asking include:
- For students graduating in 2022-23, which Act 158 pathways did they complete?
- How did pathway completion patterns differ by student demographic groups?
Descriptive information from the 2022-23 Senior Exit Survey
This is a knowledge-building study. The questions we’re asking include:
- What post-secondary education plans do students have within one year of graduation?
- What support systems did students have to help them navigate career and post-secondary education options?
2022-23 Senior Exit Survey: District-Level Report
Descriptive information from the 2024 graduate follow-up survey
This is a knowledge-building study. The question we’re asking is:
- What are the post-graduation experiences of recent SDP graduates (classes of 2019-20, 2020-21, and 2021-22)?
Results from the 2024 Graduate Follow-up Survey
Descriptive information from the 2022-23 Career & Technical Education (CTE) follow-up survey
This is a knowledge-building study. The question we’re asking is:
- What do recent CTE graduates report about their post-graduation experiences, and the extent to which those experiences are related to their CTE pathway?
Descriptive information from the 2021-22 and 2022-23 Senior Exit Survey summer melt data
This is a knowledge-building study. The questions we’re asking include:
- What is the percentage of college-intending seniors who do not enroll in college their first fall after graduation?
- Did summer melt rates vary by student demographics?
- Did summer melt rates differ from the previous year?
- What are some high school experiences associated with a reduction in the odds of summer melt?
Descriptive information from the 2023-24 Senior Exit Survey
This is a knowledge-building study. The questions we’re asking include:
- What post-secondary education plans do students have within one year of graduation?
- What support systems did students have to help them navigate career and post-secondary education options?
Evaluation of The Urban Technology Project 2023-24 Americorps Digital Service Fellows program
This evaluation is required by a grant. The questions we’re asking include:
- To what extent were the mentoring and tutoring sessions implemented as intended?
- How many students were served mentoring/tutoring through Tech Club during the program year?
- How frequently did Tech Club mentoring/tutoring sessions occur throughout the program year?
- To what extent are participating Digital Literacy Teachers and Digital Service Fellows satisfied with the program?
- To what extent are Digital Literacy teachers satisfied with their Digital Service Fellows and the Urban Technology Project program management, support, and overall programming throughout the program year?
- To what extent are Digital Service Fellows satisfied with the program management, support, and overall programming throughout the program year?
- To what extent did Digital Service Fellows’ technical, professional, and communication skills improve? How many IT certifications did they obtain?
- To what extent do Digital Literacy Teachers observe improvement in student engagement in Digital Literacy class and social emotional skills from baseline to follow-up for (1) all students in Digital Literacy class and (2) students in Tech Club?
- To what extent do Digital Literacy Teachers’ observe improvement in student engagement in Digital Literacy Class for (1) all students in Digital Literacy Class and (2) students participating in Tech Club mentoring/tutoring from mid- to end of year?
- To what extent do Digital Literacy Teachers’ observe changes in students’ social-emotional skills for (1) all students in Digital Literacy Class and (2) students participating in Tech Club mentoring/tutoring from mid- to end of year?
- To what extent do students participating in Tech Club tutoring/mentoring exhibit increased engagement in Digital Literacy Class and social-emotional skills from pre- to post-program?
- How many students who participate in Tech Club mentoring or tutoring attend at least 90% of instructional days?
Descriptive study of students perspectives on school climate and instructional environment using 2021-22 PSES data and cognitive group interviews
This is a knowledge-building study. The questions we’re asking include:
- What are student responses on the key school climate issues covered in the Philly School Experience Survey (PSES), including school safety, relationships with staff and peers, and physical school environments?
- How do student experiences in school safety and climate differ by school?
- To what extent do qualitative findings from student focus groups corroborate or offer further insights on findings from quantitative data from the PSES?
Understanding and Measuring Perceptions of School Climate and Safety: Insights from the Philly School Experience Survey
Evaluation of Schoolyard Redesign Project
This evaluation is required by a grant. The questions we’re asking include:
- What student activities are happening on the playground during school hours and how have these activities varied before and after redesign
- What school staff activities are happening on the playground during school hours and how have these activities varied before and after redesign?
- Are there any changes to school climate or achievement after playground installation?
Redesigning Schoolyards at Two Schools: Summary of Interviews and Observations
Descriptive study of student perceptions of Youth Court
This is a knowledge-building study. The questions we’re asking include:
- What are student perceptions of potential benefits of Youth Court in their school
- Are there ways in which students think Youth Court could be improved at their school?
- Is there a relationship between schools implementing a YC and the rate of suspensions?
Student Outcomes and Perspectives on Youth Court in the School District of Philadelphia
Summary of Philly School Experience Survey 2022-23 results
This is a summary of survey results. The questions we’re asking include:
- What were the main system-level trends in results for PSES core topics in 2022-23?
- What do PSES results tell us about school conditions related to the Accelerate Philly priority areas?
2022-23 Philly School Experience Survey Results
Descriptive study of Philly School Experience Survey 2022-23 support staff survey results
This is a knowledge-building study. The questions we’re asking include:
- What are District support staff perceptions of school climate, school leadership, equity and inclusion, and their professional capacity?
- To what extent do these perceptions vary by school?
Study of Education for Children and Youth Experiencing Homelessness (ECYEH) grant funded activities during 2022-23 school year
This study is required by a grant. The questions we’re asking include:
- During the 2022-23 school year, did ECYEH approaches result in the identification of school-age students as experiencing homelessness at some point during the year?
- How many students did ECYEH support in terms of the following services: enrollment assistance, uniforms, school supplies, and transportation assistance?
- Was ECYEH able to implement the TEEN program for students in addition to professional development opportunities for educators, school personnel, and others?
- How do students experiencing homelessness compare to other District students on measures of attendance and achievement?
Study of graduation rate trends over time
This is a knowledge-building study. The questions we’re asking include:
- What are the graduation rates for the 2019-20 9th grade cohort for different public school sectors in Philadelphia?
- What are the longitudinal trends in graduation rates?
2022-23 Four-Year High School Graduation Rates in Philadelphia
Descriptive information from the October 2023 enrollment snapshot
This is a knowledge-building study. The questions we’re asking include:
- What are the enrollment numbers for different public school sectors in Philadelphia in 2022-23 school year?
- What are the year-over-year and long-term trends in enrollment?
Philadelphia Public School Enrollment, 2023-24
Descriptive information from 2023-24 School Selection
This is a knowledge-building study of a major district initiative. The question we’re asking is:
- What were the patterns in student qualifications, application submission, admission offers received, and student acceptances of admission offers?
Fiscal Year 2025 Budget Priorities Survey results
This is a stakeholder engagement effort designed to inform budget priorities for Fiscal Year 2025. The questions we’re asking include:
- What issues and resources do students, staff, families and other stakeholders view as the highest priorities for the SDP?
- How do stakeholders perceive the budget development process?
- What are stakeholders’ priorities for federal title grant investments?
Fiscal Year 2025 Budget Priorities Survey Results
Study of the relationship between student performance on 2023-24 Star and Keystone assessments
This is a knowledge-building study. The questions we’re asking include:
- What is the correlation between Star and Keystone performance?
- How accurately did Star predict performance on the Keystone?
Study of the relationship between student performance on 2023-24 Star and PSSA assessments
This is a knowledge-building study. The questions we’re asking include:
- What is the correlation between Star and PSSA performance?
- How accurately did Star predict performance on the PSSA?
Study of the Relationship between the Philly School Experience Survey and Measures of School Improvement
This is a knowledge-building study. The questions we’re asking include:
- What is the relationship between the six core PSES topics and measures of student learning, attendance, dropout and staff retention?
- To what extent are school scores on the six core topics associated with improvement in measures of school improvement over time?
Last modified: November 20, 2024