When members of the United Community Schools staff and immediate stakeholders learned about the plan for a multiyear evaluation, anticipation filled the air. We have always emphasized that the impact of our initiative goes beyond mere numbers, and our school teams wondered if an evaluation would truly capture the incredible story that unfolds in their schools every day.

It’s like planning a meticulously organized party for which you hope that the guests not only show up but also appreciate all the hard work that went into creating a memorable event. Because we were well aware of the investments made to ensure a remarkable school year for our students, we found ourselves hoping that the tremendous effort, programming, partnerships and results would be acknowledged.

UCS embarked on its evaluation journey long before the COVID pandemic hit. We retained the services of Metis Associates, a New York–based research firm with access to official city education data. Our goal was to conduct a comprehensive impact evaluation that would accurately assess the changes directly attributable to specific interventions, programs, policies or models.

Metis conducted a comparative study analyzing the data for “students who attended any UCS school for all three years of the study period against students across the city who neither attended a UCS nor a non-UCS community school for all three years of the study period.” Metis outlined its findings in a report.

The findings from our evaluation tell a compelling story of positive outcomes. We have witnessed significant improvements in attendance, credit accumulation, and, notably, better-than-anticipated results for English language and multilingual learners on the NYSESLAT (New York State English as a Second Language Achievement Test). The journey that led us to these results is a story in itself.

Of course, when we embarked on this evaluation journey, no one could have predicted the pandemic that would force both our teams and our evaluators to adapt quickly to a new reality. The initial evaluation covering the school years 2019–20 and 2020–21 presented challenges in terms of data collection and fidelity to the UCS model during the citywide lockdown and subsequent hybrid schedules.

However, despite serving a student population requiring additional support, including a higher proportion of economically disadvantaged students, English language learners and students with disabilities, our outcome data has shown remarkable progress.

Here are some key highlights:

– UCS students attended approximately three more days than similarly situated non-community school students in a 182-day school year.

– On average, UCS students achieved a score 2.5 points higher – a statistically significant difference – on the NYSESLAT, a standardized test for English language learners, than similarly situated non-community school students.

– On average, UCS students accumulated half a credit more than similarly situated non-community school students. Moreover, UCS students earned half a credit more annually than what is expected of high school students.

In the words of Christine Schuch, Associate Executive Director of UCS, “Amidst the challenges presented by the pandemic and the reopening of schools, our dedication to evaluating our efforts remained unwavering. The evaluation plays a crucial role informing efforts towards minimizing learning loss, ensuring consistent student attendance, and providing steadfast support during one of the most demanding periods in our history.”

Our UCS story and evaluation continue to unfold and reach new heights. Recently, we had the honor of presenting our work and findings at the Coalition for Community Schools conference on June 7, 2023. The title of our presentation, “A Rigorous Evaluation of United Community Schools: Best Practices for Implementation,” drew a standing-room-only crowd of interested participants. As we progress, we anticipate the initial results of each school’s implementation of the UCS model through the fidelity tool developed by Metis. Donna Wilkens, the project leader at Metis, captured our eagerness perfectly, stating, “What’s so exciting is that we’re finally able to test the theory of change and examine real outcomes.” These developments mark a significant milestone in our journey of understanding and improving the impact of our UCS model.

Our commitment to reflection, data-informed decision-making, and openness to change adds strength to our UCS model. We eagerly look forward to the continued learning as we progress to the next phases of our multiyear study.