The following is written testimony Josh Boots provided to the DC Public Charter School Board regarding the adaptation of its new draft Performance Management Framework called Aspire:

Thank you for the opportunity to submit written testimony regarding the new draft Performance Management Framework (PMF). I serve as Executive Director of EmpowerK12, a mission-driven education data nonprofit here in DC, and as a board member at Center City PCS, a network of six PK-8 schools. I provide this testimony solely as a concerned DC citizen with deep insight into charter school operations, what makes schools effective, and DC school accountability systems, including as a member of the original PMF development task force in 2010. The proposed draft PMF for PK-8 schools is largely duplicative of the state’s accountability system, which will generate confusion among our stakeholders and families while also creating an extra compliance burden on charter school staff. I offer 3 recommendations for moving forward at the end.

For the new DC school report card accountability system, Center City’s data manager must validate that OSSE has correctly identified the student universe and appropriately calculated up to 148 metrics per school that make up the overall score. PCSB’s draft PMF will not just duplicate that amount of work; it will complicate and confound the accountability compliance process by using slightly different business rules for the universe of students and student groups included as well as how metric performance is calculated by using different floors and targets. With the two additional school specific measures (I fully support their inclusion as a component of a new PMF) and 8 student groups for each, the grand total number of PMF metrics Center City’s data manager must verify for each school will be 164 metrics.

Let’s say the data manager takes 5 minutes per metric to confirm whether the list of students PCSB included in the calculation is accurate and another 5 minutes to verify the underlying numbers for each student and aggregated metric value are correct. This means they spend 10 minutes per metric for a grand total of 28 hours just to validate the 164 metrics in the proposed PMF for one school; this time estimate for PMF validation assumes each metric’s universe and calculation match perfectly. When metrics differ from expected values, a time multiplier goes into effect for figuring out why numbers do not match, submitting and responding to PCSB tickets, and then re-validating everything again when tickets are closed.

For Center City’s six campuses, the data manager might spend up to four entire work weeks validating PCSB PMF data, in addition to the four weeks required to validate OSSE’s slightly different accountability system metrics. This leaves their staff with substantially less time to support school leaders and teachers with using data to improve.

Meanwhile, DCPS’s data team, lacking this duplicative accountability tool verification requirement, can spend an extra month doing data work that moves the needle forward for their schools and students. It might be more than one extra month of data support compared with the charter sector average because while a minimum of 69 charter LEA data managers are needed to validate PMF and OSSE data, DCPS likely only needs a few data folks to validate half the amount of accountability metrics for all their schools, allowing their team to spend even more time analyzing and coaching educators about student data for improvement purposes.

The proposed PMF does not represent the only source of compliance burden charter schools face. With more regular attendance, discipline, and special education collections, the number of metrics and frequency by which they must be validated has increased nearly every year, often with changes in calculation methodology from year-to-year and limited automation in the process deployed by PCSB to improve its efficiency, validity, and reliability. The percentage of errors in the validation process attributable to PCSB has not decreased over time because PCSB has not invested enough of its budget in a modern data technology platform to make compliance activities more accurate and automatic, reducing the validation time burden placed on schools.

This decision has manifested in a multi-fold increase in charter schools’ time spent on compliance back-and forth activities with the authorizer rather than on improvement efforts in their schools and classrooms. Changes in math and reading growth data suggest this approach may have negatively impacted the entire sector’s performance over time. The charts below show the average median growth percentile for charter schools compared with DCPS in the PARCC/CAPE era. The new PMF will exacerbate the sector’s compliance validation burden and limit schools’ capacity to use data to improve.

Compliance and accountability are important tools for ensuring our students receive the best possible education. We should hold the same high standard for the efficient and effective administration of compliance activities conducted by PCSB staff as we have in our standards for charter school student outcomes. The proposed PMF adds to the inefficiency and compliance burden already inherent in PCSB’s current oversight practice, which will likely impact schools’ ability to deliver what I care about most: giving historically underperforming student groups a life filled with opportunity that an effective education affords.

How the PCSB Board Can Facilitate Improvement

1. Adopt the state’s new equity-forward accountability system as a significant component of the framework, eliminating a burdensome duplicative compliance activity. Keep the part about mission specific goals as it makes the sector unique and communicates each school’s value beyond math and reading. Find ways to include growth for all students, including K-3 and high schoolers. The policies that rely on the PMF could dictate a minimum average score on the DC report card and percentage of mission specific and K-3/HS growth goals met.
2. Require PCSB staff to update their data architecture and transparently post the code utilized for every metric calculation and compliance activity through GitHub (or similar interface), so data experts across the sector can collaborate and ensure a strong codebase that reduces the amount of time spent error correcting data in the future.
3. Invest in new technology, algorithms, and process policies that make compliance activities more efficient and effective, then look for ways to reduce the authorizer fee so that schools have more resources to improve student outcomes.