TESTIMONY ON BEHALF OF POTOMAC PREP PCS by ASHOK OLI
This testimony, based upon my professional knowledge and personal opinions is not intended to represent the opinions of anyone else. Questions and comments should be sent directly to ashok.oli@gmail.com.

On Jan 14, I had the opportunity to attend the public hearing at Potomac Prep Public Charter School. 
I worked at the DC Public Charter School Board  from 2010 to 2014, where I  played an important role in developing the data validation processes. I also conducted  research and data analyses related to student growth models and the Performance Management Framework (PMF). I was conducting the data validation and analysis tasks during the first roll out of the PMF in 2011 and continued those tasks until 2014. I have also received training on the Colorado Growth model that is used by the PCSB to calculate a school’s performance from its creator, Damian Betebenn. 

A year after leaving the  PCSB, I realized that I could utilize my PMF knowledge and data management and analysis skills to help strengthen the PMF processes by directly helping schools overcome their data validation challenges and providing guidance to strengthen their data validation and safeguard them from  potential PMF-related issues. So, whenever I have spare time, I either review the progress made on the PMF or reach out to individual schools. Luckily, a few months ago, I had the opportunity to work with Potomac Prep.  

As I read through the probational goals, I asked Dr. White-Hood’s team if they fully understood what those goals meant and whether they knew the requirements and business rules that go behind the calculation of those goals. In a quick reply, they said they knew what those terms mean, but when I asked the following questions, they were not able to answer: 
If you have to pick a number that fits under the definition of  “1 standard deviation of 50th percentile on PARCC performance”, what will you pick?What is the cutoff score used in NWEA? How  is 70% calculated for NWEA?What is typical growth on NWEA? 
Clearly, Potomac Prep signed up for the probationary goals without a thorough understanding of them or visualization of what the outcomes of these goals would be. There could have been multiple reasons for this: 

(1) The school had a recent change in leadership three months prior to receiving the revocation notification  (July 2014)  and may have been extremely busy building a strategy for implementing their Turnaround Plan;  (2) They were still cleaning up the shambles -  poor physical and economic conditions the management company left the school in - which may have left them little time to deeply engage in the thorough understanding of these probationary goals; and  (3) PCSB did not thoroughly explain the probationary conditions which was evidenced by the above questions the school could not answer. The first condition was not relevant as PARCC data was not available at that time. 

In the following sections, I would like to present my argument on why Potomac should be given additional time to implement its Turnaround Plan: 
In yesterday’s public hearing, PCSB started by presenting the school’s sharp decline in PMF scores from 2011 to 2014; and then presented its PARCC scores comparing its performance to other schools. 

My comment: I think that these arguments are not relevant to make closure decision because of the following reasons: 

The management company and its leadership team that controlled the school for ten years, who were accountable for the sharp decline in the PMF scores over time, are no longer affiliated with the school.  (Dr. White-Hood took over the school starting July 2014). So, holding one year old leadership for ten years of poor performances seems unreasonable. PCSB had announced the “Hold Harmless” policy, according to which no schools will be held accountable based upon PMF in 2015. So, using PARCC performance as a basis for revocation indirectly contradicts with the Hold Harmless policy. It is worth noting that even though the schools were not tiered, PARCC (previously DC CAS) is used  as a key ingredient in ESMS PMF (NOTE: 80% of the ESMS PMF indicators are made up of PARCC (previously DC CAS), so using it for any high stakes decision looks contradictory to the “Hold Harmless” policy in my eyes. By proposing a goal on PARCC assessment long before it was released, PCSB has made a mistake of “counting chickens before they are hatched”. It simply is surprising to see that the high stakes PARCC related goal  was set without exactly knowing what it was (or without looking at the actual data first). Statistically speaking, using only one data point of PARCC doesn’t look appropriate. This school seems to have put much of its resources on NWEA rather than on PARCC, so it may have struggled to build any student intervention plan based upon previously unavailable PARCC scores. 

2. There was an argument that the school missed the attendance target by less than 2 percentage points. 
My comment: Attendance target was missed by few percentage points (0.7% to 1.8%)  . I know from experience that PCSBs system of record (i.e  Proactive ) is not 100% accurate. I don't want to go into details. So, dinging  schools because of 1 to 2 percent difference does not sound fair. Some cushion should have been given to account for 1 to 5% technology error while making such high stake decisions. 
3. School Did not meet the NWEA goal: PCSB in the meeting mentioned that Potomac Prep PCS did not meet its probationary goal on NWEA. 
My comment: The one size fits all assumption of PMF and  PCSB’s PARCC/NWEA  analysis has flaws. Putting SpEd  and non SpEd population together in an equation without any form of weighting does injustice to the school that is working very hard to improve  the education outcomes of their SpEd population. We ran Potomac’s NWEA performance by separating Non SpEd and SpEd population, and surprisingly, we found that the school meets the 70% target for all non-sped students. Similar pattern appeared when we conducted separate SpEd PARCC performance analysis. Clearly, there is a need for PCSB to think through this “one size fits all” assumption before making any future revocation decisions using PMF. 

Additional Observation: 
While reviewing Potomac Prep’s probationary goals, I attempted to compare those with conditional goals that PCSB proposed for IDEA PCS few years ago (see http://www.dcpcsb.org/charter-board-denies-idea-pcs-15-year-charter-renewal-petition ).  I could not find  similarity between the goals given to Potomac Prep and IDEA PCS, so I wondered if PCSB chooses goals using the standard procedure or uses other reasons to make these types of judgments. Moreover I could not locate a policy that tells how probationary goals will be defined and developed. In my opinion, it is important to create such policy to ensure that personal biases do not influence PCSB's staffs' decisions.

Stay Tuned