Shortcomings in “Efficiency Scores” and Logic Weaken Charter School Report, Says NEPC Review
Boulder, CO (PRWEB) July 12, 2016 -- A recent report from the Wisconsin Institute for Law and Liberty concludes that the most efficient public schools in Milwaukee are charter schools that experience greater autonomy from the district. The report’s authors created “efficiency scores” for Milwaukee schools and, on the basis of those scores, draw conclusions about the relative efficiency of Milwaukee’s charter and traditional public schools.
Upon review, the bold claims about these relative efficiencies were found to be too weak to serve any useful function for policymakers.
University of Connecticut professor Casey Cobb reviewed Bang for the Buck: Which Public Schools in Milwaukee Produce the Best Outcome Per Dollar Spent? for the Think Twice Think Tank Review Project at the National Education Policy Center, housed at the University of Colorado Boulder’s School of Education.
As Professor Cobb explains, the report includes two sets of scores. The first set was generated by dividing test scores in math and science by per-pupil funding. In a second analysis, “efficiency scores” were estimated for school types using a regression model that accounted for selected student demographic characteristics.
Five major problems arise from the report’s approach, Cobb finds. First, test scores do not comprehensively represent the purposes of schools. Second, the report does not address threats to the validity of its assumption that there is uniform financial accounting across schools and types. Third, the analytic description of the study is incomplete, making interpretation difficult. Fourth, “autonomy” is never really defined—it is just used as a loose term implying independence—so autonomous behavior is assumed by virtue of their charter status. The report then makes strong but unmeasured claims about the superior “efficiency” of charter schools based on their having this greater autonomy. Finally, while the report’s analysis controls for some school demographic characteristics, it does not appear to adjust for selection effects—effects that could prove fatal to their conclusions.
As a result of its manifold flaws, and because its conclusions are not supported by the evidence presented, Professor Cobb concludes that the report is of little if any use to policymakers.
Find Casey Cobb’s review at:
http://nepc.colorado.edu/thinktank/review-WILL
Find Bang for the Buck: Which Public Schools in Milwaukee Produce the Best Outcome Per Dollar Spent, by Will Flanders & CJ Szafir, published by the Wisconsin Institute for Law and Liberty, at:
http://www.will-law.org/wp-content/uploads/2016/05/Bang-for-the-Buck-FINAL.pdf
The National Education Policy Center (NEPC) Think Twice Think Tank Review Project (http://thinktankreview.org) provides the public, policymakers, and the press with timely, academically sound reviews of selected publications. The project is made possible in part by support provided by the Great Lakes Center for Education Research and Practice: http://www.greatlakescenter.org
The National Education Policy Center (NEPC), housed at the University of Colorado Boulder School of Education, produces and disseminates high-quality, peer-reviewed research to inform education policy discussions. Visit us at: http://nepc.colorado.edu
Find Documents:
Press Release: nepc.info/node/8127
NEPC Review: nepc.colorado.edu/thinktank/review-WILL
Report Reviewed: http://www.will-law.org/wp-content/uploads/2016/05/Bang-for-the-Buck-FINAL.pdf
William J. Mathis, National Education Policy Center, http://nepc.colorado.edu, +1 (802) 383-0058, [email protected]
Share this article