School Rankings Based on Spending and Outcomes Suffer from Severe Methodological Flaws and Insufficient Research
BOULDER, Colo. (PRWEB) November 27, 2018 -- The Reason Foundation recently published a policy brief that offers an alternative ranking of states’ education systems. The brief, which was based on a working paper from the Department of Finance and Managerial Economics at the University of Texas at Dallas, purports to offer needed adjustments and nuance, but makes its own serious mistakes, according to a new review.
Rutgers professor Bruce D. Baker reviewed Everything You Know About State Education Rankings Is Wrong and the underlying working paper, Fixing the Currently Biased State K-12 Education Rankings. He found the analyses provided did little or nothing to advance the conversation about the effectiveness of state education systems.
The twin reports begin with the presumption that high average test scores combined with lower school spending should be the basis for state rankings, which are reasonable premises, depending upon how the analyses are approached. But the reports then head off the rails, Professor Baker explains.
Offering a ‘corrected’ representation of student outcomes and a crude analysis asserting that spending has no relation to those outcomes, the reports declare states such as New Jersey and Vermont to be poor-performing, highly inefficient systems by comparison to many states. The reports then estimate a regression model and assert that the higher performing states are those with (a) weaker teachers’ unions and (b) more children in charter schools.
However, Baker’s review details how the reports’ so-called corrections involved unreasonable and illogical assumptions and adjustments. For example, the reports re-weight racial and ethnic subgroups so that they inappropriately place equal weight in states like Vermont or Wyoming on students comprising 1 to 2% of the population as the other 98 to 99%. Other problems concern a decision to ignore economic status entirely and a poorly executed adjustment for cost of living.
Regressing multiple, highly related, interdependent measures against a specious outcome measure leads to even more suspect findings and, Baker concludes, would only mislead policymakers.
Find the review, by Bruce D. Baker, at:
http://nepc.colorado.edu/thinktank/review-rankings
Find Everything You Know About State Education Rankings Is Wrong, written by Stan J. Liebowitz and Matthew L. Kelly and published by the Reason Foundation, at:
https://reason.com/archives/2018/10/07/everything-you-know-about-stat
Find Fixing the Currently Biased State K-12 Education Rankings, written by Stan J. Liebowitz and Matthew L. Kelly and published by the Department of Finance and Managerial Economics at the University of Texas at Dallas at:
https://ssrn.com/abstract=3185152
Find Documents:
Publication Announcement: nepc.info/node/9518
NEPC Review: nepc.colorado.edu/thinktank/review-rankings
Report Reviewed: reason.com/archives/2018/10/07/everything-you-know-about-stat
NEPC Reviews (http://thinktankreview.org) provide the public, policymakers, and the press with timely, academically sound reviews of selected publications. NEPC Reviews are made possible in part by support provided by the Great Lakes Center for Education Research and Practice: http://www.greatlakescenter.org
The National Education Policy Center (NEPC), housed at the University of Colorado Boulder School of Education, produces and disseminates high-quality, peer-reviewed research to inform education policy discussions. Visit us at: https://nepc.colorado.edu
William J. Mathis, National Education Policy Center, http://nepc.colorado.edu, +1 (802) 383-0058, [email protected]
Share this article