Yesterday I tackled the Education Alliance for Washoe County's claim that spending more money improves student achievement. Specifically, I showed that the chart above was based on arbitrarily selected states that were in no way shape or form similar to Nevada. I also told you I'd show you how to control for similarities.
I went back to the NAEP Data Explorer and grabbed data from all states and broke down the student scores by income (Free or Reduced Lunch status) and race. As I will show you adjusting for income and race eliminates and previous appearance of some type of correlation with spending and student achievement. It also creates a far superior method of controlling for similar states than the one used by the Education Alliance of Washoe County.
My first chart (below) shows the scatter plot of all states and the low-income student scores on the 8th Grade Math and Reading exams in 2007 (I'm still using the same student achievement data). Why low-income students? Every state has low-income students but every state has different amounts of low-income students. Comparing just low-income students creates a more apples to apples comparison between states. Remember, the Education Alliance didn't control for income they just arbitrarily chose three racial categories and arbitrarily decided that similar states had 30-60 percent of the student population as these racial groups.
The scatter plot above shows that spending isn't related to student achievement. An R2 of 0.0018 means that spending only explains 0.18 percent of the differences in student achievement scores. That means nothing is going on at all. Worse still, the regression analysis shows a P-Score of 0.766. That means the relationship observed is 77 percent likely to have occurred at random.
Those results are more random than Chuck Norris begging for food by dropping you
with a round house kick to the head.
In other words, spending more money does not improve low-income student achievement.
My next charts look at cross-tabulated reports from the NAEP Data Explorer. That is, I exam the scores of low-income African American and low-income Hispanic by state. We get pretty much the same results as above - spending is not correlated with student achievement.
Spending more money doesn't even explain 1 percent of the difference in scores of low-income African American students. There is also an 81 percent chance the relationship occurred at random. Also note there are some states missing. Not every state has enough low-income African American students to create a large enough sample size to produce meaningful data. Two of these states, Arizona and Alaska were included in the Education Alliance's comparison but are excluded here because they don't have enough low-income African Americans in the state.
The situation is the same with low-income Hispanic students. Less than 1 percent of the relationship can be explained by spending but even then there is a 74 percent chance the relationship occurred at random. Worse still, Alabama, Alaska, Louisiana, Mississippi and South Carolina (all considered similar to Nevada) are excluded from this chart because they do not have enough low-income Hispanics in the state to reliably test.
Using more recent spending and student achievement data shows pretty much the same results.
The conclusion you should reach from this is that spending is not correlated with student achievement even after controlling for race and income status. If there is no correlation, you cannot claim any causation - in other words, you can't claim spending more money will improve student achievement.
One more thing, I'm not done skewering the Education Alliance of Washoe County just yet. Stay tuned!