Polls and Statistics

Someone once said there are lies, damned lies, and statistics.  It would seem that national polls are based on all three.  2016 pollsters had Hillary Clinton as our next president in a landslide.  We all know how that turned out.

 The following quote came from the LA Times on October 4, 2016:

"...the election forecast at Nate Silver’s fivethirtyeight.com gives Hillary Clinton a 71.2% chance of beating Donald Trump (who accordingly has a 28.8% chance of winning); the New York Times Upshot blog has her at 80%; Daily Kos Elections has her at 72%; and the Princeton Election Consortium at 85% or 91%, depending on how you crunch the raw numbers."

Hillary Clinton wins by 70%

Here we are embarking on yet another presidential election year.  We are already being bombarded with studies, polls, and the results of statistical analysis.  Math was never my strong suit but I’m generally good using logic.  Hindsight is also something that has served me well in the past.  So, how accurate can these polls be?  I know I’ve never been asked a political question in response to a national poll but, then again, I don’t answer the phone to “unknown number” or similar cryptic Caller ID’s.

A friend recently put up a meme that was questioning California taxes (hinting at the socialism involved) and then pointing out that, even with its high taxes, California was 42^nd in the nation with its educational system.  I had trouble swallowing that number and went about verifying its accuracy.  He was right, California is 42^nd in the nation according to accepted statistics.

I still had trouble getting my head around that number so, like Paul Harvey before me, I decided to get, “the rest of the story.”

Paul Harvey, "Now you know, the rest of the story."

What I found was a case study on how numbers can be misleading.  National education polls generally rely on test results from NAEP or the National Assessment of Education Progress, aka, the national report card.  These tests average scores for various subjects such as math, reading, and science for different grade levels.

While this sounds logical, further analysis shows it can present an incomplete picture.  One study of these results found that most published interpretations “ignore heterogeneity.”  Yeah, I had to look that one up too and the definition still didn’t explain the problem.  It seems that they are referring to the failure of these tests to evaluate the impact of the variations in socioeconomic and ethnic backgrounds.  This was starting to sound like some type of left-wing spin to counter the raw number scores until I got into the real-world examples.

The CATO Institute published a paper in November of 2018 that used Texas and Iowa as a comparison.  Iowa (#8) ranks much higher than Texas (#33) in the usual educational rankings of students in the K-12 scores.  With those numbers, you would think that you would be better off taking a job in Iowa and putting your kids in that system rather than Texas.  You might be wrong.

Iowa is 87% White, 5% Hispanic, 3% Black, and 2% Asian.  Texas, on the other hand, is 43% White, 38% Hispanic, 12% Black, and 4% Asian.  The poverty rates for both states for their White population are equal at 7%.  This could give Iowa a significant advantage in student test scores if you believe that poverty has an influence on education.  I would personally accept that premise.  With 93% of the poverty for each state focused on their respective minority populations, and with Iowa having but 13% of their citizens occupying that classification, they might understandably have a better raw score ranking.

With just that one additional factor of poverty, you can see why the raw test scores averaged across each state might not paint an accurate picture of their educational systems.  With Iowa, outscoring Texas in each of three tests in reading, math, and science at both the fourth and eighth-grade levels; it would reasonably be assumed that disaggregated comparisons by race would show similar results.  In actuality, Texas outscores Iowa in all but one category in twenty exams given by both states.  The one exception is the reading test for eighth-grade Hispanic students where Iowa tops Texas.  The disaggregated comparison puts White scores in each state head to head and likewise compares Hispanic, Black and Asian scores in the three categories.  By this arguably more even comparison, the Texas school system is better than the one in Iowa even though the raw numbers have the results flipped.

President Harry Truman holds a copy of the Chicago Daily Tribune bearing the infamous headline "Dewey Defeats Truman" in November 1948.  The polls were so sure that newspapers pre-printed their headlines much to their embarrassment.

The point of all of this is just to show that polls and surveys are only as reliable as the methodology used for the results.  When you see that a new poll has been released and it shows favorable results that you want to believe, take it with a grain of salt.  At this point and with recent history in mind, you might want to go off your low-sodium diet, remove the cap and dump the salt shaker.  When in doubt, review the quote and national map graphic above.

Get off your ass and vote!