Last week I saw a YouTube video purporting to show how dumb the masses are. It cited a poll which allegedly found that 7% of Americans surveyed think chocolate milk comes from brown cows. It’s unlikely, if you think about it, that any significant number of people hold this belief, but it makes viewers feel good to think they’re smarter than others.

A search revealed that lots of sites uncritically reported this result. In fact, when I searched for a stock brown-cow image to stick on this post, many of the results referenced the poll. On the bright side, I found an article that looked carefully at the poll. Just now I found a Columbia Journalism Review article demolishing the claimed result.

Here’s an example of the “reporting” which Yahoo provided. I’m not honoring it with a link.

How now, brown cow? The question took on renewed relevance this week following an online survey by the Innovation Center of U.S. Dairy that found 7% of all American adults believe chocolate milk comes from brown cows. That percentage works out to 16.4 million Americans, a number greater than the population of all but four states.
 
The survey results have stoked concerns among nutritionists, farm groups and others in the food industry who fear most Americans lack basic food knowledge, and because of this they make uninformed choices for their diet.

That percentage matches the results of Public Policy Polling findings from 2013 that revealed 7% of voters think the moon landing was faked.

That’s typical of the laziness which passes for journalism today. The rest of the article discusses results from other polls which supposedly demonstrate how dumb people are. For example: “Just over than [sic] half of Americans, 54%, polled by survey company YouGov, believe in the existence of intelligent aliens.” Imagine that! Most Americans think that somewhere in the universe, there may be other intelligent species!

The CJR article quotes a bit from an exchange on NPR’s All Things Considered between co-host Audie Cornish and Jean Ragalie-Carr, president of the National Dairy Council:

CORNISH: Jean Ragalie-Carr is president of the National Dairy Council, which commissioned the survey. She says they put that question to a thousand people and gave them several options for how to answer.
 
RAGALIE-CARR: Well, there was brown cows or black-and-white cows, or they didn’t know.

Got that? The people polled were given the choice of two silly answers or “don’t know.” It sounds as if the question was stuck into the poll as a joke. But the president of the National Dairy Council chose to treat it as demonstrating the shocking ignorance of Americans (perhaps so the NDC can get more money to educate us out of our ignorance), and the hosts on All Things Considered didn’t consider the problem.

Some articles mentioned that 48% of respondents answered the question with “don’t know.” With such ridiculous options, how else could they respond? This number should have made the reader further doubt the poll’s accuracy, even without knowing what the available answers were. But the authors of the articles saw the story just as a way to let people feel superior. Everyone will assume it’s the “other” group that’s dumb. Democrats will think Republicans believe “milk plus chocolate and sugar” is “fake news.” Rural Republicans will think those citified Democrats are totally out of touch.

In this case, the bad reporting was basically harmless. But it should make us aware of how common such sloppiness is and how even respected sources like NPR will uncritically pass bad information along. As the CJR article says, “The evergreen problem is that if we feel like we can’t trust journalists to vet the small stuff for us, we worry that we can’t trust them with the big stuff, either.”

Be careful of any poll results that come from an organization that has a stake in them or doesn’t have a record of good methodology. I’m sure you’ve all gotten polls from organizations that ask heavily loaded questions and wrap up with “How much money will you give us?” Whatever results they report should be regarded with a kilogram of NaCl.

This blog is directed at people professionally involved with the written word, and the people reading this include writers who could be tempted to uncritically pass along what they find on other websites. Please don’t be one of those bad reporters.