How to Lie with Statistics

Chapter 10 serves as a practical guide for looking at statistics. Huff states that this chapter is his true reason for writing the book: giving his audience the tools to examine statistics for themselves. While he notes the importance of insufficient data, he also emphasizes the importance of looking for “sound and usable data in that wilderness of fraud” (124). While readers cannot do all the necessary tests to calculate all the information, there are other ways to discover manipulation. Huff presents a series of five questions for readers to ask themselves about a statistic:

“Who says so?” (123); “How does he know?” (125); “What’s missing?” (127); “Did somebody change the subject?” (131); and “Does it make sense?” (137). Huff clarifies what each question means and how to apply it with examples.

His first question regards the entity making the claim. The reader must look for sources of bias, conscious or unconscious. They should also be aware of entities that are hidden behind “O.K. names,” such as universities or well-known laboratories, to bolster the credibility of the results.

The second question regards issues with the study. This includes apparent biases in the sample or sample sizes that are too small. Huff also specifically includes correlations that are too small to be significant.

For the third question, Huff tells readers to look for any missing information in the statistic. These include the number of cases in the sample, the amount of error, the type of average used, an absent comparison, the raw numbers for a percentage, and an index’s base or the factor that led to the change.

The fourth question concerns statistics in which the focus is altered between the data and the conclusion. In Huff’s examples, he notes that cases of a thing being reported on, whether crime or disease, do not equate to an actual increase in cases. This question is particularly relevant in studies that rely on the people polled to be completely truthful and unbiased. Huff also discusses the issue of correlation not equaling causation and how this is used to change the results of a statistic. Finally, he warns against semantics.

For the fifth and last question, Huff urges the reader to use their common sense when looking at a statistic to see if it holds up. He says that many statistics don’t hold up when given an objective examination and succeed only with an audience that doesn’t think through what is being claimed. Numbers that are too precise to be reasonable or future trends that are only guesswork fall into this category. Huff concludes with a satirical passage by Mark Twain that relies on invective and absurdly exaggerated numbers as a warning against taking extrapolation too seriously.