Review of Damned Lies and Statistics: Untangling Numbers from the Media, Politicians, and Activists

DAMNED LIES AND STATISTICS: Untangling Numbers from the Media, Politicians, and Activists
(UPDATED EDITION)
Joel Best
University of California Press
Berkeley and Los Angeles, California
2001, 2012
Print Length: 213 pages

My Rating: 4/5

Introduction

Damned Lies and Statistics is an excellent book on the misuse of statistics by Joel Best, a Professor of Sociology and Criminal Justice at the University of Delaware. Lies was first published in 2001 and updated in 2012. There is also a sequel More Damned Lies and Statistics: How Numbers Confuse Public Issues.

Lies gives many examples of bad statistics about social problems and public policy from the 1980’s and 1990’s. It also discusses many common patterns in how the bad statistics were produced and often widely disseminated by the mass media and “experts.” In many respects, it gives a partial checklist of common flaws and outright deception in statistics about social problems. Most of these are equally applicable to statistics in medicine and public health, finance, and many other fields.

This is not a deep statistics book. It deals mostly with averages, rates, medians and other commonly quoted statistics. It touches on more advanced topics such as the “odds ratio” only in giving examples where esoteric terms from advanced statistics are innocently misinterpreted by the mass media — or perhaps used in a deliberately misleading way by officials or “experts.” Rather it focuses on common misuses of statistics in politics and the mass media in the tradition of Darrell Huff’s classic How to Lie with Statistics, an important and contentious area often avoided by professional statisticians.

The Million Missing Children

I first encountered Professor Best researching the many overlapping claims and counter-claims about missing children, serial killers, and such outre topics as Satanic Ritual Abuse in the 1980’s. Best is also the author of Threatened Children: Rhetoric and Concern about Child-Victims (University of Chicago Press, 1990) and a number of other books and articles about this topic.

In particular one of the more remarkable claims from this period, still alive today on the Internet, was the claim that one million children went missing each year in the United States, often implied to be horrific stranger abductions and murders by serial killers and even Satanic cultists. Keep in mind that in the early to mid-1980’s when these claims reached their peak, the total number of children in the United States (aged 0-17 years) was about sixty-two million. This would imply that most neighborhoods and most schools in the United States would have been experiencing at least one horrific abduction and murder of a child every year.

Of course, the million missing children number was highly misleading. Activists had rounded up from a nationwide number of about eight-hundred thousand (800,000) “missing children” reported to the FBI, that consisted primarily of short-term disappearances, dominated by teenage girls spending time with their boyfriends without parental permission. They often quoted the number in ways and in contexts that implied that it referred to the abduction and murder of children by strangers. The Denver Post won a Pulitzer Prize for reporting in 1985 dissecting and debunking these numbers.

In Lies Best discusses many examples of bad statistics from this period, including the more sober, more specific claim of 50,000 children abducted and killed by strangers, orders of magnitude larger than the approximately seventy-eight claimed by the FBI at the time. Incidentally, the total number of all homicides in the United states in 1984 was 18,690 according to official statistics. Some modern numbers can be found at the National Center for Missing and Exploited Children. In 1999,

An estimated 115 children were the victims of “stereotypical” kidnapping. These “stereotypical” kidnappings involved someone the child did not know or was an acquaintance. The child was held overnight, transported 50 miles or more, killed, ransomed or held with the intent to keep the child permanently.

In the book, Best shows a pattern in which activists produce a big number for the size of a social problem that is often a round multiple of one-million, sometimes an educated guess in the absence of solid data, and the number is then taken up and repeatedly endlessly by the mass media, politicians, and others. One million missing children. Three million homeless people. Three million jobs available in America that are not filled because too many of our people don’t have the skills for those jobs.

The alarming big number, frequently based on a broad definition or even sheer guesswork, is often combined with a specific, horrific, and highly unusual example or examples of the social problem, such as the gruesome murder of six year old Adam Walsh in 1981.

In more than a few cases, the big number mutates further into an even more alarming number. In one case, 150,000 young women suffering from anorexia nervosa, an eating disorder in which the woman may starve herself to death, in the United States changed into 150,000 young women dying each year from anorexia nervosa.

Indeed, Internet wags have coined the term citogenesis for the process by which even a flat out error may by frequent repetition turn into a “fact” that “everyone knows.”

Innumeracy

In the book, Best attributes some of these bad statistics to innumeracy, the poor math and statistics skills of activists, reporters, and the general public, although he ultimately concludes that innumeracy cannot really explain the persistence of bad statistics and their uncritical acceptance by reporters and the public who should know better.

I tend to agree with this assessment. Many of the patterns described in the book are also found in the ubiquitous STEM (Science, Technology, Engineering and Math) shortage claims that I have studied and written about before. The patterns are also very similar to the many questionable claims found in the software design and engineering literature. In both cases, the claims are both produced by and believed by people with strong, even exceptional mathematical skills. Innumeracy is no explanation.

Nonetheless, I think there are a couple of lessons about the limitations of mathematical thinking even by people with strong math skills, 800 math SAT scores etc.. First, we mostly deal with small numbers every day and in most school math problems. Most purchases are less than a thousand dollars. Most people spend more only a few times in their lives: tens of thousands of dollars for a car or college tuition, hundreds of thousands of dollars for a home. We own at most a few thousand items and usually deal with much smaller numbers of items in every day activities. We aren’t used to huge numbers like one-million or one-billion or one-trillion. We don’t always notice mathematical errors or sloppy thinking that would be obvious with small numbers.

Further, many of these big numbers require additional numbers to interpret properly such as the population of the nation (about 324 million in 2015), the Gross Domestic Product ($17,701.3 billion), the total size of the US federal budget (about $3.5 trillion), the total number of deaths per year ( 2,596,993 ) that we may not know or think about often. These additional numbers may themselves have problems in their definition and measurement not shared with the small numbers that we encounter in daily life.

Second, strong mathematical skills don’t protect us from common cognitive flaws like confirmation bias and wishful thinking. Lies is more a book about these weaknesses in human thinking in a mathematical context.

Some Weaknesses

At the beginning of the book, Best mentions that both liberals and conservatives are guilty of producing and using bad statistics. He gives examples of both in the book. He makes the point that he gives examples of bad statistics used in support of causes that he agrees with as well as causes he opposes. But he does little to tell the reader where he stands or even if he has a stand on many of the issues in his examples. I at least would like to know his personal opinion, where he stands, if he has an opinion, in part to evaluate any possible bias on his part.

One example in the book struck me as contrived in particular. Best discusses some studies of relative crime and poverty rates among blacks and whites in the United States. These show, for example, that blacks are both more likely to commit and to be victimized by crime than whites in the United States.

Black vs White Crime Rates from Damned Lies and Statistics by Joel Best

He then argues that race can be considered as a proxy for “class,” in my mind an even fuzzier category than “race.” He then shows hypothetical data comparing crime rates between blacks and whites in the United States, sorted and grouped by social class which he does not define clearly. Note that Best uses social class rather than income level, something that could be determined from official records and statistics such as income tax returns or payroll records.

Table 5. Hypothetical Data Showing How Apparent Racial Differences in Arrests Might Really Reflect Class Differences

Hypothetical Data

Best, Joel (2012-08-07). Damned Lies and Statistics: Untangling Numbers from the Media, Politicians, and Activists (p. 119). University of California Press. Kindle Edition.

This hypothetical data shows similar crime rates — both rates of committing crimes and being victimized — for blacks and whites in the same social class. Blacks have overall higher crime and victimization rates because a higher proportion of blacks are lower class than whites.

Best seems to be arguing that this hypothetical data rules out race as the determining variable in crime rates and presumably poverty. The real issue is “class.” This switch from actual to hypothetical data in the middle of the discussion is also the sort of sleight of hand that one encounters in intentionally or accidentally misleading discussions of social problems and public policy.

However, even if the actual data is similar to the hypothetical data, if race causes “class,” this is not the case. Blacks could tend to be lower class because of discrimination (the liberal explanation) or bad genes (the old school conservative explanation). This peculiar passage in the book is an example where a clear statement of the author’s personal beliefs and opinions would be helpful, rather than maintaining the pretense of absolute academic objectivity.

Although the book gives a good, although incomplete checklist of questions to ask about any statistic such as where did it come from, is it a guess, how is it defined, how is it measured, and how was the sample used to compute the statistic selected, it does not provide much guidance in how to research the answers to these questions, especially in the limited time and with the limited resources available to most of us. In the present day, many readers are likely to do a Google search and turn up a Wikipedia page. On controversial topics, Wikipedia pages often appear to be taken over by one side or another of the controversy, perhaps the side with the most resources or the most stubborn, fanatical adherents.

Other hits are articles from the mass media, e.g. CNN or the New York Times, which often exhibit the faults that Best critiques in his book. Some fringe web sites such as Alex Jones or David Icke may occasionally show up, but these are difficult to evaluate and often kooky, even if they occasionally make valid points. Many other hits appear to be Google advertisers trying to sell something vaguely related to the search term or phrase.

Conclusion

Lies is an excellent book about bad statistics in the mass media and politics. In fact, many of the issues discussed in the book have broader applicability to medicine and public health, finance, and many other areas. It is not a rehash of Darrell Huff’s How to Lie with Statistics, and covers many additional topics.

The book gives some good guidance on how to apply critical thinking to statistics on social problems in the mass media and politics, as well as other arenas. As I illustrate in the case of some black versus white statistics in the book, readers should apply critical thinking to the book itself as well.

Most readers will probably benefit from referring to the book and its incomplete checklist of common flaws in statistics whenever they encounter a scary or outrage-provoking number such as “one million missing children.”

© 2015 John F. McGowan

About the Author

John F. McGowan, Ph.D. solves problems using mathematics and mathematical software, including developing gesture recognition for touch devices, video compression and speech recognition technologies. He has extensive experience developing software in C, C++, MATLAB, Python, Visual Basic and many other programming languages. He has been a Visiting Scholar at HP Labs developing computer vision algorithms and software for mobile devices. He has worked as a contractor at NASA Ames Research Center involved in the research and development of image and video processing algorithms and technology. He has published articles on the origin and evolution of life, the exploration of Mars (anticipating the discovery of methane on Mars), and cheap access to space. He has a Ph.D. in physics from the University of Illinois at Urbana-Champaign and a B.S. in physics from the California Institute of Technology (Caltech). He can be reached at [email protected].