Book Review: Weapons of Math Destruction

Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy
Cathy O’Neil
Crown New York 2016
Book Web Site: https://weaponsofmathdestructionbook.com

The Author’s Blog: https://mathbabe.org

My Rating: 4/5

Cathy O’Neil is a Harvard Ph.D. mathematician, former assistant professor at Barnard College, former Wall Street “quant” at the D.E. Shaw hedge fund, former Wall Street risk metrics developer at RiskMetrics Group, a former data scientist with Internet advertising firm Intent Media, and former Occupy Wall Street activist who blogs at https://mathbabe.org Weapons of Math Destruction is her third book (her first was Doing Data Science: Straight Talk from the Frontline). Weapons of Math Destruction has been excerpted in The Guardian, reviewed in the New York Times and other venues, and been the subject of several podcast interviews and video talks and interviews by and with Cathy.

Weapons of Math Destruction focuses on the potential or actual harm of the mathematical models that are proliferating in the current Big Data/Data Science/Machine Learning craze. The book is primarily concerned with models, often secret and proprietary, used to generate scores such as teacher performance scores, criminal recidivism scores, and credit scores that are used in critical decisions such as hiring and firing, approving loans, sentencing criminals. In the author’s terminology, a Weapon of Math Destruction is a bad model or a good model used badly that adversely affects a large number of people. The book and the author is especially concerned with unfair discrimination against minorities, particularly African-Americans, and the poor. The book is dedicated to the underdogs.

Weapons of Math Destruction is a play on the infamous Weapons of Mass Destruction or WMD’s used to justify the US invasion of Iraq in 2003. The book frequently uses the acronym WMD but for the Weapons of Math Destruction.

The book gives many examples of mathematical models in current or recent use. The book gives good reasons for doubts about many of these models. A number of the models, such as the controversial Value Added Models (VAM) for teachers, are models I have run across before and have substantial doubts about. The book gives a good introduction to and overview of the actual and potential dangers of mathematical models today.

Significant Weaknesses

The book has some significant weaknesses. It has little actual math: not a single formula and only a few numbers and very brief explanations of a few concepts such as Simpson’s Paradox — cited in an attempt to explain the decline in SAT scores between 1940 and the publication of the Nation at Risk Report in 1983 during the Reagan Administration. It relies on limited analyses, often referencing external sources, and proclamations from the author that a model is a Weapon of Math Destruction. While it raises doubts, sometimes strong doubts, about several models, it does not provide a detailed analysis demonstrating that the model is wrong and harmful.

Almost all the examples are complex and require further research by the reader to evaluate the book’s claims about the models. This is probably unavoidable given the complexity of real-world mathematical modeling but it remains a weakness.

The book has a chapter on the U.S. News and World Report college rankings, blaming them for the run-up in tuitions — after adjusting for inflation — at many colleges and universities since the rankings debuted in 1983. I found this argument unconvincing. The book argues that the rankings forced colleges and universities to spend large sums of money on sports stadiums, building, winning sports teams, whopping salaries for more and more administrators, and many other expensive activities to maintain or raise their ranking.

All of this activity takes place within a vast ecosystem surrounding the U.S. News rankings, whose model functions as the de facto law of the land.

O’Neil, Cathy (2016-09-06). Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy (Kindle Locations 924-925). Crown/Archetype. Kindle Edition.

The U.S. News rankings are certainly not the law of the land. Colleges and universities have always had the option of ignoring them and emphasizing both individually and collectively through college and university associations the cost benefits and low tuitions from avoiding unnecessary expenditures on sports teams, administrators, and other white elephants.

When companies and other institutions raise prices, often reporting record profits or record administrator salaries in this case, and appear to be gouging their customers, they usually cite a variety of excuses. “Mommy, the government made me do it!” In the United States, the most common excuse seems to be the government. The pharmaceutical industry funds widely cited studies by the Tufts Center for the Study of Drug Development blaming FDA approval costs and other scapegoats, rather than the huge profits of the industry, for soaring drug costs.

When gasoline prices soared in the late 1990’s after a series of oil industry mergers that created ExxonMobil, ConnoccoPhilips, and other giants, the industry and its apologists blamed state and local gasoline taxes, restrictions on drilling on public lands, and various environmental regulations for the sudden run up in prices (not to mention Saudi Arabia, Osama Bin Laden, the never ending conflict between Israel and the Palestinians in a country conspicuous for its almost total lack of oil, and perennial strife in Nigeria — foreign governments and foreign politics).

While the U.S. News and World Report is not actually the government, describing it as “the de facto law of the land” places the blame somewhere far away from the colleges and universities who actually set the prices and make the decisions.

The chapter “Civilian Casualties: Justice in the Age of Big Data” actually cites data showing a sharp drop in homicides and other violent crimes correlated with the adoption of the “broken windows” and “zero tolerance” policing policies that the book criticizes. A former Occupy Wall Street activist, the author expresses strong liberal and left-wing convictions that result in contradictions like this in several places in the book.

Some credited these energetic campaigns for dramatic falls in violent crimes. Others disagreed. The authors of the bestselling book Freakonomics went so far as to correlate the drop in crime to the legalization of abortion in the 1970s. And plenty of other theories also surfaced, ranging from the falling rates of crack cocaine addiction to the booming 1990s economy. In any case, the zero-tolerance movement gained broad support, and the criminal justice system sent millions of mostly young minority men to prison, many of them for minor offenses.

O’Neil, Cathy (2016-09-06). Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy (Kindle Locations 1303-1307). Crown/Archetype. Kindle Edition.

Yet many in the public associated the program with the sharp decline of crime in the city. New York, many felt, was safer. And statistics indicated as much. Homicides, which had reached 2,245 in 1990, were down to 515 (and would drop below 400 by 2014).

O’Neil, Cathy (2016-09-06). Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy (Kindle Locations 1370-1372). Crown/Archetype. Kindle Edition.

Figure showing the sharp drop in homicides in the 1990s’ from Bureau of Justice Statistics, Special Report, August 2007, NCJ 214258, Black Victims of Violent Crime by Erika Harrell, Ph.D. BJS Statistician

Ineffective Solutions

The book offers two possible solutions to the perils of the mathematical models, machine learning, and “algorithms” that are proliferating in the modern world. Both are unlikely to be effective.

The first solution is a “Hippocratic Oath” for data scientists. Simply put, pledge to “do no harm.” The Hippocratic Oath has already proven ineffective in preventing a wide range of abuses and simply mistakes in medicine. As the grim saying goes: “some people are monsters.” Perhaps more significantly, most data scientists are employees who derive all or most of their income from a single employer who can easily dismiss them if they object. This is a substantial deterrent to most people, especially those with families to support. Most people are not monsters but neither are they super heroes.

The second solution is for the government to require auditing and some sort of certification of mathematical models analogous to the credit ratings provided by Moody’s, Fitch, and other firms in the lead up to the financial crisis in 2008. Numerous laws and regulations of this type were in place at the time and as Weapons of Math Destruction rightly notes with considerable outrage, proved totally ineffective.

A more skeptical public unwilling to borrow for new homes and home improvement projects might have prevented the housing bubble and subsequent crash but government regulations including extensive auditing requirements clearly did not. The crony capitalist system with its bipartisan revolving doors between government and industry that has been increasingly called out during the current bitter election in the United States renders government regulation and reform unlikely to succeed.

To be fair, I cannot at present offer better solutions than the book proposes, but they are unlikely to work.

Conclusion

I recommend reading the book or at least listening to some of Cathy O’Neil’s podcast audio interviews and YouTube video presentations on Weapons of Math Destruction. Although the book has some significant weaknesses, it is a comprehensive and detailed introduction and overview of a growing problem caused by the misuse or deliberate abuse of mathematics and statistics coupled with modern powerful computers and high bandwidth networks.

Cathy O’Neil’s Weapons of Math Destruction Interview at the Ford Foundation

© 2016 John F. McGowan

About the Author

John F. McGowan, Ph.D. solves problems using mathematics and mathematical software, including developing gesture recognition for touch devices, video compression and speech recognition technologies. He has extensive experience developing software in C, C++, MATLAB, Python, Visual Basic and many other programming languages. He has been a Visiting Scholar at HP Labs developing computer vision algorithms and software for mobile devices. He has worked as a contractor at NASA Ames Research Center involved in the research and development of image and video processing algorithms and technology. He has published articles on the origin and evolution of life, the exploration of Mars (anticipating the discovery of methane on Mars), and cheap access to space. He has a Ph.D. in physics from the University of Illinois at Urbana-Champaign and a B.S. in physics from the California Institute of Technology (Caltech). He can be reached at [email protected].