New STEM PhDs and Industry

This article takes a look at the job prospects of freshly minted STEM (Science, Technology, Engineering, and Math) Ph.D.’s who pursue a job in private industry, which frequently means some form of software development in the “technology,” formerly known as “computer,” industry. On September 19, 2013, Chand John, a recent Ph.D. graduate from the top rated Stanford University computer science department, published an article “The Ph.D.-Industry Gap” in the Chronicle of Higher Education recounting his lengthy, difficult search for a job in industry. Judging from his account, his job hunt in no way resembled the claims made by his graduate program at Stanford or the many “talent war stories” found in the mass media about the alleged battle for programming talent in the Silicon Valley. The article spawned many comments on the Chronicle of Higher Education and Hacker News web sites. This article discusses the realities facing many freshly minted STEM Ph.D.’s seeking a job in private industry and a number of related issues that received scant mention either in Chand John’s article or the many comments.

Generally, I will argue that Ph.D.’s in practice are not sought after by most industry, including the “technology” industry. In fact, a STEM Ph.D., especially a recent one, will usually make it harder to find a job in industry and closes some doors entirely due a widespread prejudice against Ph.D.’s. However, there are special cases where a Ph.D. or a Ph.D. in a specific area will open some doors — including some doors one should not go through. Because a Ph.D. program usually takes at least five years — and often longer today — it is difficult or impossible to accurately predict if a specialized area will be “hot” when the degree is issued and the fresh Ph.D. needs to find a job.

The Plural of Anecdote may be Data

At least based on word of mouth, experiences such as Chand John’s lengthy search with many rejections are common amongst newly minted Ph.D.’s even from top rated programs, and even sometimes with allegedly “hot” skills. As an example, many moons ago, when I was finishing up my Ph.D., I knew a recent Ph.D. in EE (Electrical Engineering) who had worked in an area of biomedical engineering. Like Chand John, he had extensive programming skills but found Silicon Valley employers rather uninterested despite the perennial claims of a desperate shortage of programmers. His job search took over a year and eventually resulted in a good job at an industry sponsored research institute where his Ph.D. thesis research had a direct, specific application. Chand John also reportedly eventually found a job at a research institute more closely related to his Ph.D. research topic.

In the “technology” industry, Ph.D.’s and especially newly minted Ph.D.’s often report encountering strong prejudices against them which include: Even exceptionally difficult programming projects in a research setting are dismissed as not “real” in some often ill-defined way. A fear that Ph.D.’s may leave and found their own company. A general hostility that may disguise a fear that the Ph.D. is smarter or more skilled and poses a threat to the non-Ph.D. employees. This final prejudice is almost never stated :-).

Business Leaders Say They Want Ph.D.’s

It is common when recent Ph.D.’s complain that they encounter little interest in their degree and indeed active hostility in some cases, for commenters to express astonishment: “How could you think a Ph.D. makes you special or anyone in industry would want a Ph.D.?”

In fact, in certain contexts, business leaders and their allies claim or strongly imply they need and want Ph.D.’s. This often occurs in the context of the ubiquitous STEM shortage claims, lobbying for various visa programs such as the controversial H1-B visa, and lobbying for ostensible education reform. A particularly egregious example of this widespread practice is the editorial “Our Ph.D. Deficit” by former Lockheed Martin CEO Norman Augustine and former Stanford Linear Accelerator Center (SLAC) Director and Nobel prize winning physicist Burton Richter in the May 4, 2005 Wall Street Journal. Another specific example is the failed STAPLE (Stopping Trained in America PhDs from Leaving the Economy) Act to give new Ph.D.’s green cards (permanent resident status) that was introduced in the United States House of Representatives in 2009. Indeed, the graphic image of stapling a Green Card to a fresh Ph.D. diploma occurs frequently in debates about high-tech immigration policy to this day.

Many Businesses Exaggerate Their Research Activities

In certain contexts, especially political lobbying, many businesses exaggerate their research activities. The most extreme example of this is found in the United States pharmaceutical industry where research is routinely invoked to justify large and rising prices of medicines and huge profits and profit-margins, mostly funded by the public Medicare program. For example, the industry-funded Tufts Center for the Study of Drug Development (CSDD) has for decades put out widely quoted studies claiming astonishingly high costs for the research and development of drugs, currently $1.2 billion per drug in 2005 dollars. These estimates are based on secret industry data and questionable accounting that bears little resemblance to common household accounting.

The pharmaceutical industry is not alone. As oil and energy prices and profits have soared over the last decade and a half, the petroleum industry has sought to replicate the evident political success of the pharmaceutical industry. In 2008, as oil and gasoline prices soared, ExxonMobil launched an advertising and public relations campaign featuring profiles of photogenic scientists working on new energy technologies at the giant oil company, seeking to portray the company as a public spirited research lab staffed by lovable nerds. So too, British Petroleum (BP) has a “BP Beyond Petroleum” advertising and public relations campaign focussing on BP’s obviously token investments in alternative energy research. In fact, many studies show minimal investment in research and development by major oil companies. Remarkably, they seem to have few concerns about “Peak Oil” despite invoking the alleged rising cost of oil exploration and recovery to explain rising prices (and profits — how does that work?). The research that produced the reported breakthroughs in hydraulic fracturing, “fracking,” were funded by the US Department of Energy.

In the “technology” industry, it is common to find that the core technology was researched and developed at public expense, often at a university such as Stanford, with funding from government funding agencies such as DARPA (the Defense Advanced Research Projects Agency). Nonetheless, the high tech companies that license the technology, often from universities under very favorable terms due to the obscure Bayh-Dole Act, often strongly imply they did the original research and development. Their public relations and advertising activities often portray these firms as public spirited research labs full of lovable nerds with a sales and marketing department tacked on as an afterthought, which is grossly inaccurate even for the few companies such as Hewlett-Packard that do invest heavily in scientific research.

Successful research with perceived large benefits, “breakthroughs” such as penicillin, is very popular in the United States and probably around the world. It is also very long term (usually over five years), risky (high failure rate — often claimed to be eighty to ninety percent), and sometimes very expensive, e.g. the Manhattan Project at $20 billion in current dollars. Thus many businesses avoid research in practice, leaving the heavy lifting to government programs. But the popularity of successful research means that businesses that do little research, whose business model may be the commercialization of proven technology rather than true research or even something much farther removed from research, seek to wrap other agendas in the mantle of science. The major mass media in the United States, which receives millions, if not billions, of dollars in advertising revenues from these campaigns, rarely points out the factual problems with these claims in ostensibly objective news articles.

It is not surprising that many STEM students believe there is a strong industry appetite for freshly minted Ph.D.’s given these numerous advertising and public relations campaigns by many businesses.

The General and the Specific

It is impossible to understand the many claims and counter-claims about STEM jobs and careers without drawing a careful distinction between general claims such as “we can’t find Ph.D.’s” and specific claims such as “we can’t find Ph.D.’s in finance with a thesis on the application of stochastic differential equations to the valuation of collateralized debt obligations (CDO’s) in Sweden and at least three years of paid industry experience at the trading desk of a major European bank” :-). Businesses do seek Ph.D.’s with very specialized skills, usually with additional “industry” experience that freshly minted Ph.D.’s rarely possess. The vast majority of Ph.D.’s have not done research in the “hot” fields — meaning specialized sub-fields of broadly defined disciplines such as Computer Science or Physics. This is inherent to the high risk, which means “high failure rate,” of scientific research. Most research never succeeds; most research never becomes “hot.”

Businesses are generally interested in the research that produced not merely academic/scientific success but immediate business success. Anecdotally, only 10-20 percent of research is successful as defined by academics and scientists which often means peer-reviewed and widely cited by other academics and scientists. Many genuine scientific breakthroughs were not widely cited until decades or even centuries after publication :-). Industry is interested in the subset of successful research that may be immediately profitable, or at least investors can be convinced that it may be immediately profitable. So, for example, the recent discovery of the Higgs Boson at CERN has produced a big giant yawn from the business community, whereas the World Wide Web, originally a tool for sharing research articles at CERN, produced a frenzy in the business world.

Dennis Ritchie Couldn’t Code

One of the things that exasperated Chand John, and many other freshly minted Ph.D.’s over the years (the author included), is that his, by his account, extensive experience writing computer software, coding, starting at age eight and including, again by his account, a widely downloaded and presumably widely-used research program was frequently dismissed as not “real” coding experience by prospective employers. What does it mean to say someone knows how to code? A bright six year old can program a “Hello World” program or, indeed, more complex examples of software. When technology executives say they cannot find applicants who can “code,” what does this mean?

In practice, the industry definition of “coding” is remarkably difficult to clarify. It may really mean “coding” by people I happen to like, such as people like me who tell me what I want to hear, for example, how smart I am :-). This is not a problem only encountered by new college graduates and freshly minted Ph.D.’s. The history of the technology industry is littered with the failures of companies that for some reason decided that their successful, widely-used, even highly profitable flagship product was not “good code” and needed to be rewritten from scratch, which turned out to be very difficult, even impossible. The code empirically worked, was written by professional programmers (they were, in most cases, paid and paid well to write the code), and it even made lots of money, but, well, it was “not done right.” Joel Spolsky wrote a famous, widely read, widely ignored blog post on this striking phenomenon in the software industry: Things You Should Never Do.

Just because you are Dennis Ritchie and your resume says you invented the C programming language, wrote the first C compiler, and a good chunk of the original Unix operating system (and your resume is actually true), doesn’t mean you can “code”. After all, Dennis Ritchie was (ick) a “research programmer” at an airy-fairy research lab (Bell Labs). Not real coding at all. And besides, he probably never used Git (a popular version control system that is notoriously difficult to actually use) a day in his life.

Incidentally, the standard way to get around Spolsky’s article is to call “rewriting” code “refactoring” code which makes it all OK, in fact a mandated part of many versions of the Agile software development methodology.

Opening Doors, Including Some You Should Not Go Through

In general, a Ph.D. closes many doors due to the widespread prejudice against Ph.D.’s in industry. But, it is true that a Ph.D., especially in some narrow, specialized areas, can open some doors that would otherwise be closed. A Ph.D. can help getting a job at a genuine corporate research lab such as HP Labs. It can help getting a job at some aerospace and defense contractors or government agencies that pay competitive, non grad-school, wages. A STEM Ph.D. will often help convince employers that you have strong mathematical skills, even if you don’t, even though your coding experience, which may be extensive, may be dismissed as “not real.” At present, around one percent of software development jobs could be described as highly mathematical, going beyond basic arithmetic and bookkeeping. In my opinion, this is likely to increase as we learn to use advanced mathematics to exploit the enormous processing power of present day and future computers. Many of the current highly mathematical jobs are concentrated in probability and statistics, often so-called “data science” where Chand John also unsuccessfully sought work.

In his article, Chand John expressed some frustration with his unsuccessful, and apparently lengthy, interview process with Palantir, the murky start-up funded by PayPal founder Peter Thiel and In-Q-Tel, the venture capital arm of the Central Intelligence Agency (CIA). The Palantir, by the way, are the cursed seeing stones used by the diabolical Dark Lord Sauron in J.R.R. Tolkien’s Lord of the Rings trilogy :-).

“They are not all accounted for, the lost seeing stones. We do not know who else may be watching.”
—Gandalf to Saruman, in The Fellowship of the Ring

The government, especially “members of the intelligence community,” has some truly strange projects, the true nature of which is disguised by secrecy and perhaps active disinformation. Some of these projects employ STEM Ph.D.’s. and pay well. One should think long and hard before going through some of these doors if they should open.

There are also some companies and projects that are based on grossly unrealistic ideas about what mathematics or science can achieve, especially on the short time scale, often a business quarter or two, favored by private industry. These do seek and employ Ph.D.’s but often end badly, for example when the planned breakthrough doesn’t occur by the next quarter. These companies and projects often suffer from a magical picture of mathematics or science similar to that portrayed in much popular science and in the more realistic-seeming science fiction (think The Hunt for Red October instead of Iron Man).

Concluding Remarks

Industry does express an interest in hiring STEM Ph.D.’s — loudly in certain contexts. In most cases, this interest appears to be feigned, primarily for political lobbying purposes. Whether this claimed interest is a deceit or a sincere delusion, most fresh STEM Ph.D.’s will find the Ph.D. more of a hindrance than a help when actually applying for a job in industry. Especially in the “technology” industry, in most cases they would have been better to stop at a bachelors or masters level. The Ph.D. often results in a lower salary than would have been possible if they got a job immediately after college or, in some cases, a master’s degree and worked for the same time period as their Ph.D.. Further, industry jobs for new college graduates in “technology” usually pay much better than grad school teaching assistant or research assistant stipends.

That said, a minority of employers are sincere in seeking STEM Ph.D.’s, either because they think they are smart or have other useful general characteristics, or, more often, because the Ph.D.’s have narrowly defined specific skills they seek. A Ph.D. may open some doors that would remain closed with only a B.S. or M.S.. On the other hand, a typical Ph.D. will close more doors than it opens due to a widespread prejudice against Ph.D.’s. Note that I don’t agree with this prejudice (after all I have a Ph.D.) but it is a part of the real world today.

© 2013 John F. McGowan

About the Author

John F. McGowan, Ph.D. solves problems using mathematics and mathematical software, including developing video compression and speech recognition technologies. He has extensive experience developing software in C, C++, Visual Basic, Mathematica, MATLAB, and many other programming languages. He is probably best known for his AVI Overview, an Internet FAQ (Frequently Asked Questions) on the Microsoft AVI (Audio Video Interleave) file format. He has worked as a contractor at NASA Ames Research Center involved in the research and development of image and video processing algorithms and technology. He has published articles on the origin and evolution of life, the exploration of Mars (anticipating the discovery of methane on Mars), and cheap access to space. He has a Ph.D. in physics from the University of Illinois at Urbana-Champaign and a B.S. in physics from the California Institute of Technology (Caltech). He can be reached at jmcgowan11@earthlink.net.