This is the third in a series of articles that started with The Mathematics of the Ph.D. Glut. This article discusses claims of STEM worker shortages in private industry and how specific work-experience based skills are confused, intentionally or not, with general science and mathematics taught in schools and universities.
One often encounters claims of a shortage of STEM (Science, Technology, Engineering, and Mathematics) workers in the United States. This often seques into arguments for more STEM education, teaching, students, and specifically Ph.D.s in these fields. Indeed, the current Great Recession has been blamed on a “skills gap” in which employers have allegedly been unable to find appropriately skilled employees despite a desperate need. The “skills gap” often seems to mean a shortage of STEM workers, although it seems to have spilled over into everything from machinists to cotton candy machine operators.
These STEM shortage claims have been strongly challenged by a number of academics including Norman Matloff, Peter Cappelli, Ron Hira, Paula Stephan, Hal Salzman and a number of others. They have also been challenged by practicing engineers and their families — most famously by Jennifer Wedel, the wife of an out of work semiconductor engineer, who confronted President Obama about the claims.
Purple Squirrels
It is important to understand a subtlety of STEM worker shortage claims. When high tech companies and their lobbyists claim there is a shortage of skilled high technology workers, they usually use general language such as “there is a shortage of engineers,” “there is a shortage of programmers,” or “there is a shortage of technology talent.” However, when pressed about seemingly highly qualified, often older workers who cannot find jobs, they refer to both extensive and very narrowly defined specific skills that they claim they must have. Older often means over forty or even over thirty-five.
There is actually a term in the human resources business for such narrowly defined, extremely difficult or impossible to find job candidates: purple squirrels. This term has been popularized by Google recruiter Michael B. Junge in his recent book Purple Squirrel.
It has been common for over twenty years in the computer industry to encounter large numbers of extremely narrowly defined job descriptions.
For example, a job ad may say something like:
Required
3+ years work experience developing iPhone game apps using XCode, at least two apps in the Apple app store
5 years Objective-C (the programming language for the iPhone)
3 years experience with CoreAudio (a particular iPhone software compnent)
3 years Agile software development (Agile is a software development methodology)
etc. (it is not uncommon to see lists of twenty requirements like this for one job)
Since iPhones have only been in existence and hot for a few years, there are very few if any software engineers who can meet these qualifications. Keep in mind in 2008 (four years ago), Blackberry was the King of the smartphone world. The iPhone was the “iWhat?”
These lists of requirements are often so long and picky that there may not be a single engineer in the world, including genuine iPhone experts, who would match them.
When high technology companies refer to shortages or to battles for talent they are almost always referring to this sort of extremely rare or nonexistent “engineer” with extensive work experience in dozens of very specific, often new, tools. It is almost always expressed as years of experience using a specific tool, programming language, etc. rather than a more general knowledge of the activity — such as science or mathematics. It is often like demanding that a car driver have experience driving a very specific make and model of car; we only hire experienced Volvo drivers!
Yes, there is a shortage of purple squirrels, almost by definition, but it has little to do with science, technology, engineering, or mathematics skills or education as most people would define them. It is probably impossible for colleges to anticipate fads such as the iPhone displacing the Blackberry. Schools and colleges do not provide work experience. In fact, traditionally, schools and colleges have sought to provide general foundational skills such as reading, writing, basic arithmetic, critical thinking, and so on that will remain valuable for the life of the student — with good reason.
The fixation on purple squirrels means that STEM careers are precarious and often short-lived. If you are a purple squirrel and you can find the one or two employers interested in you, you may do well for a few years. But what then? What if a new fad or fashion comes along?
A Personal Experience
Back in the 1990’s the author worked for a startup company that wanted to get into the then new, suddenly hot world of the World Wide Web and the Internet. This startup had essentially no in-house expertise on the Web; actually, direct experience with the Web and Internet was quite limited at the time. The startup was considering streaming digital video over the web in 1995 — a technically challenging project to say the least.
The author introduced the startup to Alistair (name and some details changed to protect the privacy of the people). Alistair was a Ph.D. in physics from one of the top universities in England. As an experimental particle physicist at the Stanford Linear Accelerator Center, he had been heavily involved in the early days of the World Wide Web which started out as a tool used by the particle physics community.
Alistair actually had several years of paid professional experience at a government research laboratory working on the Web, something almost no one could legitimately claim in 1995. His work involved creating browsers, servers, and other software tools, not just or even mostly creating web pages or web sites, although he did some of that as well.
Alistair was an accomplished software developer and had done a number of consulting jobs for private industry on the side. Alistair also had good communication skills and, in my experience, did not suffer from the notorious arrogance of physicists. He was a genuine purple squirrel.
The startup turned up their collective noses at Alistair, hired a much less experienced consultant who admittedly dressed better than Alistair, and spent several weeks and several thousands dollars fumbling around without success.
How real is a purple squirrel hunt? It is not uncommon to see genuine purple squirrels like Alistair turned down in a purple squirrel hunt in the Silicon Valley.
Why did the startup turn down Alistair and go with a much less experienced consultant who failed? In all seriousness, I suspect that Alistair’s casual dress contributed. The winning consultant wore a suit and Alistair dressed — well — like many of the engineers at the startup.
It is likely that the startup executives did not realize that the particle physics community developed the World Wide Web. Even though the core technologies in the Silicon Valley can be traced time and time again back to government sponsored research programs, many Silicon Valley executives believe the myth that these technologies were developed in garages by teenagers or in sometimes non-existent corporate research labs. Gordon Crovitz’s notorious recent op-ed piece Gordon Crovitz: Who Really Invented the Internet? Contrary to legend, it wasn’t the federal government, and the Internet had nothing to do with maintaining communications during a war. is a prominent example of these faith-based beliefs.
The Market is my Shepherd; I shall not want.
The Market maketh me to lie down in green pastures:
The Market leadeth me beside the still waters.
The Market restoreth my soul:
The Market leadeth me in the paths of righteousness for His name’ sake.
Yea, though I walk through the valley of the shadow of death,
I will fear no evil: For the Market art with me;
Thy rod and thy staff, they comfort me.
The Market preparest a table before me in the presence of mine enemies;
The Market annointest my head with oil; My cup runneth over.
Surely goodness and mercy shall follow me all the days of my life,
and I will dwell in the House of the Market forever.
Market-Based Version of the 23rd Psalm
The startup executives were looking for a software developer with at least three years of paid professional industry experience working for the fantasy private companies that invented the World Wide Web!
Many (not all) hot new technologies are developed by relatively small groups of people working for government sponsored research programs, often in universities or government research labs. When the technologies suddenly become hot like the World Wide Web in the 1990s, of course, there is a “shortage” of people with direct experience. That is inevitable. In many cases, the few people with direct experience will not have industry experience, narrowly defined.
What does this “shortage” have to do with STEM education? Absolutely nothing.
Leprechauns
Laurent Bossavit’s recent book The Leprechauns of Software Engineering discusses several widely repeated and widely accepted shibboleths of software engineering, tracing them back to the few original studies, research papers, and, in some cases, unsubstantiated claims that they are based on. In every case that he discusses, the supporting evidence is very weak or nonexistent, hence “Leprechauns.”
Two chapters deal with the concept of the “10X programmer,” a widely-held belief in the computer industry. In its most extreme form, the “10X programmer” belief holds that there are superprogrammers who can write code ten times faster and better than the typical or average experienced programmer (NOT the worst programmer). This idea has been promoted by software engineering consultant Steve McConnell and Bossavit takes aim at McConnell’s supporting data in two chapters and a lengthy appendix detailing his hunt for the primary references and data on the 10X claim.
An obvious problem is that salaries for experienced software developers do not vary by a factor of ten for the vast majority of software developers including those with strong, impressive credentials. Rather the variation is more like a factor of two or three at most.
Bossavit found numerous problems with the 10X claims including the difficulty defining software productivity. In many cases, 10X referred to the difference between the best and worst programmers, in some cases studies of undergraduate computer science majors not professional experienced programmers. Some studies referred to differences in seeming productivity between different projects/teams, not broken down by individuals.
A widely reported claim of a 28:1 difference in a 1968 study referred to an apples to oranges comparison between one programmer coding a task in a high level language on a time sharing computer (like a modern computer) versus a different programmer coding the same task in low level machine language (zeros and ones) on a batch processing system (punch cards probably).
Although Bossavit found some evidence for the 10X claim in one detailed NASA study, there were significant issues in interpreting the NASA data that make even this one study open to serious debate. The study found much higher variations between programmers in smaller projects than longer projects which is difficult to understand; the study attributed this to less experienced programmers being assigned to the shorter/smaller projects. The bottom line is that the evidence behind the 10X belief is quite limited and subject to alternative interpretations.
That does not mean that companies and hiring managers don’t believe in 10X programmers (or scientists or mathematicians) and try to find them, repeatedly passing up seemingly highly qualified candidates. They may be hunting for non-existent or exceptionally rare “leprechauns” instead of the real 1X, 2X or at best 3X programmers, scientists, engineers, or mathematicians.
Could such exceptional people, if they exist at all, be due to STEM education? Perhaps, but such a large variation may have a genetic or unknown organic cause (eating lots of spinach?). In fact, flooding STEM fields with non-10X people, a logical consequence of hyping STEM education, may make it much harder to find these rare people even if they exist, much like SPAM making it difficult to spot serious e-mails from people that you do not know. Flooding STEM fields with non 10X people may even drive the rare 10X people (if they exist) out of the STEM fields altogether.
Should Kids Learn Math and Science?
Absolutely. In the modern world, essentially everyone needs to use basic arithmetic to buy food and other items, balance their checkbook, and perform other activities of daily living. We all use math every day. People need to understand raising numbers to powers to properly understand interest on savings accounts and loans. In the housing bubble, many people were bamboozled by complex mortgage terms and formulas. Algebra, trigonometry, and geometry — more advanced high school math — does actually turn up in various real world activities including building and architecture, mechanical design, navigation and other activities.
With the spread of computers into every home and device, the use of mathematics including quite advanced mathematics is becoming commonplace. Most people are now using sophisticated mathematical programs, often without realizing it: video compression such as YouTube and Skype, the GPS system which tells people where they are, and computer generated imagery, for example.
Complex mathematical models are increasingly being invoked in public policy such as the models underlying claims of “global warming.” In particular, a good understanding of probability and statistics at the college level is needed to understand and evaluate claims about expensive medical treatments such as chemotherapy for cancer. Should you rely on the claims of “experts” who often have conflicts of interest or do the calculations yourself?
However, while most kids should learn mathematics, should they get a Ph.D. in mathematics or pursue a career in mathematics? Most kids learn English, reading, and writing, but few become authors or newspaper writers, for example. Most kids play sports and many enjoy one or more sports, but few become professional athletes.
There are some jobs in the private sector that employ heavy mathematics, especially statistics. Some of these jobs pay quite well. However, they are few and far between, and often highly specialized. Probably less than one percent of present day software jobs are highly mathematical. The vast majority of software jobs utilize basic arithmetic heavily — essentially some form of bookkeeping — and little else. The competition for the rare highly mathematical jobs is fierce.
When private companies complain about shortages of STEM workers and specifically people with mathematical skills they are generally referring to difficulties finding specialists in a few narrow areas to fill very narrowly defined jobs: the purple squirrels.
Some companies such as some Wall Street investment firms are seeking or believe they are seeking truly exceptional scientists and mathematicians, e.g. the rare graduate student who has just received the Fields Medal for his dissertation on stochastic differential equations — the best of the best of the best. One can think of trading on Wall Street as a zero-sum game, a winner-take-all tournament in which even a tiny advantage can translate into vast returns. There is always a shortage of the best in the world. Only one athlete wins the gold medal at the Olympics in a given sport. Only one player can win the World Chess Championship.
Billionaire Land
STEM shortage claims can usually be traced back to blue ribbon panels, commissions, or private groups of corporate CEO’s and “power scientists,” politically active Nobel Laureates, major research university presidents, and similar academics. The current STEM mania can be traced back to the 2005 Rising Above the Gathering Storm report, chaired by former Lockheed CEO Norman Augustine.
Norman Augustine, for example, certainly seems sincere when he talks or writes about science and mathematics education.
Thirty, forty years ago, at least according to most media reports, there were only a handful of billionaires in the United States: Howard Hughes, Daniel Keith Ludwig, and perhaps one or two others. These men had fortunes that reportedly barely exceeded one-billion dollars. The number of billionaires has soared in the last forty years, as has the number of people with fortunes in the hundreds of millions of dollars.
These personal fortunes are so large, that most billionaires and centimillionaires probably have to live like ancient Roman Emperors, surrounded by bodyguards and handlers 24/7. Like Howard Hughes who may have become a prisoner of his mysterious personal security force, these folks live in a very different world even from the small rich with a few million dollars. Who in their right mind would disagree with a billionaire other than another billionaire? Not too many people.
When a billionaire CEO talks about a shortage of STEM “talent,” he or she may be sincere but they are often talking about something very different from common English usage. If, for example, one is having trouble hiring the next Nikola Tesla, a very unusual person, or even Thomas Edison, or at least one’s preconceived notion of what these people would look like in a modern resume, this is really very different from what a shortage of scientists or mathematicians means to most people, including students with strong or even exceptional scientific or mathematical talent.
Conclusion
STEM worker shortage claims in most cases refer to difficulty finding job candidates with extremely narrow, often lengthy job specifications, almost always several years of paid professional industry experience in specific tools or narrowly defined methodologies: purple squirrels. In some cases, the shortage claims probably refer to searches for truly exceptional candidates who may not exist: leprechauns. There is little relationship between these hiring difficulties and STEM education at the school or college level.
In most cases, companies can find job candidates by being less picky and not hunting for impossible or extremely difficult to find purple squirrels, especially in the current depressed economy.
Companies and hiring managers should also take a hard look at the actual evidence for 10X programmers or other varieties of supermen. Such people are certainly extremely rare and their existence is debatable. No one is going to find an actual leprechaun.
© 2012 John F. McGowan
About the Author
John F. McGowan, Ph.D. solves problems using mathematics and mathematical software, including developing video compression and speech recognition technologies. He has extensive experience developing software in C, C++, Visual Basic, Mathematica, MATLAB, and many other programming languages. He is probably best known for his AVI Overview, an Internet FAQ (Frequently Asked Questions) on the Microsoft AVI (Audio Video Interleave) file format. He has worked as a contractor at NASA Ames Research Center involved in the research and development of image and video processing algorithms and technology. He has published articles on the origin and evolution of life, the exploration of Mars (anticipating the discovery of methane on Mars), and cheap access to space. He has a Ph.D. in physics from the University of Illinois at Urbana-Champaign and a B.S. in physics from the California Institute of Technology (Caltech). He can be reached at jmcgowan11@earthlink.net.
References/Suggested Reading
Purple Squirrel
By Michael B. Junge
Why Good People Can’t Get Good Jobs: The Skills Gap and What Companies Can Do About It
By Peter Cappelli
The Leprechauns of Software Engineering
The Leprechauns of Software Engineering: How folklore turns into fact and what to do about it
By Laurent Bossavit
Antitrust
A very fun, very unrealistic movie with Tim Robbins, Ryan Phillipe, and Claire Forlani that depicts the classic mythology of teenagers in a garage developing breakthrough technologies in the Silicon Valley.
Hidden in Plain Sight: The Secret History of the Silicon Valley
Serial entrepreneur and high technology startup expert Steve Blank’s presentation on the role of the government, especially “members of the intelligence community,” in the Silicon Valley.
“Purple Squirrel” job postings can arise when a US company wants to hire someone who needs a visa (e.g. an Australian citizen). As I understand it, the US company is required by law to post a public ad, so that qualified American citizens can also apply. So what does the company do? They create a job description that matches exactly one person: the person they want to hire.
A company I worked for did exactly this to justify keeping several immigrant workers. Back when H1-Bs still required “proof” of no qualified domestic workers in the 1990’s (before the 1998 loopholes), I was subject to two identical surreal interviews by Microsoft asking about my birthplace, citizenship, and the like., but completely ignoring my professional qualifications. I was stunned at how blatant it was. Back then, the laundry list ads so common now were a dead giveaway of such sponsorship; definitely not worth applying to.
This is Steve McConnell’s reply to an earlier version of Laurent Bossavit’s criticism of his 10x programmer citations and data:
https://forums.construx.com/blogs/stevemcc/archive/2011/01/09/origins-of-10x-how-valid-is-the-underlying-research.aspx
I agree that employers have been needlessly selective — but one wonders why the market hasn’t cleared and why consulting wages for contract workers remain relatively low at $60-$65/hour for programming work in the financial industry on the East Coat. I’ve seen it myself–we’re getting on old chap.
Your citation of Gordon agrees with my comments in previous postings!
First, the productivity growth slowdown around 1970:
“This account of the role of IR #2 and IR #3 share the common feature that many of these transformations could only happen once. Figure 4 is a bar chart showing the average growth rate of U.S. labor productivity over four time intervals: 1891-72, 1972-96, 1996-2004, and 2004-2012.
These intervals are chosen to reveal the contributions of the industrial revolutions. IR #2 and its subsidiary developments were able to keep productivity growth going for 81 years
between 1891 and 1972. It is puzzling that all the benefits of the computer enumerated above did not prevent the significant productivity growth slowdown by half from 2.3 percent per year during 1891-72 to only 1.4 percent per year during 1972-96. My interpretation is that the spinoff inventions from IR #2 explain most of the rapid growth of productivity between the 1890s and 1970s, and that diminishing returns to the benefits of these inventions was the basic cause of the post-1972 productivity growth slowdown.”
Next, I enumerated four headwinds, but Gordon enumerates six. I’ll mentions the headwinds that coincide:
A. The influence of the womens’ movement:
“(1) The “demographic dividend” is now in reverse motion. The original dividend was another one-time-only event, the movement of females into the labor force between 1965 and
1990, which raised hours per capita and allowed real per-capita real GDP to grow faster than output per hour. But now the baby-boomers are retiring, no longer included in the tally of total hours of work but still included in the population. Thus hours per capita are now declining, and any tendency for life expectancy to grow relative to the average retirement age will further augment this headwind. By definition, whenever hours per capita decline, then output per capita must grow more slowly than productivity.”
B. This one you doubted: that the productivity gains of the 70s went to the upper 1/10th of 1%. What say you now? You trust Gordon, and so do I. Following Kahneman, who said, You have to recognise that people will accept scientific conclusions from people that they trust, not from anybody. Arguments and evidence are much less important than trust.” no argument from me will be persuasive and I doubt that I could ever earn your trust. I have only to cite your own reference:
“The most important quantitatively in holding down the growth of our future income is rising inequality. The growth in median real income has been substantially slower than all of
these growth rates of average per-capita income discussed thus far. The Berkeley web site of Emmanuel Saez provides the startling figures. From 1993 to 2008, the average growth in real household income was 1.3 percent per year. But for the bottom 99% growth was only 0.75, a gap of 0.55 percent per year. The top one percent of the income distribution captured fully 52% of the income gains during that 15-year period. If what we care about when we talk about
“consumer well being” is the bottom 99 percent, then we must deduct 0.55 percent from the average growth rates of real GDP per capita presented here and elsewhere.”
Curiously, Gordon notes this gap, but does not relate it to another finding in his own report (see above where Gordon attributes diminishing returns to productivity, but does not include rising the rising inequality facilitated by increases in productivity–executives paid themselves more. Where exactly does Gordon think the income disparity came from? He writes, ” It is puzzling that all the benefits of the computer enumerated above did not prevent the significant productivity growth slowdown by half from 2.3 percent per year during 1891-72 to only 1.4 percent per year during 1972-96.”
Globalization, outsourcing:
(4) The interaction between globalization and ICT is a daunting headwind. Its effects include outsourcing of all types, from call centers to radiologist jobs. Foreign inexpensive labor competes with American labor not just through outsourcing, but also through imports. And these imports combine lower wages in emerging nations with growing technological capabilities there. This is nothing more than the Hecksher-Ohlin-Samuelson factor-price equalization theorem at work, and it inevitably has a damaging effect on the nations with the highest wage level, i.e., the United States.”
Gordon does not mention immigration.
Why are none of your hypotheses mentioned: the Ph.D. glut is responsible, and the relative lack of development in propulsion and energy systems compared with computers and electronics? I have some overlap, at least.
You definitely did not read Gordon’s paper, at least not in its entirety. For the record, Gordon mentioned and even advocated for immigration of “high-skilled workers” (pg 21, 2nd – 4th paragraph). He even went as far as suggesting that “open immigration” could be beneficial to the US as it did prior to 1913.
I agree with most of Gordon’s facts but not with his diagnosis of the problem.
https://av.r.ftdata.co.uk/files/2012/08/IS-U.S.-ECONOMIC-GROWTH-OVER-FALTERING-INNOVATION-CONFRONTS.pdf
Even at the factual level, I disagree with his interpretation of what he calls IR#1 and IR#2. In particular, the steam engine is not a single invention; there were a series of generations of increasingly more efficient and powerful steam engines up to the turbines used in power plants today.
In IR#1, you have
1) the Newcomen engines
2) the separate condenser engines, differing from the Newcomen engines by a radically new component. (the Watt-Boulton steam engines)
3) high pressure steam engines which are quite different from the Watt-Boulton separate condenser steam engines and much smaller and more powerful.
What he calls IR#2 includes
1) generations of internal combustion engines
2) jet engines (major impact in 1960s)
3) rocket engines
4) atomic/nuclear power plants (underutilized at present)
and some other technologies.
It was not just the invention of a practical internal combustion engine in 1879 (the idea had been around for generations).
There was a series of revolutions in power and propulsion from the early 1700s (Newcomen engine) to 1970. As each technology topped out, people were able to devise new, better engines and power sources, fueling long term growth of the per capita GDP.
See this article:
Of Flying Cars and the Declining Rate of Profit
By David Graeber
https://www.thebaffler.com/past/of_flying_cars/print
for a somewhat different interpretation that overlaps some of my concerns with modern scientific research.
For example,
Yet most observers agree that the results have been paltry. Certainly we no longer see anything like the continual stream of conceptual revolutions—genetic inheritance, relativity, psychoanalysis, quantum mechanics—that people had grown used to, and even expected, a hundred years before. Why?
Part of the answer has to do with the concentration of resources on a handful of gigantic projects: “big science,” as it has come to be called. The Human Genome Project is often held out as an example. After spending almost three billion dollars and employing thousands of scientists and staff in five different countries, it has mainly served to establish that there isn’t very much to be learned from sequencing genes that’s of much use to anyone else. Even more, the hype and political investment surrounding such projects demonstrate the degree to which even basic research now seems to be driven by political, administrative, and marketing imperatives that make it unlikely anything revolutionary will happen.
Sincerely,
John
amazing research I can sense in this article, but I’m afraid all of this is much too “mathematical”. I mean, the previous articles relied on something called “GDP”, but it isn’t explained how the GDP of a country gets calculated. on a site on that topic mentioned in the first article they do give a rough definition, but don’t explain any further. to me this seems like a misinformation campaign. as some comment there complained, nowhere the article-series explains why GDP-growth would be correlated with growth of knowledge. and as I say in the beginning of this paragraph, all that is excusable if the author is just a mathematician, it’s just a model someone else provided and with that model some mathematics is done.
the problem is just that this model has some problems. the major being that resources are not counted in. for example efficiency in making money out of raw resources would much better depict modern development of technologies. when I were young I played computergames like Civilization or its outer-space equivalent. there one particular invention created a big jump in productivity: mass production. the next jump of similar scale is expected with the invention of actual robotics, autonomous production. so knowledge may rise to whatever heights, but actual production wont become better. additionally, when there’s nobody to buy those things produced, the production definitely cannot grow.
But I share with the author the feeling that somehow technology isn’t growing as quickly as it used to. however, I see a reason for it in higher formalism and more tense research-regulations. i.e. I blame the “need” for wearing a radiation-proof overall when dealing with radioactivity, I blame that for a lower speed in research of these topics. but seriously, science has become much more complex now than it was in past, and for this reason people are not that willing to actually do any science regardless of their Ph. D. title. true, would there be less Ph.D. people then this problem would solve itself, with many good knowledge being lost forever and nobody telling you to actually wear whatever protective gear.
please take a look at an article I wrote in my own blog:
https://ganderblog.wordpress.com/2012/08/30/economy-from-a-mathematical-pov/
and here we come to the actual article here, STEM shortages. a problem the author did not consider is that when talking of globalization it is “enough” to consider the whole world as being a single country. it is a problem because data for other countries is not available or maybe only available in their respective language. and it is a problem because the STEM shortage could be such a global phenomenon and one would need to take into account how much PH.D. exist in the whole world, and the percentage of that global number who happen to be in the usa. there are countries where about 50% of the population have a Ph.D. (communism had such goals), how high is that percentage in the usa? in my (little) experience, politicians or patriots talking of some “shortage” is much alike to a person who owns a little house with garden in the suburbs and who then keeps on complaining that the neighbour has a longer car. and more seriously, it’s alike to some european governments complaining their country is lacking some little (unimportant) details which coincidentally can be found in the european country which has the best pisa-results. I really don’t take such media all that seriously, even less I consider “filling up” that shortage with my own person. I am listening only to myself when it comes to education, I learn what I like learning, and if everyone would do just that then the statistically even distribution of people’s desires would make sure that also workforce would have evenly distributed education in general. it’s just the media which mess up such probability-distributions by creating short-lived hypes.
purple squirrels are not just formed by governments trying to legally hire some particular person, economy as a whole is corrupt and many jobs are given only to friends and relatives. similarly the leprechauns are not the invention of idiots but rather of smart people who want to convince their idiotic boss that their brother in law is the best choice for the job.
all this has nothing to do with the Ph.D. shortage/glut. more relevant I think is the behaviour of all these Ph.D. in their search for jobs. when they read that a 21year old Ph.D with 15 years of experience is needed, they don’t even dare to apply for the job! additionally most Ph.D. will believe their knowledge is obsolete and therefore they wont consider themselves as Ph.D. after about 10 years have passed. and ironically none of them actually gets the idea to keep on studying after the Ph.D. to evade such feeling inappropriate. in other words we have the “shortage” of STEM because nobody admits being STEM, because no STEM people actually look for a job in their profession. of course for a few years they do, but about 10 years later they give up regardless of whatever hype making them attractive again.
another interesting number from economy is the fluctuation-rate: how much percent of workers stop working for that company each year? especially when talking of unemployment and iPad programmers with Ph.D. these numbers should be looked at. they represent how unhappy the workforce is with the company. if a 1000 person company has exchanged 30% of their whole workforce in a year, there’s those 300 unemployed former workers which wont count as Ph.D. for that company, and thereby an artificial shortage is created. and if the company then wants to return to status quo they need a purple squirrel who fits the description of the person who just left the company. reduce that percentage and you reduce the shortage while also improving efficiency (since a single scientist will be much better for some project than a group of scientists where 30% leave the project each year to be replaced by newcomers). how fast does google grow? how many workers did stop working for them last year? see any correlation when compared to other companies?
finally about mathematics, it is true we don’t need Ph.D. degree for everyone. but as I see it, just about everyone needs a bachelor in mathematics nowadays, since middle-school is not even enough for understanding basic economy and what is called “derivate” at the stock market. technology has become more complicated, especially in the area of statistics and probability. in middle-school I have never learned to make use of matrices and infinite series (except power-series and arithmetic series), but in order to use even the mentioned Octave, you need that knowledge. a company which wants to be successful needs mathematics as a means to predict the market, a single person in the risk-management department wont do, people would still need higher mathematics to understand what this person is saying and to heed his warnings!
A couple comments from the Author.
1. The Gross Domestic Product (GDP) is computed by the Bureau of Economic Analysis (BEA), a part of the US Department of Commerce. I downloaded the BEA data from the repository of economic data at the St. Louis Federal Reserve.
The BEA
https://www.bea.gov/
FRED (Federal Reserve Economic Data)
https://research.stlouisfed.org/fred2/
Some people question the way the GDP numbers are gathered and calculated by the BEA, suspecting economic growth in recent decades is lower than the official GDP numbers, which would only strengthen my points. For example,
https://www.shadowstats.com/article/gross_domestic_product
2. On propulsion and power technologies.
Robert Gordon’s article discusses some of this at length which is why I added a link to it. Just to repeat.
https://av.r.ftdata.co.uk/files/2012/08/IS-U.S.-ECONOMIC-GROWTH-OVER-FALTERING-INNOVATION-CONFRONTS.pdf
In particular, power and propulsion technologies contribute heavily to per capita GDP and our standard of living. It is not just the many things we can see: lights, power for computers and electronics, refrigerators, heating, air conditioning, our car or other transportation, etc. Almost every item in a home as well as the home itself is built with machines and power tools. We have running water and treated water because we have pumps and filtration or sterilization systems that require generated power.
Without advances in these areas, growth will falter as it appears to be doing.
3. The Remarkable Ineffectiveness of Producing Large Numbers of Ph.D.’s
Both in general, and specifically with respect to Physics, the production of large numbers of Ph.D.’s, demonstrably far more than there are jobs, over the last forty years has quite clearly failed to translate into advances in power and propulsion technologies comparable to previous decades.
4. The Remarkable Ineffectiveness of Computers in Accelerating Power and Propulsion R&D
Another remarkable thing is that power and propulsion fields were early adopters of computer software and tools such as NASTRAN (https://en.wikipedia.org/wiki/Nastran), many simulation programs in physics, MATLAB, and so on. However, these tools have visibly failed to accelerate progress as many hoped and expected; actually progress appears slower than during the era of pen and paper and slide rules.
5. The Remarkable Ineffectiveness of Huge Budgets
We have spent tens, if not hundreds of billions of dollars on power and propulsion R&D in the United States since 1970 with remarkably limited practical results. This money has translated in to large numbers of Ph.D.s, large numbers of academic publications, a plethora of R&D modeling and simulation programs like NASTRAN, petabytes of data, and other impressive quantitative measures, but the practical benefits have been quite limited (yes, we have better batteries, solar cells are a little better but still seem far from commercial viability, tracking, etc.)
The low growth rates of the per capita GDP very probably reflect this lack of progress. Yes, there are other issues such as inequality, but our raw ability to produce more per person simply has not increased much in the last forty years.
Sincerely,
John
if GDP per person was used, then it’s even worse a model than I thought. maybe one should calculate GDP per Ph.D.? 🙂
it’s companies and various organizations which increase GDP, not people! why divide the GDP number by the population-numbers? if your county is close to syria, and government decides the neighbour’s fleeing population will henceforth be population of this county, would GDP suddenly make a jump downwards? what about a baby-boom? does doubled population mean halved economy?
joke aside. GDP is good enough for estimating a country’s social state, the higher the richer the people — well, some people at least. power and propulsion might be important for GDP, but what do they have to do with sketching out the situation with research and development? this is the major gap in these articles! there are lots of mathematicians working for big bank companies, to increase their income, to optimize the only thing that really matters for most people. but do they increase GDP? their way of making money is not related to whatever physical products, they even aren’t paying taxes for that! if you were a mathematician at the beginning of your education, would you choose to research and develop energy and propulsion, or would you choose to earn lots of money instead? in economy there’s waiting much more than those few hundred billions put into R&D over the length of many many years! the point is that in order to research plasma and electricity you need lots of money to even get started, there’s no chance to do that without government! well, I really don’t know if this really is true, but urban legend says it is, and therefore nobody dares to fund such research on their own. truth is that in matters of electricity and powerplants, there has already been quite fruitful research, but in the whole world there is noone willing to do the development (because such a person would then become enemy to the established and politically active companies that currently ave their powerplants running). for example it would be possible to save on energy by transporting electricity in direct current instead of alternating current. just replace the old power-lines. usa powerlines have been said (in near past) to be likely to fail soon. why hasn’t anything happened in this area? is there even any mention of that relatively new technology in the media? also, europe is building a first attempt at a fusion-power-plant, is there anything alike in the usa? and finally, there’s a swiss company which claims of being able to build solar panels into just about everything made of plastic, be it mouse or laptop or whatever. couldn’t it be that the current state-of-art in power-research is more about power-saving than power-production?
as for computers in research, I would explain the failure in lack of actual software-designers. a computer is no black box that offers all the answers, it is only as smart as its programmers were. in my experience with artificial intelligence (I visited lectures and read papers and such, in hope to write a program), there really is no actual research in that area, never have been! what people research instead is how to make computers do what we cannot, and some might also research how to get computers to parrot human behaviour. but have you ever seen any paper on the question of how to take a neuronal network and translate it into a combination of common mathematical tools, tools which commonly get branded under the name “AI”? if a neuronal network fails, how do you search for bugs, how do you figure out what it still needs to learn? instead there’s plenty of claims that this or that AI does the same things a neuronal network would do. why isn’t there research into neuronal networks anymore? are we so afraid of all those Terminator fantasies? of course not, I’d say it’s just because of that urban legend on some computer-simulated neuronal network failing to predict the fly’s flight-path…
More on Purple Squirrels at the New York Times:
Skills Don’t Pay the Bills
https://www.nytimes.com/2012/11/25/magazine/skills-dont-pay-the-bills.html?hp
John
I too find the 10x claims suspect, unless you are measuring newbies against experts, which is meaningless. But I’ve even seem claims of 25x, 100x, 1000x! Clearly preposterous. Reminds me of the popular 1970’s myth that people used just 10% of their brains capacity. Why would evolution generate such wasteful overcapacity there but nowhere else?
I’m also not surprised that companies routinely turn down obviously awesome experts like your buddy Alistair. I’ve seen it happen myself. Companies claim they want the best, but at the same time unquestioning robotic obedience, which tend to not correlate.
The 10x thing is similar to the more common A/B/C player model that presumes there are superstars that will do well anywhere, anytime. Which ignores that performance is very context dependent. As we’ve seen with some rockstar CEO’s that failed miserably after switching companies.