A recent widespread custom in the high technology industry is the “coding interview” where applicants typically stand in front of a white board for three or more hours answering questions about data structures and algorithms typically taught in college-level computer science (CS) programs: sorting algorithms, searching algorithms, estimating the Big O order of the algorithm, implementing linked lists, trees, and hash tables.
The coding interview has become so common and pervasive that it has spawned a cottage industry of books, courses, and coaches, most notably Gayle Laakmann McDowell’s Cracking the Coding Interview (now in its 6th edition) known among cognoscenti by the acronym CTCI. Competitors include Programming Interviews Exposed: Secrets to Landing Your Next Job (3rd edition) by John Mongan, Coding Interview Questions by Narasimha Karumanchi, and dozens more.
The coding interview has come under increasing fire recently with many critical blog posts and a high-profile critical tweet by Ruby-on-Rails Creator David Heinemeier Hannson.
Hello, my name is David. I would fail to write bubble sort on a whiteboard. I look code up on the internet all the time. I don’t do riddles.
The Problem with Algorithm Interviews
As someone who researches and develops algorithms with a high mathematical content professionally I can say that these algorithm interviews do not generally test skills used in professional algorithm development let alone many common software development tasks such as user interface development, most web sites, and most database software development.
Data structures such as linked lists, trees, and hash tables are solved problems in computer science. Sorting and searching algorithms are also largely solved problems in computer science. The mantra in software development is code reuse, widely used as a pretext for rewriting and reinventing working code written by other developers. Your code is not maintainable (reusable), hence I must refactor it (rewrite it from scratch) rather than reuse it.
However, for those of us who actually practice code reuse, many libraries and example source code snippets exist for all of these data structures and algorithms. Many of these can be found, if needed, by using Google or other search engines. More to the point, the C++ Standard Template Library (STL), for example, contains templates implementing nearly all of these.
The algorithm interviews thus tend to discriminate against experienced developers who know what they are doing and are long past CS 101 or CS 201. They tend even more to discriminate against “self-taught” developers who may never have taken CS 101 or CS 201. The 2015 Stack Overflow Developer Survey found that forty-eight percent of respondents did not have a CS degree and 41.8 percent described themselves as “self-taught.”
So called self-taught software developers are typically people with non-CS STEM (Science, Technology, Engineering, and Mathematics) degrees who learned software development by doing in these non-CS fields, some of which such as Physics were using computers and developing cutting edge software and algorithms before CS even existed as a degree. Those self-taught software developers who originate outside of a STEM field or lack a college degree probably tend on average to be at an even greater disadvantage.
Cracking the Coding Interview
To be sure, experienced software developers with free time can get Cracking the Coding Interview or similar texts and spend months practicing these non-representative algorithm problems. That is exactly what many do.
About two years ago, I assisted a friend and colleague with transforming from software development in the rareified world of High Energy Physics to what is now being called “data science.” Of course, he had all the requisite and generally hard to find skills. I had however to recommend that he get Cracking the Coding Interview and practice for several months. We also set up several mock interview sessions where I bombarded him with the arcane algorithm and brain teaser questions that currently dominate the interview process.
Not surprisingly, he did find a good job after several months of looking and jumping through these bizarre hurdles. But, practicing these algorithm questions is time consuming, tedious, and a significant hardship for those experienced scientists and engineers with families and other post-college obligations. It also takes away from time spent learning or practicing skills actually used on the job.
The Problem with Job Interviews
In my experience, job interviews of any type are not very reliable — for either the employer or the potential employee. Job interviews are a high-stakes test or examination where both the potential employer and employee have a strong incentive to put their best face forward and avoid potential problem areas. Many books like Cracking the Coding Interview, courses, coaches and consultants, and other resources exist for both employers and employees to learn to present themselves well in a short encounter that is generally not representative of the actual work and work environment.
There is probably a weak positive correlation between the impression formed in a job interview and what will actually happen, but it is weak. There are many cases where the job interview seems great and the job is a terrible fit. Indeed, there are a fair number of cases where the job bears no resemblance to the job seemingly described in the job interview.
Modern algorithm interviews focus almost entirely on non-representative questions about technical skills, largely avoiding discussions of what the job is, what the potential employee wants to do, what the potential employee has done in the past, and other relevant “touchy feely” issues. While a more rounded interview process would probably be more effective and somewhat better correlated with results, such interviews were common in the 1990’s and early 00’s and still had a high failure rate for both employers and employees.
Imperfect Alternatives
While portfolios of software with example code can demonstrate specific expertise, it can be difficult to verify that the potential employee actually developed the code. Many full time employees and contractors cannot build up a representative portfolio to show potential employers because of secrecy and confidentiality requirements. Apple, to give a notorious example, is extremely secretive. Although an extreme example, it is hardly unique among employers.
A trial consulting project (paid) lasting a few weeks to a few months is probably more reliable than a simple job interview, and many have suggested these trial projects as a possible solution to hiring difficulties. However, even a trial project is not a perfect test for many issues. Some problems take longer to manifest than a few months. It is still possible for a trial project to be non-representative of the bulk of the work, starting after the trial project is completed. Generally, a conventional job interview precedes a trial project.
Trial projects are fine for consultants with good finances and a good market for their skills, but clearly don’t work so well for full time employees, for example with families that need support and attention.
Conclusion
The current CS 101/ CS 201 algorithm interviews that spend most of the interview on technical questions about data structures and algorithms such as linked lists and sorting algorithms are not a good test either for general software development or even for most professional algorithm development. Professional algorithm developers typically reuse code or libraries with these data structures and algorithms rather than reinvent them. The widely used C++ Standard Template Library (STL), for example, includes most of these. Many advanced algorithms such as video compression, speech recognition, neural networks, and many others in fact make little or no use of these textbook algorithms.
About the Author
John F. McGowan, Ph.D. solves problems using mathematics and mathematical software, including developing gesture recognition for touch devices, video compression and speech recognition technologies. He has extensive experience developing software in C, C++, MATLAB, Python, Visual Basic and many other programming languages. He has been a Visiting Scholar at HP Labs developing computer vision algorithms and software for mobile devices. He has worked as a contractor at NASA Ames Research Center involved in the research and development of image and video processing algorithms and technology. He has published articles on the origin and evolution of life, the exploration of Mars (anticipating the discovery of methane on Mars), and cheap access to space. He has also worked for a number of startup companies and Apple. He has a Ph.D. in physics from the University of Illinois at Urbana-Champaign and a B.S. in physics from the California Institute of Technology (Caltech). He can be reached at jmcgowan11 AT earthlink.net
Excellent article. I agree with you. Another alternative is: the employer prepare some specification/scenario/problem closer to their environment to evaluate a candidate. Give a week to solve the problem, and request to candidate to explain the solution. IMHO, it’s a very effective form of problem solving evaluation.
Just to be clear, in response to some comments I have received, I am specifically critiquing coding and algorithm interviews that focus on CS 101/201 type questions such as implementing various sorting algorithms, searching algorithms, etc. Generally the types of questions that Cracking the Coding Interview (CTCI) and similar books cover.
I have both endured and inflicted such interviews.
There are also for example “coding interviews” that focus on specific methods and problems actually used on the job. Although there are some issues with these as well, they are not the subject of my criticism, nor are various other technical interview techniques that differ substantially from the CTCI type interviews.
There are many subjects taught in colleges and universities, that are “good to know,” and helpful in building a solid foundation for a life and career, but are simply not used on the job. Many professional writers have English or English Literature degrees, but testing a professional writer on his or her knowledge of Shakespeare is not a good way to identify good writers and will discriminate in favor of new or recent college graduates who have just taken Shakespeare, those few writers who actually write about Shakespeare, and those who have sufficient free time to study and practice Shakespeare for the test.
Many advanced algorithms that are under active development in academia and industry do not use or make limited use of college textbook algorithms and data structures. Where these are needed, it is often faster and more efficient to practice code reuse and use libraries such as the C++ Standard Template Library (STL) or available code snippets, rather than reinvent the wheel. Thus, while many professional algorithm developers know in general terms what a hash table or linked list is, know loosely how some sorting or search algorithms work, this is not what they work on the vast majority of time. If they need to know, they look up the answer.
Video compression, for example, uses the Discrete Cosine Transform (DCT), run length encoding, various forms of entropy coding (Huffman codes etc.), filters, two and three dimensional arrays, and various moderately complex operations on these arrays (motion compensation and estimation). There is negligible use of sorting, linked lists, etc. Nor is this unusual among advanced algorithms.