@Rivet2000 – Exactly! You can’t simply rely on a resume or how well they can talk about things. You have to see how they think on their feet as a programmer. At the company I worked for in the 1990s, candidates would go through a series of 1-on-1 interviews during the day (IIRC, 4-5). The candidate would come to your office and you’d send feedback to the other interviewers after they left. I wasn’t a CS major so I didn’t ask the CS non-code trick questions (the answers or the methods to solve them can be memorized). I would start with simple code problems to loosen them up then progress to more involved problems. None were difficult or tricky. I wanted to know if they could “see” the problem and come up with a decent implementation.
Two interviews were memorable. One guy came in with a decent resume and good “talkies” (I think I was the third interviewer), so I asked the simple code questions. He took forever and every implementation was ridiculously convoluted. Definite no-hire! On the opposite extreme was a guy who would practically start writing his answer before I’d finish stating the problem – and his solutions were great.
"The problem is that writing algorithms on a whiteboard has almost nothing to do with modern software development.
In real life, you would rarely just whip up an off-the-cuff algorithm from memory in the middle of a coding session. You are almost always going to use an existing library, which has its own test suite, and has survived the scrutiny of other developers.
The only world where you would actually need to be able to recall an algorithm would be a post-apocalyptic one, where the hard drives of all the computers connected to the internet were fried, and all copies of foundational academic papers and computer science textbooks had been reduced to ashes.
Whiteboard interviewing is a discrete skill, much like being able to remember Pi to a thousand decimal places. And students of programming are spending a disproportionate amount of their time mastering this skill instead of practicing real software development by building projects, maintaining legacy code, or contributing to open source.
This ultimately reduces the quality of entry-level hires, as it becomes difficult for the average interviewer to figure out who’s good at developing software, and who’s merely good at whiteboard interviewing.
It also freezes out many of the people who are under-represented in the software development field. If you’re busy working and raising kids, you want to spend as much of your scarce time as possible learning to code — not performing rote memorization that won’t matter once you start your job."
@theloniusmonk, I would suggest that the google interview is not working for them. They claim they want more women and minorities in technical positions where they are severely underrepresented. They aren’t doing all that well. This is an area they could tweak if they were serious about it. There’s a lot more to it than just the interview of course but it’s a place to start.
I agree that these whiteboard interviews don’t work. Back in the 80’s I did on-campus recruiting for entry level programmers for the company I worked for (Banking industry). I never asked these coding questions (they weren’t as popular back then.) It was easy to tell who knew their stuff and who didn’t.
Interviewing for many jobs have changed from back then. My daughter has had a lot of interviews where they ask situational/problem solving questions. Sometime they are silly and sometimes serious - how would you approach/solve this situation? These were for jobs as analysts (econ/math major).
@Rivet2000 “Just like admission to colleges, companies will miss great candidates, but in general they will land more than they miss.”
They may land more than they miss, but most of those they land will be exactly the same. While that really doesn’t affect a college, in an industry as fast changing as high tech, having a diverse workforce means that you’re more likely to produce the next novel product, instead of repeating the same thing with minor alterations time and again. That is why, despite bombastic claims of “newly designed from ground up” for every new model, there were almost no innovations in the automobile industry for decades. All you got were minor changes and some more bells and whistles were added. The designers were all engineers, and, like CS, these tended to be fairly homogeneous. This is changing somewhat, which is why suddenly there are real electric cars, hybrids, smart cars, and self-driving cars.
That only worked with car companies, because there were three or four companies and they are all competing for the same, fairly conservative, costumer base. In software, the product itself is so much more versatile and the costumer base loves true innovation. So hiring 342 middle class boys who grew up on the same types of households, have the same cultural reference points, same hobbies and interests, and who are good at the same exact things, may be simpler, but it’s not a good recipe for producing any innovative products. Girls and women are half of the costumer base, and expecting that boys who who spend all of their time with other men, and have almost no understanding of women to create products that will appeal to the huge untapped costumer base is ridiculous.
Not hiring women, not hiring minorities, not hiring LGBTQ folk is not only an ethical issue for high tech, it’s an extremely bad business strategy. If nothing else, the fact that so many tech companies are refusing to do better at hiring women, despite knowing that this is hurting their business practice, tells me that there is a wide streak of misogyny in the field.
@3girls3cats I like the article you cite and I have to admit that the whiteboard process maybe frustrating to many. If you look at it from the company point of view, it may make sense. The questions they ask are not really that intensive - it’s material covered in most first year data structures and algorithms classes. However, white boarding may show something a little more. Basically, if your interested in company X, and company X uses white boarding in their process then (wait for it…) - accept that and prep for it-easy. Maybe they’re just looking for kids that will make the effort to prep. If you don’t want to jump through hoops to get to the next stage then simply move on (as the article recommends). You’ll be happier in the long run. Depending on the job, many companies have processes that are IMO even more rigorous. Stories from S indicate that many SV startups give code tests that are notoriously difficult.
SV startups, depending on who’s funding them, also give personality and IQ tests.
“it’s an extremely bad business strategy.”
How is it a bad business strategy, these are the most valuable companies in the world. They make tons of mistakes of course, but if one of these companies discovered a better business strategy around hiring, wouldn’t they implement it? Once one of them discovered that state flagship grads were really good and easy to work with, they all adopted it.
“This ultimately reduces the quality of entry-level hires, as it becomes difficult for the average interviewer to figure out who’s good at developing software, and who’s merely good at whiteboard interviewing.”
I disagree about the quality of of entry-level hires at places like Google et al.
Let’s say for discussion that the interview and hiring process is bad. How would you assess technical capability and software development for a new college grad to get a higher quality workforce? You have to assume that at the interview stage, the GPAs, courses taken, school will largely cancel themselves out, leaving internships and school projects as the differentiator along with the interview. So you still have to figure out who can do the work, and who can fit in.
This is the very best strategy imo. High intelligence should be the initial screen. The next screen - conscientiousness. Get those two right and you are more than 50% of the way towards an effective workforce.
The other trait characteristics like openness, creativity, extraversion, etc. could also be very important depending on the particular culture of the company and the nature of the position being filled. There are tests for each of these.
Whiteboarding can be almost like taking a CS test in college. It tests your understanding of the fundamentals. In the real world, you can Google most of the answers or look them up in a textbook, but have to recall from memory on the test. The same argument against whiteboarding could be used to argue GPA isn’t a good predictor of ability to do modern software development.
@SatchelSF When you’re looking at IQ testing for the people who are churning out code, you very quickly get to overkill. For that you need, as you wrote, people who can work quickly, efficiently, and conscientiously. They don’t need to have SATs of 1600 or degrees from MIT. When you’re looking at the people who are supposed to have the ideas, you want the people who are able to think “why don’t we try something different?”. Standardized testing tests how well you do at providing the solutions that are already known for the problems that already have been considered.
The first graphic web browser wasn’t developed by one of engineers at a huge software company, who had top test scores and a great undergrad GPA in engineering. It was developed by some dude with an undergraduate degree in physics who wanted to figure out the best way to do something he wanted to do.That’s how innovation happens - you have people who want to do something different than what everybody else is doing in the company. If your company is full of people who all do the same thing, all you will get are variations on a theme. They may all be very smart, but they’re all smart in the same way. Moreover, they really all want the same things, all see the same problems, all find similar solutions.
My wife is doing amazing things and garnering a lot of attention, because she had one foot in my world (ecology). So she started thinking about what to do with CS in ecology and conservation. So while most of the people in her field were figuring out how to do a little better at calculating speed, or how to produce slightly better graphics, she was looking at using computation in ways that it had never been used before. She wasn’t faculty at MIT or Stanford, she wasn’t a top researcher at Google or Microsoft. In fact, many of these railed at her work, because it “wasn’t REALLY computer science”. She was somebody who thought differently and saw how to use computers in a different way.
If you want to improve existing products, you look for the people who did best on tests, and are great at learning how to do everything in the manner it is taught, in the fastest and best way possible. If you want to create new and innovative products, you look for people who think differently and look at the world differently. You do not hire another clone. Most tests identify clones of the existing workforce as the best people to hire.
Everyone does not need to garner attention to themselves. Many will be happy working on improving exiting products. Some may find the quest to a 3% improvement on video compression to be a rewarding endeavor. Some want to transition from 4G to 5G. There’s room for all.
CS as a whole is more popular in recent years, for both men and women. The more relevant question for this thread is whether the gender balance has changed over time. As noted earlier in the thread, highly selective colleges including the listed ones tend to have a better male/female gender balance in CS than the overall average. MIT, Caltech, and similar tech schools have a far higher admit rate for women than men, which distorts the numbers; so I’ll look at how Stanford’s CS enrollment and gender balance has changed instead.
Some specific numbers about how enrollment for both genders in Stanford’s majors is at http://stanfordvisualized.darkmirage.com/ and summarized below. Stanford first offered a CS major in 1986, so Stanford trends during the 80s may differ from the national trend, being a small, new major. However, Stanford does show a similar trend to the national one I noted during the 2000s, with a sharp decline in percent female that ends in ~2007. As CS enrollment decreases until reaching the minimum enrollment on ~2007, female CS percentage also decreases until reaching a minimum of 11% in 2007. As CS enrollment has rapidly increased from 2007 to present, % female CS has also had a rapid increase. Berkeley shows a similar trend. This Stanford and Berkeley trend differs from the overall national CS enrollment, which has seen relatively small % female increases since 2007 in spite of a larger total number of CS bachelor’s degrees.
One contributing factor is Stanford and Berkeley actively made changes in an attempt to increase female CS enrollment. Stanford changes include things like having more intro CS levels, and making the first course easier to start out for persons with limited HS CS experience. They also increased the number of TAs. In the fall quarter, there were only ~6 students per section in the lower level intro CS class, which is a smaller section size than nearly any other CS class. Stanford also added a wider variety of CS tracks majors can choose among, including some in fields that are more traditionally female, such as computational biology. They also added a CS + X program, where students can combine CS with a more traditionally female humanities major, integrating the two of them together. Berkeley also targeted women with less HS programming experience, including bringing dozens of female CS students with less experience to campus a week early to get a head start. Some of these changes may have also contributed to the rapid rise in overall CS enrollment among both genders, becoming the most enrolled major in the 50+ years for which Stanford has tracked major stats.
1986-90 – 296 enrolled in CS, average 19% women
1991-95 – 471 enrolled in CS, 15% women (min % female is 13% in 1994 and 1995)
1996-00 – 827 enrolled in CS, 20% women
2001-05 – 943 enrolled in CS, 19% women
2006-10 – 848 enrolled in CS, 14% women (enrollment min is 2006/7, min % female is 11% in 2007)
2011-15 – 2334 enrolled in CS, 25% women
2016-Current – ~3800* enrolled in CS, 34% women
*Extrapolated to 5 year period
I’ve been working in tech for a long time. I’ve interviewed at a variety of companies, including at some SV start-ups. I’ve also known many CS employees who have discussed past interviews. I’ve yet to hear of anyone giving personality and/or IQ tests, although smaller companies often do not have structured interview process. At many small companies, the interviewer may ask the applicant to solve whatever type of problem they happen to think is important for the job, which may be extremely simple or extraordinarily complex. It may be a problem the interviewer happens to working on as part of his job responsibilities, or it may have nothing to do with CS. I interviewed at one small SV start-up in which the interviewer told me about how large a portion of the employees shared the first name of the founder (1 in 4 or so, don’t recall exactly). It was far too high to happen randomly, so apparently they emphasized the name of applicants, when making hiring decisions. Small start-up type companies may favor applicants for a wide variety of unique reasons.
One common theme that does occur in tech interviews is asking the applicant to solve a variety of problems to determine both technical ability and problem solving style. I’ve had interviews where the interviewer barely said high before handing me a paper and asking me to solve problems. CS interview questions often emphasize fundamentals in data structure and algorithms, although there are many exceptions. Interviewers rarely give direct personality tests, but they still care about personality and how you will interact with the team. Many applicants do not receive job offers primarily for personality/fit reasons, rather than how well they answered the technical questions. This does not just mean they are looking for more White programming bros. Things like being able to work in a team and communicate thoughts well can be important and are not always obvious from resumes. When I first started doing interviews while in college, it took me a few to realize that I needed to appear more enthusiastic to get job offers. In retrospect, it would have also been helpful to do more research about the company beforehand.
Technical questions may be answered verbally (first interview round is usually a phone call), on a whiteboard, on a piece of paper, or on an actual computer. Some companies give more complex take home problems, where you code at home prior to the interview and bring in your finished code to the interview. Some applicants bring in a portfolio of past coding they have done and/or have it online for reference. Solving questions on whiteboards are not some bizarre interview abstract that has noting to do with actual CS jobs. Most tech companies I am familiar with have whiteboards all over the place. Most individual office/cubes have whiteboards, and they get a lot of use. It’s quite common for software employees to talk over their plans/ideas with other team members, using whiteboards, including drawing block diagrams and writing code segments.
Some example actual interview questions reported by applicants for a entry software engineer at different companies are below. Applicants without a strong CS background are unlikely to pass this type of interview question, regardless of how high their IQ is. This relates to why many positions state that they require applicants to have a particular major(s) – CS or similar.
Google – When would you want to use a hash table over a binary search tree? What are the big O’s?
Apple – Implement strstr using a linked list of linked lists.
Microsoft – Implement a bounded bifocii data structure using an array and efficient enqueue and dequeue algorithms.
Amazon – What is HashTable? How does it work in the backend perspective? What should you do if collision occurs? What is the time complexity of inserting? Deleting? Searching? What’s the time complexity for the worse case?
I would honestly consider to be an issue of trend rather than sexism. Men seem to be more interested in computer programming than women. There are most definitely women who code, some far better than any man, but I feel it is more the varying degree of interest between the two genders. It is the same development being seen in veterinary medicine. For the last several years, most veterinarians were men, but according to the American Veterinary Medical Association, there is currently 55 women to every 45 men in the veterinary market. An even more intriguing statistic is the 2017 class, which was 80% female. Although my major is Equine Studies, we are also seeing an interesting trend. In a department with around 100 students, there are only about seven men in the program, myself included. Such statistics suggest our department is 93% female however, the departmental faculty is 4:3 with a male majority. The industry itself is still a majority male, but a landscape change could be likely given the current state of the equine departments around the nation showing similar trends. I believe the shift in gender distribution may not be a direct result of gender discrimination, but rather an unforeseen shift in interests among the two genders. With society being as progressive as ever, there is no telling how the cultural landscape will look, even within the next decade.
The reversal in computer science from the mid-20th century to now has little to do with the time commitment or the investment for computer science. Being a software developer isn’t particularly more demanding from a time investment perspective than being a doctor or a lawyer, and those fields have a much better gender balance. It’s not even necessarily more demanding, than being a teacher or a social worker, both of which are definitely more heavily female. In fact, if we want to argue that the gender split is due to women being involved in child-rearing or biological clock considerations, medical school actually hit up MORE against that - since you have do training for much longer. Yet the distribution of medical school graduates is pretty close to a 50/50 split.
The thing is, the article addresses very clearly when and why the reversal in computer science happened:
Social scientists have done research for decades that provides strong supporting evidence for this. As others in this thread have already mentioned, girls and boys perform about equally well on math and science up until late elementary/early middle school. Girls sometimes outperform boys, actually. But right around puberty is when gendered expectations for what are “boys’ subjects” and “girls’ subjects” start to emerge, often unconsciously. Boys are expected to be better in math and science and are rewarded for playing to type; the same happens to girls in humanistic and social sciences fields. There’s research to show, for example, that teachers are more likely to call on boys who raise their hands to answer math and science questions in classes and are more likely to praise and encourage boys in those subjects, even when they underperform. Girls are gently, often unconsciously, guided into “girls’ careers.”
Even the old canard about boys being more interested in “things” while girls are more interested in “people” is simply part of the gendered set of assumptions that people have about careers. It grows from the centuries-old notion that women’s place is at home, taking care of families, while men’s place is out in the world, taking care of the gears that make it turn. When that preference exists, it doesn’t exist in a vacuum; it exists through decades of gendered socialization.
No, it doesn’t. This is another common myth about math and science, but many of the most successful scientists and mathematicians are/were broadly experienced and were interested in lots of different fields. The earliest mathematicians and scientists were often known as Renaissance men, experienced in many fields across the humanities and sciences. And even contemporary technologists often have broad experience and knowledge. Steve Jobs talked openly about how his interests in art and humanities influenced the development of Apple products; Mark Zuckerberg double-majored in psychology at Harvard; Evan Spiegel (the founder of Snap) studied art and design in high school and college; Larry Page studied music seriously in high school and credits his music education for inspiring him with Google (and one of his first product designs was a music synthesizer); Mike Krieger (one of the two founders of Instagram) actually studied symbolic systems at Stanford (an interdisciplinary program similar to cognitive science)…I could continue. (This is also addressed in the article.)
Pretty much all of the science on gender and race, as well as the entire history of gender and race relations up until now, strongly disagrees with this assertion. Sexism and racism won’t go away if we close our eyes and pretend it doesn’t exist; it won’t go away until we actively engage with them as concepts and aggressively push them back at their roots.
Women used to be a small minority in Medical schools. They are now a majority, and have been for decades. They deal with lots of Science and math, and trust me, there are plenty of toxic workplaces in health care. Wonder why CS is so different?
@1NJParent - I don’t know since most inventors did not bother doing IQ tests, and many were born and died before the first test was formulated. A lot of the reasoning then becomes circular - because Einstein did such great stuff, he MUST have had a high IQ. How do we know that IQ measures intelligence? Because Einstein had a high IQ.
Tim Berners-Lee, who created the first graphic browser did great things, but, as far as I know, he never did an IQ test. In general, people who do great things don’t bother with IQ tests, because it’s not important for them. No famous inventor or academic is a member of Mensa. A PhD is a much better indication of whether a person can produce something original than any much-touted test. However, I’m pretty sure that nobody has been able to figure out how to estimate intelligence based on a PhD thesis or on an innovative discovery.