No-Hiring somebody after they get one wrong answer is ridiculous. A wrong answer during an interview and how it's followed up on is one of the most telling parts of the whole process.
If you want to hire people that are good at taking tests then by all means require every answer to be correct. If you want people that understand themselves enough to recover and think critically about their own mistakes then treat a wrong answer as what it is: being a human being in a stressful situation.
edit: also any company that still asks "brain teaser" questions in 2011 should not be one that you consider working for.
A reversion on Wikipedia doesn't necessarily indicate that the reverted edit was wrong.
Wikipedia is a highly politicized and point-of-view pushing place, so it's quite common for edits get reverted by others who simply have a different opinion or a contrary agenda.
Some of this list is pretty much expected (McKinsey, Bain, Jane St, Palantir, Bridgewater), but some of it seems pretty ridiculous. BP? Really? My sister interned there (and was in charge of on-campus recruiting for oil companies at her grad school), and thought they were a bunch of idiots, something apparently confirmed by Deepwater Horizon. Amazon and eBay ahead of Google and Apple? I think anyone in Silicon Valley would think that a bit off. And the only hedge funds were Jane St and Bridgewater - where's D.E. Shaw, or Renntech, or Citadel?
For what it's worth, my Amazon interview was probably the hardest one I ever did -- much harder than my Google interview. At Amazon, every single one of the 7 or 8 people I interviewed with asked me a coding question, including the guy that had done the phone screen, and he'd asked me a coding question in the phone screen too. I also got a coding "homework" problem in the phone screen. I don't have any problem with coding questions in interviews, but it seemed like total overkill.
I had a phone screen with Amazon where the interviewer asked me a coding problem (don't remember the problem). He had a text editor open and wanted me to tell him exactly what to type, not pseudocode, the actual characters. If the program didn't compile and run, my answer was incorrect. I actually got it right, but it (along with the rest of the interview) was such an abnormal experience that it really turned my excitement level for Amazon off.
I was called back to come in for face to face interviews, so I apparently passed the phone screen. I declined to continue pursuing Amazon after that experience and took a job with a different startup I was talking to.
I'm also OK with coding questions, including on a whiteboard, but caring about exact, compilable syntax of code entered in a way you'd never do is worthless.
> For what it's worth, my Amazon interview was probably the hardest one I ever did -- much harder than my Google interview.
Same here.
One of the interviewers asked me to name a software I used and liked. I said emacs. He then asked to draft a design of emacs on the whiteboard. First, a high-level design, then refine one of the modules, etc.
In my experience, the only other interview that was about as hard as Amazon's was at Jane Street Capital, which is also on the list. However, the Jane Street interview was a much more interesting experience -- after the interviews I felt I'd really like to work there.
One of the interviewers asked me to name a software I used and liked. I said emacs. He then asked to draft a design of emacs on the whiteboard. First, a high-level design, then refine one of the modules, etc.
That's a fantastic interview question. Much better than the usual "solve this tiny problem in 20 lines of code on a whiteboard". After you've reversed the words in a string more than twice it gets a bit boring.
Were the coding questions themselves extraordinarily difficult? It just doesn't seem unusual to be consistently asked coding questions during interviews for a software engineering position (assuming that's what you were going for).
No, it's not unusual for all your interviewers to ask you coding questions. It is, however, a lot more work to have 7 or 8 people do that rather than 4 or 5, which is the point I was getting at. That was just a really long and tiring interview.
The hardest one I ever had was to make word ladder perform in linear time. The only time I saw anything similar was in an old ACM-ICPC competition problem set. It's not just that they ask coding questions, it's that they demand you wring ever optimization possible out of them even when you might not see any other way to do it.
My interviewing experience was much different. I found my Amazon interviews to be much easier than Google, Facebook, Microsoft, etc. Maybe it changes with different teams?
Having interviewed at Apple, Google and Amazon (and working at one of them now), I assure you the Amazon interview was hardest, by a bit. Google was actually last. I think it is that Apple and Amazon are going more for culture fit, which can be harder questions to answer for a tech position, if only because you are not as prepared for them.
I don't see how you can compare interviews at companies that do different things against each other in terms of difficulty. In fact I don't even know what this means for interviews within the company. For instance, Amazon: is it a difficult interview for a programmer, any technical person, or even for a marketer? If it's for a software engineer, is it as difficult for new grad entry positions as it is for higher positions? Which is being evaluated?
This comes to mind mostly because I'm surprised at who ended up on the list in terms of tech. Palantir, for instance, is indeed one of the hardest interviews out there, if only because the variety of programming questions they ask you is so large.
Salesforce on the other hand, while not a cakewalk, is definitely not on the level of difficulty of a Google or Facebook technical interview, at least for their university hires.
Lastly, I'm glad to see Teach for America up there. It's become a pretty competitive program to get into and this just further proves it, and I hope it gets expanded so more people can be out there doing good.
Coding questions are often not the toughest interview questions.
Tough questions could be some "soft" non-coding questions like: design questions (how to design a class hierarchy for a blog), behaviour questions (if your boss is wrong, what to do?), experience questions (why did you use tool X in your previous project Y? I think tool Z is better.)
Those questions are tough because your performance on those questions are very subjective. It depends highly on if the interviewer likes you or not.
Coding questions can be tough, but they are much more objective. Google's engineering interview mostly asks coding questions. Questions can be difficult, but at least, if you write great code, you will pass the interview.
In this sense, Google is not a very tough company for job interview.
In general, the degree of interview toughness depends on the job supply/demand ratio. When many people apply for positions at company X, the company has to apply a high-rejection rate, based on whatever (sometime very random) criteria.
I'm surprised not to see D. E. Shaw here. Unlike, say, Bain, which actually needs to hire people, D. E. Shaw seems to hire about three people a year but actively collect thousands of CVs just so they can brag about how many people they reject.
Having interviewed with Amazon before, they ask lots of optimization type questions and if you can make it faster. For instance, take some algorithm and can you make it linear? They do the same thing with their coding questions, they'll have you solve a problem and then ask you to make it faster. I wouldn't say it was the hardest interview I've seen, but it was definitely more challenging than most others I've done.
My favorite interview questions are the ones where they have you figure out an algorithm for something and then say, "Ok, but you can do it better (faster or with less memory)."
It's slipped into how I do my work. I write something that I want to be optimized and say "Ok, but I can do it better", regardless of whether I know it can actually be done better or not. Very fun.
As someone who has recently interviewed at most of those software related companies and got offers from most of them, I can confirm that some of these ranking seem really arbitrary.
Amazon was definitely one of the easier interviews I had. Microsoft had one of the harder ones (but this might have just been a case by case thing).
Jane Street and Palantir, however, I can say for sure had two of hardest and most intense interviews I've ever done.
I've now learned, after being trained to conduct interviews, that these interviews aren't meant to be fair. Companies are afraid of getting any single person that isn't competent because people tend to hire others like them. A single bad person is a cancer. Thus, the hardest interviews are meant to reject 99% (or even more) of the qualified candidates so their false positive rates are ridiculously low. This is how scared these companies are of false positives.
I did 2 rounds of interviews, 9 interviews in total if I recall before starting at McKinsey. Though I have to say, the whole case interview thing is a hack that can be taught to anyone. I knew a lot of not very brilliant people who got in because an alumni pretty much showed them exactly how to do it. Like any test, you can study the fuck out of it and get in.
I interviewed at Amazon once. They gave me three phone screens but I didn't get invited for the 1:1. I greatly respected the interview and it was my first wake up call that I had lost some of my technical edge as a manager.
At Google, at the same time, I almost got a job offer. I got all the way through the interview (phone-screen, all-day in-person interview) and then they said they didn't know what to do with me could I come back in 6 month and make a presentation on a business idea. I took them up on it 6 months later but they passed on my business idea (2004, a rich text wiki).
What I don't understand is that this report seems to contradict other reports of software engineers being in high-demand, which presumably should lead to lower interview barriers?!
It's a market for lemons. When devs are in demand, that brings in applicants who are only in the industry because the wages went up, while the good ones are more likely to be taken. These each lead to a greater proportion of incompetents to filter out, and the cost and risk of accidentally hiring one have not improved.
What risk though? Instead of these days-long interviews with multiple people (and the inevitable meetings after to discuss their results) why not just hire people provisionally and see how they do? Especially if you pair-program, code review, or anything that will expose them to evaluation.
I'd advertise with something like "Job coding X at a company that does Y. Finish the Ruby Metakoans and be prepared to discuss your answer. You will be asked to code in Ruby and C during the interview."
These huge processes just seem like HR trying to justify their existence.
After a phone screen, we spend maybe five hours of developers' time on each candidate and reject 80–90% of them. If we provisionally hired all of them expecting useful work, we would invest far more time ramping them up on our system, risk outages from their early mistakes, disclose trade secrets to a bunch of devs who end up unhappy with us, and lose out on all the employed candidates up front (who aren't going to quit just for an extended interview).
Then there's the moral hazard of paying anyone who can get into our filter, rather than just the ones who can get all the way through it. It's the same reason we can't expect candidates to compensate us for the cost of interviewing them—it's better if a bad match is lose/lose than if anyone could reliably expect to be rewarded for wasting everyone else's time.
I wasn't suggesting you hire everyone who walks through the door, but I bet my hour-long test is within 80% as accurate as those crazy all-day stress tests, and much cheaper. Asking for code (even a few lines) in the application itself would further reduce the number of applicants.
Your system/circumstances seem like they'd make trusting and benefiting from new hires quite a hassle. Ideally you'd have some public and open-source app to have new hires hack at, but there should be tests to be written, etc, which shouldn't leak trade secrets. (If you don't have docs for which files/components implement your trade secrets, and therefore which ones do not, I don't see how you have any hope of keeping anything straight.) If their early mistakes cause an outage I think it says more about your build processes than them.
As for people not quitting for what's in essence an extended interview, that's what they are doing for you and others already. You fully intend to dump them if they don't prove to be good, but you don't have a quick process to determine this so you aren't as up-front about it.
But yes, not every place has it as easy as Github, for instance. They have a ton of publicly available code an applicant could work on, and in doing so they'd show they could use the company's products and tools. I don't know, but would bet they have a comparatively easy interview process.
The interesting thing is that almost all of those companies are companies where I wouldn't want to work - with the possible exception of Jane Street (though that depends on whether or not that company pays is programmers 10% of what it pays its traders) and Amazon (assuming that the stories Steve Yeggy tells are exaggerated and they have brought actual desks for their programmers, instead of the doors they used to have).
I interviewed at Salesforce.com in mid January 2011. It was at 300 Market Street San Francisco, bright and early 8:30AM. Linda Ellis, Raja Bhamidapati, Allison Nishioka, Kei Tang, Ranjani Salur, Rhoita Misra, Sagar Wanaselja, Sam shoute. I didn't get the job. I was put through the ringer. I completely agree with this (difficult interviewers) report. Here are some notes:
I met with 10 people during the interview. A complete battery of psychological analysis. In that group was every kind of co worker I've ever dealt with. I could tell some were acting the part to try to throw me off base and test me for any sociopathic or personality problems. Some were insulting me in my face, some were super charismatic, some people with impossibly hard questions. Some questions were strangely easy. The whole gambit, it gave my personality side a workout (Programmers usually don't work hard on dealing with the 50 personality types).
They planned the sit down coding test an hour after lunch, to test if you know how to deal with the 'afternoon dip' in energy. The coding test I did well on, it was to create functionality to replace the "String" class with char * in java. And more importantly is to unit test. Do this in one hour.
On the plus side, they paid my entire way. and got me into the "Omni hotel" one of the best hotels I've ever stayed in. Paid for the Car. It was by far one of the most exciting experiences of my life.
They analyzed my personal life as well and ask questions about what I'm doing when I'm here, what I like to do, what kind of person I am.
The majority of the questions they asked me were a subset of the questions in the book "Cracking the coding interview" by Gaylle Laakman. They were tough questions, but if you are prepared you can give them canned responses.
After that Interview, I felt like I've never been so probed in my life. From stem to stern. Every part of me, everything that makes me a human was analyzed. I left feeling like I've just been scanned by the Borg. I didn't get the job. but I made it to the 'short list' of 13 candidates up for consideration. I left my heart in San Fran California.
If you want to hire people that are good at taking tests then by all means require every answer to be correct. If you want people that understand themselves enough to recover and think critically about their own mistakes then treat a wrong answer as what it is: being a human being in a stressful situation.
edit: also any company that still asks "brain teaser" questions in 2011 should not be one that you consider working for.