This is a really excellent post. It talks about a bunch of things that are hobby-horses of mine (I help run recruiting for a large software security firm), and I find myself agreeing with more of it than I disagree with.
I would go a little further than Laurie does. I think several of the goals he sets up for his process are not in reality achievable in an interview process.
Starting axiom: job interviews are among the most hostile experiences professionals endure in our industry. I think back to my own interviews, and compare them to public speaking, professional disasters, death march projects with insane deadlines, intractable politics, and it's pretty much the interviews alone that increase my heart rate. For the past two years, I've tried to make an effort to peek in on the interviews we do here at Matasano, and what I've seen corroborates the belief. Several of our best hires were physically shaking during their interviews. And we try to be nice! In no other common situation does a tech worker find themselves interrogated by a panel of strangers whose implicit goal is to knock them out of contention.
Given that interviews are a psychologically challenging experience, and thinking about things like the concept of "depletion" of ego or willpower, it's straightforward to see some severe limitations to what can be accomplished in an interview setting. If you're spending lots of energy trying to keep from jumping out of your skin in an unpleasant situation, it's much harder to solve problems that themselves require lots of energy.
Past that, a hypothesis, which is unpleasant for some tech workers to hear (cold comfort: I'm 100% certain it applies to me as well). Software developers are not, as a cohort, particularly effective intuitive psychologists. Virtually none of us have any training in the subject. We tend sharply towards introversion. We train our brains day-in and day-out by repeating tasks that do nothing to developer our ability to read in-person social cues. For that matter, we tend as a group to eschew forms of communication in which tone of voice, body language, and emotional cues are even transmitted!
But several of the objectives Laurie sets out demand exactly that kind of analysis. "Can the candidate intelligently discuss technology?" Well, that's subjective, and worse, vague and abstract. Laurie tries to nail "intelligently" down, but I think we can all see that there are other ways in which someone can be "intelligent" about technology that evade those criteria. Since we all intuitively know that, we substitute our own cognitive biases for "intelligently". All of the sudden, we're gauging "confidence" and "comfort level"... we've decided to be psychologists instead of engineers.
So, two changes I would urgently suggest Laurie consider for his process:
* Audit the whole process for tasks that could generate false positives from a nervous candidate. You aren't interviewing people to determine how good they are at interviewing, because interviewing doesn't generate money for your company. Try to build a process that is immune to discomfort and lack of confidence. It can be done! Another thing that we've found very effective: "prime" candidates early and repeatedly with non-adversarial conversations that aim to disarm them. We start our whole (multi-week) process with an hour-long version of this. We also try to innoculate our process by communicating in as much excruciating detail as we can what it will entail.
* Eliminate all subjective questions and standardize what you're left with. Engineers, in my experience, fucking hate this. But it's the right thing to do: ask every candidate, as much as possible, the same set of questions. We have a question structure that minimizes open-ended questions but has some structured "exercise" questions that give the candidate some freedom to move around --- the results of those questions can be put on a scoresheet and, to some extent, compared apples-apples to other candidates.
1. I tried to emphasize how important it is to keep the candidate relaxed and comfortable. Your suggestions are good, though we are too small and resource constrained to spend several weeks on interviews; each candidate gets about 6 hours in total, at least an hour of which is with me.
2. I had this process at my last job -- I asked everybody the same opened-ended architectural design question, and found it very useful for hiring back-end engineers, but at npm it's obviously inappropriate for our front-end and CLI engineers, so I've yet to come up with a "standard" question. It's a great suggestion.
As an aside, I am pleasantly surprised to see how many constructive comments there were on this article. Sort of feels like HN from a couple of years ago.
> In no other common situation does a tech worker find themselves interrogated by a panel of strangers whose implicit goal is to knock them out of contention.
I take the opposite approach. I want you to succeed. I want you to be the best candidate I've seen so far. I want you to be awesome. In fact, I want to hire you right now and never have to do another interview again. I'm in your corner, and I'm rooting for you every step of the way. There's no need to be nervous -- just be yourself.
My explicit goal is to hire someone -- not reject a candidate. I have a high bar, yes, but I start off with the assumption that you can pass it, or I wouldn't have brought you into my office to begin with.
It may be semantics, though. Your implicit understanding is that the glass is half empty, and mine is that the glass is half-full. We're both measuring the glass to see how much is actually there, and if it's not half-full, then that's not good enough. But at the end of the day, I'm on your side.
Having said that, I come from an acting background. I understand how casting works. I'm a tall, dark, lean male that can appear anywhere between 25 and 45 years of age. When I go audition for a role, I'm competing against other tall, dark, lean males that can appear anywhere between 25 and 45 years of age. Depending on the number of other actors auditioning, it may simply be a crap shoot. It may simply be that we were all good in our own way, and they just went with THAT guy because he was the last one in the room, or because he wore that one funny shirt that made him stand out just a bit more.
When you come in for an interview, you're one of dozens of people we've phone screened, and you're one of a half-dozen people that we bring into the office. You've already made it past the first round. Now you're competing with the best of the best that we've talked to. If you don't get the job, it doesn't mean you suck, it simply means that we went with someone else. And if you do get the job, it may not necessarily mean that you're the best -- just that you're one of the best.
Kind of something to keep in mind here.
But again, at the end of the day, I'm rooting for you. I want you on my team, or I wouldn't have brought you into my office. Try to relax a bit, be yourself, and show me that I'm right. After all, I love being right. :)
If rejection wasn't such a powerfully negative psychological force, if loss aversion wasn't such a fundamental part of our psyche, we'd all be successful startup CEOs, because sales would be no problem. But in reality, very few of us have the stomach for sales.
I'm not sure why it makes you sad. On a germane-to-the-parent-comment sense, that's pretty much exactly what tptacek said. But on a wider scale, if you are trying to help someone, wouldn't you want to give them advice that truly helps them in a big and cross-task way? Much better than "get good at Rails", which is useful only for a subset of engineering roles (albeit a subset I may enjoy). In terms of time investment... you can get pretty okay at sales in the time it takes you to be as expert as is potentially relevant in Rails. So why not choose sales?
Yes, for the current state of the industry it is good advice. That is the part that saddens me, that "get good at sales" in order to get a job that has nothing to do with sales is sound, even the best, advice.
Software development is still sales. To successfully roll out a new internal product you need to sell your colleagues on it. To overhaul an existing architecture you need to sell your manager on its necessity. To prevent your coworker from making what you feel to be a poor choice, you need to sell her on your point of view.
Agreed. There is a sense in which almost all meaningful and social tasks are "sales". Scare quotes are used there, not because the same skill sets don't apply, but because you typically aren't actually transacting over the results of the conversation.
> job interviews are among the most hostile experiences professionals endure in our industry
I don't find being questioned during an interview hostile, although I do feel I need to be prepared to be tested.
The most bothersome part for me is companies asking for me to list several references, often specifically asking for previous managers. I am usually given a standard application form to write their names, phone numbers, e-mails etc. before I've even talked to anyone about the position.
An interview is just something I'm involved in. Since I have to hand over contact details of several former managers/coworkers before I even talk to anyone, I have to call up and ask a number of people for the favor of giving me a reference before applying for a job.
Due to this, I almost never apply for one job in a one-off sense. I usually work a few years, then decide I definitely am going to get a new job. I line up my references then apply to jobs until I get one. Then I'm completely off the market for another few years.
I could just not hand over the references until later in the interview process, even though they're requesting them from me before I meet anyone, but that starts things off on the wrong foot. As if I had something to hide.
Of course, there may be legal reasons for companies to ask for everything up front. Asking for references too early is the biggest thing that puts me off from looking around. I'm not going to ask several former managers/co-workers for a favor unless I am seriously looking.
I just don't fill in questions on the application form I don't want to answer. Particularly references. They are unavailable -- since I can only impose upon them so often -- until after all interviews and a salary discussion and a serious decision that I want the job, etc.
There are hostile interviews, but not all interviews are. I'll borrow from Michael Church [1] in describing what I consider a hostile interview. A hostile interview is any interview in which the interviewers are actively trying to remove the candidate from the pool. This can be a very subtle thing; as an interviewer, you have to ask yourself: Am I asking questions in order to attempt to weed out this candidate? Does this question have a right and wrong answer? And if so, what does it mean if they get it wrong? Did I simply discover a limit of their knowledge, or is it more ominous?
An important point is that you can be quite friendly and amenable, while still asking hostile questions intended to directly weed out candidates, as opposed to plumbing the depth and breadth of their knowledge and experiences.
I've had former manager (and friend) call me and ask: "Are you looking for work?" When I told him "no", he told me that a recruiter had called him to "check my references" as a pretense so they could ask him if he was looking to hire anyone.
I would be very careful about giving out references.
> You aren't interviewing people to determine how good they are at interviewing, because interviewing doesn't generate money for your company.
Depends. At most companies, how you handle a simple interview is still relevant if it's for a revenue generating or client facing role (e.g., sales, marketing, and project management).
Granted, most technical roles aren't generating revenue or in contact with clients, so this isn't nearly as important.
Your employers -- or, usually, your employer's bean counters -- may be like many humans in that it's easy to see cash flows and harder to see where value is produced.
This is one of the reasons sales and marketing often have a lot of power in modern business organizations (at least, as long as sales are good).
I've often wondered whether for technical roles involving little or no client interaction[1], negatively-weighting skill at answering very pat interview questions (like where they want to be in five years, what they liked most about their degree course or why they're passionate about writing server side code for an ad platform) relative to their overall performance might actually be a useful, if very counter-intuitive heuristic?
Someone who gives halting, but reasonably acceptable answers throughout an entire interview might be better at understanding technical problems than giving verbal answers in a pressured situation, and find the latter aspect far more difficult than the actual problem-set covered in the technical questions.
On the other hand, someone who confidently, thoroughly and unhesitatingly reels off answers to all manner of non-technical questions should, at least theoretically, be similarly comfortable talking about technical problems they don't find especially hard...
And if they're excelling in the pat interview questions because they've done far more interviews than average than may also be a negative mark
[1]many of which, as others have pointed out, bring in a lot of revenue
I think that falling into the trap of treating social skills and technical ability as a trade-off is a mistake. It's a hackneyed stereotype that I don't think has ever been true--but I'd imagine it has sometimes led to people expressly trying to seem socially abnormal in order to fit the idea of someone technically adept. I recall an anecdote about how a younger mathematician at a top research institution took up running his hand along the wall as he went places in order to seem odder, since it seemed like the best senior mathematicians there all had odd quirks. I'd imagine that the idea that social skills and technical ability are inversely correlated has led to similar things.
But also, my experience in TAing the weed out course for the CS major (and from seeing my fellow TAs teach and work, who were more or less the best in class at my University--so much so that I was flown out by a company simply because of my involvement in that class) was that social skills had absolutely no correlation to technical ability: your ability to understand how pointers or complex data structures worked had absolutely no relation to how socially fluent you were. In fact, it often went the opposite way, up to a point: there was some base line of social fluency that everyone I worked with and the best people I taught seemed to meet. The single best coder I have /ever/ seen work (he alone makes me absolutely certain that the best people in our industry are 10x/100x more productive than the average) was also one of the most comfortable with people and had a very clear idea of his career goals. I'd expect him to give confident, unhesitating responses to "Where do you see yourself in 5 years?" and "What's your greatest strength/weakness?" Negatively weighting that would mean you'd miss the hands down most productive person I've ever seen work: and you'd probably miss a decent number of the other people I taught with. (Who, as I said, were basically the cream of the crop from the program I was in. Not all of the best people were there, but all of the people there were among the best.)
Now, I don't think they're particularly good interview questions, even though I understand why they're asked. I think that asking questions about actual workplace soft skills (conflict resolution, for instance) would be a much better use of everyone's time, since in the end it's those things that actually hold a team together.
It's less to do with social skills (agreed, there's no trade-off) and more to do with the probability that people scoring 6 out of 10 for their ability to discuss technical problems whilst getting 10 out of 10 for their ability to answer general interview questions are good interviewers with a weakness for understanding technical problems, whereas those scoring 6 out 10 for technical problems questions but 5 out of 10 for generic interview questions are likely underrepresenting their technical understanding due to the awkwardness of the interview situation. I'm also assuming that some common non-technical questions (like biggest weaknesses, and what they actually did in their last job) usually elicit useful information as opposed to answers synthesised from the job spec.
Since I've introduced arbitrary numerical scoring, I guess by "negatively weighting" I actually mean doing something equivalent to mutiplying their overall performance by 1 + scalefactor*(technical answers score - generic interview answers score) , but I'm assuming in practice most interviewers use their intuition rather than subjectively scoring every answer.
I'm assuming your amazingly productive friend aces the technical questions at least as impressively as the "convince me you'll have grown with the company after 5 years, even if you actually only see this as the bottom rung of the ladder" ones.
I think you miss the point. It's not that people with social skills are likely to be worse technically, it's that social skills help you even on the technical questions in a setting where you are basically trying to sell yourself and being judged socially. Weighting the purely-social questions negatively might serve as a control.
A reasonable theory; I'm skeptical as to whether it would turn into better performance in hired candidates.
I think that you're right about this. People with poor social skills are going to be worse at negotiating pay and in much lower demand, overall. On the other hand, I'd assert that they're going to be harder to manage and lead to more interpersonal issues (almost by definition).
I would go a little further than Laurie does. I think several of the goals he sets up for his process are not in reality achievable in an interview process.
Starting axiom: job interviews are among the most hostile experiences professionals endure in our industry. I think back to my own interviews, and compare them to public speaking, professional disasters, death march projects with insane deadlines, intractable politics, and it's pretty much the interviews alone that increase my heart rate. For the past two years, I've tried to make an effort to peek in on the interviews we do here at Matasano, and what I've seen corroborates the belief. Several of our best hires were physically shaking during their interviews. And we try to be nice! In no other common situation does a tech worker find themselves interrogated by a panel of strangers whose implicit goal is to knock them out of contention.
Given that interviews are a psychologically challenging experience, and thinking about things like the concept of "depletion" of ego or willpower, it's straightforward to see some severe limitations to what can be accomplished in an interview setting. If you're spending lots of energy trying to keep from jumping out of your skin in an unpleasant situation, it's much harder to solve problems that themselves require lots of energy.
Past that, a hypothesis, which is unpleasant for some tech workers to hear (cold comfort: I'm 100% certain it applies to me as well). Software developers are not, as a cohort, particularly effective intuitive psychologists. Virtually none of us have any training in the subject. We tend sharply towards introversion. We train our brains day-in and day-out by repeating tasks that do nothing to developer our ability to read in-person social cues. For that matter, we tend as a group to eschew forms of communication in which tone of voice, body language, and emotional cues are even transmitted!
But several of the objectives Laurie sets out demand exactly that kind of analysis. "Can the candidate intelligently discuss technology?" Well, that's subjective, and worse, vague and abstract. Laurie tries to nail "intelligently" down, but I think we can all see that there are other ways in which someone can be "intelligent" about technology that evade those criteria. Since we all intuitively know that, we substitute our own cognitive biases for "intelligently". All of the sudden, we're gauging "confidence" and "comfort level"... we've decided to be psychologists instead of engineers.
So, two changes I would urgently suggest Laurie consider for his process:
* Audit the whole process for tasks that could generate false positives from a nervous candidate. You aren't interviewing people to determine how good they are at interviewing, because interviewing doesn't generate money for your company. Try to build a process that is immune to discomfort and lack of confidence. It can be done! Another thing that we've found very effective: "prime" candidates early and repeatedly with non-adversarial conversations that aim to disarm them. We start our whole (multi-week) process with an hour-long version of this. We also try to innoculate our process by communicating in as much excruciating detail as we can what it will entail.
* Eliminate all subjective questions and standardize what you're left with. Engineers, in my experience, fucking hate this. But it's the right thing to do: ask every candidate, as much as possible, the same set of questions. We have a question structure that minimizes open-ended questions but has some structured "exercise" questions that give the candidate some freedom to move around --- the results of those questions can be put on a scoresheet and, to some extent, compared apples-apples to other candidates.