But they will. The interview process is stochastic; exactly the same performance from you will lead to different results on different attempts.
> Getting rejected at Triplebyte was actually a pretty good investment time wise. Guess the whole thing costed me three hours in total and I got quite a list of things to improve and how. It was clear it was tailored towards the interview not just a larger generic mail.
For another perspective, here's the entire feedback I got when they rejected my application:
> This was a tough decision and one that we were on the fence about. We really appreciate you taking the time to work on the take home project. We're aware this requires a substantial time commitment and we are really grateful that you invested the time in completing it. We thought you wrote a great, very full featured regular expression matcher. It was especially impressive how much you dug into the academics behind regular languages.
> However we made the decision because we felt that while going through the project together during the interview, we didn't see the fluency of programming when adding to it that we had hoped for. While we specifically designed the take home project track to help overcome the difficulties of coding under time pressure with someone watching, we do still need to see a certain level of programming during the interview. This didn't seem to be the case here, where making changes to the project seemed to be slower and more difficult than we'd have liked.
While we specifically designed the take home project track to help overcome the difficulties of coding under time pressure with someone watching, we do still need to see a certain level of programming during the interview.
“How to set up a test that works and then completely ruin the results by doing nonsensical things during a dumb ritual that our entire industry seems set on preserving.”
Question for Triplebyte: when’s the last time that a single hour of coding — while being watched — determined whether you’d get to keep your job? Not acquire a job, but keep it.
You need something that confirms the take-home test was really done by the candidate.
I have received take-home tests, passed via recruiters, where the recruiters literally sent me the solutions given by previous successful candidates "for reference".
Although that seems like a valid reason, consider how many justifications have been used throughout history to scare people into submission. “There May be cheaters out there” is of the same form as “there may be <insert statistically unlikely thing> out there, so you’d better <unnecessary overreaction>.”
If there is data to support that most people are cheaters, that’s fine. But at Matasano I believe the statistics were ~30 candidates who were invited for an on-site interview after a take home test, and ~30 happy hires.
The on-site interview was also mostly a formality; the fact that they could do the work was enough to all but guarantee an offer.
And when you reduce it to those terms, it seems ludicrous that the world should be any other way. You can either do the work or you can’t. And if you can, nothing else should matter.
There are other counter arguments: what if someone is a huge introvert and not suited to working in a team environment? Bring on the introvets, I say. You won’t regret it. The most talented coworker I’ve ever seen was also someone who had stammering problems and would barely talk. But he was very nice, and much more skilled than I was at the time.
You don’t know someone until you work with them. And if they’re a bad hire, it’ll work itself out within the first month. It’s far better to deal with a possible cheater than to miss out on a skilled, solid hire. The latter makes or breaks companies; the former are just a temporary thorn.
I went through TripleByte 2 years ago and got much the same feedback. I was left with the same feelings, on top of the feeling of having wasted a bunch of time. TripleByte only makes sense if it can fast track you to enough companies to make the investment worthwhile. And, even then, I think, at the time (maybe still now), they only let you skip technical phone screens. Never mind that their process is more or less equivalent to the standard tech on-site (which, IMO, means it should at least count for something at the client companies.)
The bigger difference that I see is your statement that "Every few months, I check that all our recommended resources are still up-to-date, available, and free."
In some unfruitful further communication with TripleByte, I asked how I was supposed to improve, given that their feedback seemed to be "we have no qualms with your ability, but we don't think you can pass an interview". They recommended that I sign up on interviewing.io. I did that, and a year later, I was still on the wait list. (I eventually got off the waitlist when complaining about this same sequence of events on HN, and an interviewing.io employee saw the complaint. I don't think that approach scales well.)
Also, this looks like a feedback email after a take home project? I guess the rejection emails after the technical interview should be much more detailed.
All the ways I discussed in the article - more nuanced, more thorough, more detailed, more focused on constructive advice you can take to your next interview. Plus, we've just improved our process in general so you aren't tested on skills you don't need. It's easier to give constructive feedback about an interview process you have a lot of confidence in, and much of the work we've put in as a company over the last few years has been designing an interview we think really works.
> It's easier to give constructive feedback about an interview process you have a lot of confidence in, and much of the work we've put in as a company over the last few years has been designing an interview we think really works.
It's hard to take this at face value, as 100% of the TripleByte messaging I've seen since before I received that email focused heavily on how confident they were that their interviews were thorough, high-quality assessments.
I think a lot of the dichotomy here is that nobody really knows what a thorough, high-quality assessment for a software engineer really is. I mean, yes, the standard "work sample über alles" line that comes from research is great, but what really constitutes a valid work sample? How do you set up expectations so the candidate knows where the bar is, much less how to get over it? Things like that.
Edit: You also need to consider that TripleByte's idea of what works is probably different from yours. Their idea is probably more along the lines of "people pass our assessment and get hired." All you really need to do to hit that (not that it's a trivial thing at all), is conduct an assessment that's similar enough to what the hiring companies are doing. And, many of us know that hiring companies frequently aren't very good at interviewing.
We don't have the project track anymore, because most candidates weren't willing to do it and it wasn't successfully predicting hires. (There's actually a blog post about this upcoming). We have generalist, front-end and mobile tracks but none of them involve take-home work anymore.No matter which one you do, the feedback will be a lot more thorough than we were able to be two years ago.
> But they will. The interview process is stochastic; exactly the same performance from you will lead to different results on different attempts.
I agree that's in practice true for many places. But I hope companies have the good sense to be too embarrassed to admit that their inability to be consistent is the reason to reapply.
I think the traditional reason for the "try again in three months" bit is that paper-based processes make it hard to find and re-contact promising candidates who didn't happen to make the grade this time. But if any place really thinks, "We won't hire this developer now but they would be good in the future," I think a much better solution is for the hiring company to make a list of people to re-contact next time they're hiring.
That's quite a bit less than I got. Perhaps you're just better than me? ;) But I didn't know there might be another take home assignment.
> But they will. The interview process is stochastic; exactly the same performance from you will lead to different results on different attempts.
Yeah I thought about that. But "try x months later" really means that you didn't pass some kind of bar. If I would be one of the five people that passed there would be no problem if I applied next week again, right?
> Why? I wouldn't do anything different.
But they will. The interview process is stochastic; exactly the same performance from you will lead to different results on different attempts.
> Getting rejected at Triplebyte was actually a pretty good investment time wise. Guess the whole thing costed me three hours in total and I got quite a list of things to improve and how. It was clear it was tailored towards the interview not just a larger generic mail.
For another perspective, here's the entire feedback I got when they rejected my application:
> This was a tough decision and one that we were on the fence about. We really appreciate you taking the time to work on the take home project. We're aware this requires a substantial time commitment and we are really grateful that you invested the time in completing it. We thought you wrote a great, very full featured regular expression matcher. It was especially impressive how much you dug into the academics behind regular languages.
> However we made the decision because we felt that while going through the project together during the interview, we didn't see the fluency of programming when adding to it that we had hoped for. While we specifically designed the take home project track to help overcome the difficulties of coding under time pressure with someone watching, we do still need to see a certain level of programming during the interview. This didn't seem to be the case here, where making changes to the project seemed to be slower and more difficult than we'd have liked.