We do basically what OP said, but we ask the candidate to spend more or less 4 hours on the exercise, in one sitting or two. We also give them a fairly large project to do and say "We know you can't remotely complete this in 4 hours. Pick a nice subset that you find compelling and that demonstrates your strengths, and code that. Explain your choices." It's worked well for us - it demonstrates not just coding ability but also planning, communication, and time management.
this isn't "burden and cost".... it's providing a large set of requirements and giving clear expectations about what is realistic, and the flexibility of choosing which requirements you satisfy to demonstrate your skills.
The issue is the value the employer derives from this set up vastly outstrips the value the average application receives (which is severely negative for most candidates). The fact is, most applicants for such a job have practically zero chance at getting hired. Just that they don't know this. So they go above and beyond for the sake of making a good impression, and you get to throw them out after 30 seconds of scanning their submission or looking at their resume. You get the candidate to put in hours of work which you will use to cut them with as little effort on your end as possible. It's exploitative.
To be clear, this is just a general critique of up front multi-hour tests. There are ways to make these things not so lopsided. For example, make clear in the job announcement the minimum requirements so that people will filter themselves prior to putting in the effort to create some impressive application. Ideally you would have such a test later in the process so there is some investment on both sides. First pass filtering can be done with fizzbuzz-esque 5 minute exercises.
We're up-front about this test being part of our hiring process. The test is also not the first step, but follows an hour-long interview. We don't want to waste anyone's time either, so we only give it to people who really think have a good shot at passing it.
> Pick a nice subset that you find compelling and that demonstrates your strengths, and code that.
That would be fairly reasonable. The tests I got, had a heavy project offloaded upfront as effectively part 1 of the interview, with no such communicated expectation.
Do you explain why, to those who don’t make it to the next round? I did one of these, and the response I got was essentially “You didn’t make it”, i.e I had no idea of what was wrong with my code or why I didn’t make it through.
Yeah we do, but it's usually just a sentence or two, something like "we wanted to see a deeper use of language features" or "we're looking for a deeper understanding of framework x". We used to write more, but too many people would try to argue.
