Are you telling candidates in advance what type of code to expect or surprising them?
Do you know what I do when I see a problem for the first time that I don't immediately know the answer to? I Google it. Because A) it's faster B) many minds greater than mine might have already come up with an optimal solution that I'm unlikely to self-produce in 2 hours.
What if a developer hasn't been deep in debugging production code for a while? Those edge cases might not be top of mind.
Maybe they're doing documentation or planning or writing Jira tickets and remembering all the ways to nullcheck a JavaScript variable isn't their current life goal. Instead they might try running the code locally or write some tests for it or research to see what others have done in similar patterns.
Dreaming up magical solutions while under the stressful glare of people judging your life's work may sound reasonable to you because presumably it matches up to some internal signals you value in each other, but every workplace is different -- different codebases, different problems, edge cases prevalent in one may not be prevalent in another.
You're better off doing a traditional Q&A about their experience and a take home mini-project (or asking them about theirs). If they don't want to spend time on that, then and only them give them some timed test you feel is applicable.
None of the code that we use for this is too difficult. To date we've only hired for generalist engineers, and are not using some obtuse algorithm example. It's things you'll need to do in a day-to-day like filtering logging outputs, checking HTTP response codes, throwing errors, etc. We want to see how a candidate preforms on an example like this rather than writing something from scratch.
Of course there's Q&A and their reasoning/explanation is just as valuable as the code they right. This is just the technical portion of the interview. However, I think we've really hit the sweet spot with this approach.
Of the 3 you mention "filtering logging outputs, checking HTTP response codes, throwing errors" only the last one seems universal to me.
Take logging. Super important - I spend a lot of time evangelizing logging to younger devs and how to do it well. But filtering? How? In what way? For what values? I don't have context but the expectation I should be able to filter for specific things you have secret preferences for strikes me as a signal that's not clear or universal.
Checking HTTP codes. In what context? There are plenty of times when it's not helpful -- for example there are many page not founds that DON'T return as 404 (even tho they should). So this a priori belief that all devs should automatically have this built-in preference for checking HTTP codes as some universal signal of quality in programming is, I think, a larger assumption than you might realize.
I don't think there is a perfect way to interview, but I do think if you're going to quiz developers it helps to give them some advance warning of what areas you prefer to focus on so it's not such a random, out-of-left field line of questioning.
You may not consider your questions abnormal -- but that's the problem, neither do the people who ask obscure algorithm questions! It's very hard to validate your assumptions.
Do you know what I do when I see a problem for the first time that I don't immediately know the answer to? I Google it. Because A) it's faster B) many minds greater than mine might have already come up with an optimal solution that I'm unlikely to self-produce in 2 hours.
What if a developer hasn't been deep in debugging production code for a while? Those edge cases might not be top of mind.
Maybe they're doing documentation or planning or writing Jira tickets and remembering all the ways to nullcheck a JavaScript variable isn't their current life goal. Instead they might try running the code locally or write some tests for it or research to see what others have done in similar patterns.
Dreaming up magical solutions while under the stressful glare of people judging your life's work may sound reasonable to you because presumably it matches up to some internal signals you value in each other, but every workplace is different -- different codebases, different problems, edge cases prevalent in one may not be prevalent in another.
You're better off doing a traditional Q&A about their experience and a take home mini-project (or asking them about theirs). If they don't want to spend time on that, then and only them give them some timed test you feel is applicable.