Stats from the experts show that even great engineers fail technical interviews [1]. Anecdotally: I've bombed interviews myself! Sometimes due to anxiety (a real problem for some of us), sometimes because the problem was difficult. So false negatives are very common. Especially if you are interviewing 200 people - you are wasting SO MUCH TIME. I simply can't believe that you're actually getting that 1 in 200 dev. Why?
Because devs also pass interviews they have no business passing. You get a technical question that you already know the answer to and so you get through it easily (this has happened to me as well). I'd consider this a false positive. And today, I suspect this is just as likely as a false negative - we have many books on coding interview prep, probably hundreds of sites with coding problems to practice. Are you getting a good dev or are you getting someone that practiced 20 questions on the same subject last week so they happen to know exactly how to solve your interview problem?
And the worst types of false positives are not ineptitude, in my opinion. I can make a mediocre dev (or a dev who is bad at algorithms) into an excellent dev. But there are a lot of devs who are very smart, but very lazy or toxic. And those people will sometimes impress you the most in interviews... because they have to do it a lot because they don't last.
If anything, if the industry insists on technical coding challenges, we should stop the random bullshit and commit to a standardized exam and license. That's how other professions do it (and it works quite well with rare exceptions), and it would save software engineers a lot of time and headache, and make the hiring process smoother for everyone.
Experience should count the most. How recently have they been coding. Look for signals like: they got promoted. Listen to them talk about things they worked on and find out how much detail they can give on those things.
> Listen to them talk about things they worked on and find out how much detail they can give on those things.
This always stresses me out, not just in interviews (which are actually a bit easier since you try to prepare beforehand there, although I guess the anxiety and stress can cancel this out anyway) but also in everyday chit-chat with other devs. I strongly suspect that I may have ADHD and I'm just unable to randomly talk about things I've been doing even as recently as last week without preparation. I can recall a lot if I actually sit down and calmly trace back some stuff I've been doing years ago, but if I got surprised with a question like "you said in your CV that you used X at job A, what was the thing you liked the most about it?" it could take me the whole interview to just get into the mental state necessary to even recall my experiences with X, let alone put them into proper words. I'll eventually find a perfect answer, but chances are high that it will happen at evening under shower accompanied with a huge regret titled "I should have said that back then" :P
Every single thing on it? I've worked with dozens of technologies across multiple fields in my career and I'm still in my (late) twenties, so there's probably more to come. I don't think I could be well prepared for every single question about my past that could come up during an interview. I once got surprised with a question about "the story that led to" discovering a CVE that's attributed to me, for example. In an asynchronous chat that would be a very fun question to answer, but not in a real-time conversation. Playing a detective trying to reconstruct your own memories clue by clue isn't compatible with keeping the interview going.
Nevertheless, I can be prepared on what I was working on during my time at company X, but a question like "what I liked/disliked the most" could still put me off track if I haven't thought about it before. I just don't know without a longer analysis.
I've actually prepared a document with common interview questions / answers / examples of work to talk about, which I typically review before an interview and keep open in my browser while I'm doing video interviews. I think it helps, you might find it to be a useful exercise.
>Anecdotally: I've bombed interviews myself! Sometimes due to anxiety
I used to get pulled in on anxious candidates to help make a final decision. Interviews are rarely an environment were a person does its best work. Experienced interviewers know this.
Typically, I didn't make them code. Coding and anxiety don't mix well.
I usually approached the interview with roughly the following structure:
- Reduce anxiety
- Work on a problem (figure out how it is to work together)
- Reverse interview
To reduce anxiety, what has worked for me is basically making them feel in control. I first make them talk about themselves (i.e. something they know), and then I try to find a passion, something they feel strongly about, related to a project they did (hobby, uni, whatever).
Have them walk me through it (still something they know and control the narrative on, and also something they care about).
I hear actively and engage with them. I'm non judgmental and positive about whatever they tell me.
At this point most people feel at ease.
Then I present a simple, open ended design problem, no code required, but I may probe for specific knowledge here on DB, distributed systems, concurrency, probability, security, whatever I'm looking for. Usually as a discussion of some sub-topic in the design.
I make it clear at the beginning, that I don't care much about the solution and more about the process. Picture us working together working on this problem, this is what I'm trying to figure out. How is it to work with you, and hopefully you'll figure out how is it to work with me.
Then I typically close with an interview reversal, I give them a chance to ask me anything and I answer truthfully so they can assess if they would really like to work here.
This has worked well for me, but I'm an extrovert, so the putting you at ease part is relatively easy for me (it's emotionally exhausting sometimes).
>specially if you are interviewing 200 people - you are wasting SO MUCH TIME. I simply can't believe that you're actually getting that 1 in 200 dev. Why?
Most of the 200 don't pass basic screening, 30 will do a simple code screening, maybe 10 will get to an actual interview. Then one or two of those will get hired.
>But there are a lot of devs who are very smart, but very lazy or toxic.
We care a lot about this, we've passed on really smart candidates with lots of experience because we thought:
- they wouldn't be happy working with us
- they were hard to work with
BTW we measure post-interview experience, whether you got hired or not, to improve our interview process and assess interviewer performance, we might pull somebody from the interviewer pool if they perform poorly at it.
>a standardized exam and license
I really don't think it will change anything. You have a CS degree, I will still make you code a bit.
In any case, hiring requires many signals. Coding is just one, there are many others at play that can make or break a hire.
Also, the company moment matters: are we risk averse at this point? A startup usually would rather lose a good candidate than hire a bad one, larger companies can take more risk, and give you time to prove yourself.
Because devs also pass interviews they have no business passing. You get a technical question that you already know the answer to and so you get through it easily (this has happened to me as well). I'd consider this a false positive. And today, I suspect this is just as likely as a false negative - we have many books on coding interview prep, probably hundreds of sites with coding problems to practice. Are you getting a good dev or are you getting someone that practiced 20 questions on the same subject last week so they happen to know exactly how to solve your interview problem?
And the worst types of false positives are not ineptitude, in my opinion. I can make a mediocre dev (or a dev who is bad at algorithms) into an excellent dev. But there are a lot of devs who are very smart, but very lazy or toxic. And those people will sometimes impress you the most in interviews... because they have to do it a lot because they don't last.
If anything, if the industry insists on technical coding challenges, we should stop the random bullshit and commit to a standardized exam and license. That's how other professions do it (and it works quite well with rare exceptions), and it would save software engineers a lot of time and headache, and make the hiring process smoother for everyone.
Experience should count the most. How recently have they been coding. Look for signals like: they got promoted. Listen to them talk about things they worked on and find out how much detail they can give on those things.
1. https://medium.com/free-code-camp/you-will-randomly-bomb-tec...