I have interviewed senior candidates (more than two, fewer than five) who couldn’t write code to sum an array of ints (ignoring overflow). *

Not in the sense of missed a bit of punctuation, but rather in the sense of “couldn’t get started with anything that vaguely smelled like an answer, didn’t understand the type system of the language they claim 5 years of experience with, etc”.

I’ve probably interviewed fewer than 400 candidates in-person and maybe 200 of those would be senior ICs. To have more than a percent of them be utterly incapable here is a signal that there are candidates out there taking their best half-court buzzer-beater shot to try to land the prize. If you’ve done 100 first round in-person interviews and never run into someone that causes you to check up on how the phone screen happened, your screening processes work pretty well.

Is it possible that these candidates are capable programmers in daily work? Sure, it’s possible, but I’d bet against it.

* We used to ask a candidate to average an array of ints. It’s a question that was designed to tease out whether the candidate would discuss/inquire about overflow, what the return type should be, etc. After a couple years, we realized that candidates would stumble on the loop and sum more than anywhere else and we got more signal/minute from asking the vastly simpler summation question. Candidates who could fluently write the loop usually knew about overflow and the fact that the average of ints isn’t mathematically an int but that many times an int is what you want and can discuss the reasons.

I love this because I agree 100%. I accepted the idea that they were "out there" but only really internalized it after I interviewed someone with a Masters in physics who couldn't make any progress on my simple problem. (The problem does assume you can understand the concept of a geometric line, but I give the two line equations most people first run into in junior high, among other things, I'm not testing recall.) Especially on the part about making a question even simpler in the name of getting what you actually care about faster. I've also found it useful to combine things from separate questions into one question that can answer both in less time.

I still wonder if we couldn't remove most of these games though if we improved our hiring, onboarding, and firing processes in other ways than just trying to make a better hiring filter. Accidentally letting in someone wholly unsuitable for the job shouldn't be a big deal if 1) you can detect that like in the first day, or at least first week 2) the amount of damage they can do before detected is small 3) they actually start very soon after the decision to hire is made 4) you can fire them immediately after deciding a mistake was made and they don't work another day.

Startups can handle this better already and I think is reflected in a lot of them still doing more casual "single lunch conversation" style interviews before a "welcome aboard, see you on Monday" decision. For big cos, work will have to be done on getting away from the mentality of "it's ok if you're not contributing much of anything for the first x days because onboarding" where x is high (I've seen over 30), and firming up access controls (no facebook style all-access of new hires (or anyone if possible) being able to just query the production db and access the private messages of exes or whoever), and removing the "ok you're hired, see you in x weeks" policies where new hires can only start after a significant delay or all on a certain date for whatever reasons, and only getting rid of someone through a long and drawn out PIP process... Maybe it can be done some day.

On the “we can take a chance here” front, I’ve sometimes struggled with hiring someone out of a stable situation and putting them into a role where they might earn a termination. If someone’s already unemployed and that happens, they’re no worse off (and likely better off). If it happens that they resigned a stable job and that happens, they’re made worse off.

As a hiring manager or interviewer, I’m not their fiduciary, but when facing a risky hire, I wish there was a way to let the candidate make a more fully-informed choice without the knowledge that we think it’s a high-risk offer influencing them negatively in any way.

Right now, they interview and get an offer or don’t. If they get an offer that we think is high-risk, they have no reasonable way to know that.

Could you indicate it through your choice of interview questions?

Things like, "How do you feel about stretching your skills in the work force?" "What do you do when you make a mistake at work?" "How do you handle a situation where you feel out of your depth?" Etc.?

I know I would take that as a signal not to go for it if I wanted to retain stability in my life.

That's a great thought; however, we most often discover the conflict during the hiring debrief when we have mixed feedback on a candidate leaving the candidate "on the bubble".

It does give me some ideas to consider when we extend the offer (as @Jach's sibling comment also suggests).

Yeah, it's trickier when they're coming in with an existing job, and the current professional quitting culture dictates that you couldn't start any time sooner than in two weeks at least. Add that to the list of difficulties involved in lower time-to-start dates.

I've been fortunate (or not) that the hiring decisions I've played a part in have all been non-urgent with no struggle to find more candidates, so in the face of risk the org could afford to wait. The moral hazard you describe is interesting though. I don't think I would worry about it, as you say the hiring side isn't their fiduciary, and you don't necessarily know that their current situation is all that stable. I might try and assure myself that even if things don't work out, they're unlikely to be put out on the streets, since if they got past us (even with our "risky" assessment) they can probably get past someone else quickly enough, and possibly even go back to where they left depending on how stable it really was.

I'd consider being honest with them about the risk if an offer is extended. If I were on the receiving end, I'd prefer to be told before I accept something along the lines of "Honestly we're a bit unsure how well you'll adapt to our work [because of risk factors x], but we see your potential and think you'll rise to the occasion. We hope you'll come join us!" I don't think having my risk factors pointed out would affect me negatively at all, even if I think they're BS things the company shouldn't worry about, though I can see it possibly affecting the company somewhat negatively in that they're potentially giving the candidate who all-in-all they think is worth taking a chance on an extra reason to decline (and thus they'll need to continue searching which sucks). Whether that decline happens really depends on the candidate. With that information they might conclude/agree it's too risky to leave their current role for you right now, or they might actually be motivated to do better and start proving themselves right from the start especially at a company that is so forthcoming about such things. And they might fail, but at least they leave with the knowledge of what went wrong, and even if what went wrong wasn't an identified risk factor they still have your list of risk factors (some possibly valid, some BS, some that can be worked on, some that can't) that other companies might be looking out for to have for future interviews.

> 1) you can detect that like in the first day, or at least first week 2) the amount of damage they can do before detected is small...

But the cost is actually minimized already by rejecting in the interview. Just like finding a bug while writing unit tests is cheaper than QA finding it after you make a build, even if QA does so in the first hour of testing.

There's a tradeoff between the costs of trying to detect possible issues at various stages, and the costs of damage if those issues aren't caught until later. This makes the caught-earlier-is-cheaper analogy not always hold, because the costs you spend to detect an issue at some stage can easily outweigh the costs of the issue itself at any later stage. Hence the industry doesn't follow NASA practices, or even more reasonably make use of TLA+ as basic professional practice like unit tests have largely become. Many bugs have such negligible impact that they remain unfixed, regardless of when they were caught, regardless even of whether the customer found them or knows about them. We also realize that some bugs still make it past everything and get to production, so it's worthwhile to spend effort trying to reduce the cost of production bugs, which might for instance involve having a faster and smoother deploy process.

So in hiring, there are costs companies pay to avoid a bad hire, and costs that result as a consequence of bad hires -- which they mostly can only estimate because of bad hires getting through anyway, everything on the consequences side is formally probabilistic. You can think of it as hiring-filter-costs + P(bad-hire) * consequences-of-bad-hire. So yes, if P(bad-hire) is 0 because of your amazing hiring filter, then you'll never have to pay for the consequences-of-bad-hire. And supposedly, increasing hiring-filter-costs should decrease P(bad-hire), though over time we've found many practices that don't actually do that, and smarter companies have stopped doing them.

My thinking boils down to we can and should reduce hiring-filter-costs (which I want to think of as including opportunity costs and negative externalities on the whole industry, but it's not necessary), but fear prevents even experimenting at many places because there's an idea that any reduction will increase P(bad-hire) too much, and consequences-of-bad-hire are too large. So, why not focus more on identifying and lowering the consequences-of-bad-hire? Do that, and we'll also be able to lower hiring-filter-costs even if they increase P(bad-hire), which they aren't guaranteed to do anyway.

I wish America would adopt what I've seen in some European countries -- a paradigm where they introduce a fixed-time contract first, then only hire you if they like you after the contract. I have seen that in some jobs in America, but just not programming jobs.

It happens in programming jobs here but in different ways. The general style I think you're thinking of seems more likely to me for startups to try experimenting with (and more informally), but we also have a different style where you work for a general contracting agency who has you work for another company for a fixed term. Some of those after the fixed term can transform into that other company hiring you directly (and giving you higher pay not cut by the contracting company, benefits, etc. that you lacked as a contractor), or if not the contracting company helping you find a new company to contract for. The final style I can think of is already institutionalized at big companies for entry level roles in the form of hiring interns almost done with college. They work for you for a few months of the summer (often at equivalent 'hourly' salaries as a senior role) and if you like them, you offer them a full time position (at a lower entry level salary) starting next summer after they graduate. It can be a smart way to get talent that doesn't yet know it's great, or will become great, without paying a premium.

The biggest problem when hiring people with experience though, and why the probation style is pretty rare in the direct form I think you're thinking of, is that it's going to filter out the best people -- just like many other too onerous hiring practices filter them. They're just not going to put up with it when they have much better options. For Europe to the extent such a practice works at all is probably related to how universal it is for employees and employers to have a symmetric long period of notice of termination/notice of quitting that often ends up being at least a month if not 60+ days. In the US, every state is at-will, you can leave at any time and be dismissed at any time (within the bounds of certain legal protections, paranoid risk of violating which has driven larger companies to an insane PIP-style process of nudging someone out the door over a period of months). I think US companies and employees both should continue leaning in to that more as an advantage.

I have screened or interviewed hundreds of people over the past three decades. That is not a strong claim, it is simple truth. There is a large minority of so-called "professional" software developers who literally don't know what they're doing and can't anything but the dead simplest code.

They may only represent 2% of the population, but they are a much larger % of candidates searching for jobs. If you think about it a bit, you'll see why.

It's entirely believable that many programmers couldn't write fizzbuzz without searching Google in the days before it was universally memorised. The modulo operator is somewhat rare in real code and is defined slightly differently in different languages.

> The modulo operator is somewhat rare in real code

Even if a candidate didn't know that modulo even existed they could still get the remainder with simple division and subtraction. Modulo might be slightly different depending on the language, but for fizzbuzz that's not important and most likely not what you're expected to know in such an interview question. And while it is not the most common operator it is still one of the most basic ones and present in all languages you'd use during a job interview.

> Even if a candidate didn't know that modulo even existed they could still get the remainder with simple division and subtraction.

And they'd probably fail some interviews for doing so.

> And while it is not the most common operator it is still one of the most basic ones and present in all languages you'd use during a job interview.

This is true, but I also expect some otherwise-competent programmers would fail a test that expected them to remember the specifics of bitwise operators or bitshifting. It's been years since I've needed to use any of the three in a paid job - they're just not very common in my day jobs, whereas they crop up a lot in side projects.