One thing I realized recently while interviewing candidates at my company is that it’s one of the few opportunities that individual contributors like myself have to exert significant power over someone else, and not everyone is prepared to handle it well.
Recently, we had a candidate come through that wasn’t a good fit for us (he claimed to be a cypress expert so we asked him to set cypress up in a dummy react repo and he floundered the entire time). One of my coworkers took some pot shots at the candidate, being deeply critical and rude, cutting the candidate off mid-sentence, etc. I was honestly embarrassed to be on the call and I thought it reflected poorly on my company. I’d never seen my coworker act like this before and it changed the way I felt about him.
I wonder how much of the leetcode interviewing culture is rooted in this desire to dominate candidates, since ICs rarely have opportunities to wield such power over others.
> he claimed to be a cypress expert so we asked him to set cypress up in a dummy react repo and he floundered the entire time
If I was floundering because the interviewer found a hole in my knowledge (and there are many holes in Albert Hall) I'd proactively state that and say where the holes weren't and ask for a question about that.
For example, I've implemented a complete C compiler, front to back, so at one point I knew everything about C. But in implementing a new C compiler (ImportC) a few months back, I had to go over the Standard again, as I'd forgotten bits and pieces.
There's no reason at all why the interviewee cannot drive the direction of the interview, rather than being driven. For example, when I interviewed for a software job after leaving my mechanical job at Boeing, I was concerned that I wouldn't be taken seriously as a programmer. So I brought along listings of the code I'd written and pulled it out during the interview and proceeded to explain it to the interviewer, on my own initiative.
A long time after that, I was talking to the interviewer, and he said that that was what got me the offer.
> There's no reason at all why the interviewee cannot drive the direction of the interview, rather than being driven.
You know how they made a program pass the Turing test? They didn't make it smart, they hardcoded a conversation where the program drove the conversation to where it wanted it to be. That way the person interacting with the program was passive, so they never noticed that they were just interacting with a stupid program in the 5 minutes they had with it.
The same happens in interviews, if you take a person who don't understand tech, but have watched a lot of conference talks and read some papers and he can re-tell those, and he even remembers answers to common questions from the question parts there, but he doesn't understand much at all of what was said he just remembers conversations, then that guy will look like a world class expert if he gets to drive the interview. But once he starts working he didn't know anything at all, he just learned to talk about these things not do them. Some would say "just dive deeper into his knowledge!", but you can't always do that, if you aren't an expert yourself in his subject area you wont be able to drive down deep enough to find his flaws, and since he is driving the conversation most likely you wont be an expert on what he is talking about.
The person could have just learned to code in all that time spent watching conference Q&A sessions to con interviewers.
This does seem to be the threat model many interview processes are meant to protect against though.
I understand there are costs involved in hiring then firing someone, but maybe that should be reserved as a way to deal with the most outlandish fakers and let the rest of us have a quick Google or two during coding screens.
If you are smart and a hard worker then learning it properly is easier, yeah.
But if you are lazy and prefers talking to people and watching videos, so you spend your day talking about these conferences with your colleagues rather than doing work then you have the scenario I'm talking about. It doesn't require some genius conman, basically any guy who prefers to talk about things instead of doing things will more or less fit this description. The degree they lack real skills depend on the degree they prefer talking over doing, but such people will always over perform at such interviews.
Lastly since the talkers get rejected by all the leetcoding firms, you will have an abundance of talkers that can't perform in the hiring pool. So the problem with talkers is way worse today than 20 years ago.
When I have to game an obviously flawed interview because I want the job, I try to drive the interview. Make no mistake, this is a social skill. It’s not something I enjoy, it’s just a learned skill for me. But I do it because I want the job.
Often I feel like a con artist for doing this, especially if works to overcome a superficial interview. Sometimes it turns out my perception was correct and yet it turns into a better interview because I’m adding value to the interview where the interviewer had gaps…
What's really great is if you can get the interviewer talking about themselves. If you're lucky you'll spend the entire interview nodding and saying things like, "wow", "that's really cool. How did you come up with that?". At the end, if it's the kind of interviewer that will talk about themselves for an hour straight they're going to love you, just sat there and let them be the center of attention. The problem is do you want to actually work for someone like that.
This is how I got a job with a bunch of really smart folks in the early 2000s (they were all MIT grads and I was a graduate of a state liberal arts college in NY.) It was several rounds of interviews and a couple of the folks interviewing me (who clearly liked me) warned me that the interview at the end would be the hardest because the interviewer was an old non-nonsense engineer (MIT CS degree and MIT MBA.) It was intense for sure, but I had read somewhere that (as you stated) if you can get them to talk about themselves and engage them: A) they will love you and B) you'll run out the clock. I can't recall how I got him to talk about 'the good old days' of programming, but I did and I don't think I had to answer more than a handful of questions in that hour.
His feedback read like this: “Good culture fit, good soft skills. I really liked them. They seemed to understand and appreciate difficult and interesting topics.”
I assure you there are worse ways to behave in an interview. I’ve passed on people who were smart as a whip but nobody wanted to work with. And nobody wants a dead fish who just sits there waiting for the next problem to solve.
I suppose in my case he might have been less effective at assessing my technical abilities (there were multiple technical interviewers throughout the day.)
Watch out for this. It is a real phenomenon. If you "enjoy" thr interview, you are more likely to think the candidate did well. And if you spend a lot of time talking about yourself, you are more likely to think the candidate did well. Don't be fooled. Have a rubric that you can objectively measure the candidate against.
I agree that an interviewee should be an active participant (though not being is fine too given the state of interviewing), however attempting to drive it themselves can really backfire if the interviewer isn't expecting it and/or is bound by policy to actually ask certain questions or go through some process. I still remember one guy who wanted to drive things a bit too strongly, which for most of my section was fine with me because I was still able to get the data I wanted out of it and hopefully he got some of his, though at a point I really insisted he actually code a working solution to a problem in the IDE setup for him (requires like 15 lines of code, it's embedded in the context of a larger program) rather than continue to whiteboard/pen-and-paper general thoughts. He did, and I gave him a positive recommendation. But in the after-meeting my coworkers were less happy with their sections (one of them even brought up the sense he was trying to 'run the clock'), and some of them were still new to interviewing and didn't know how to handle a more 'aggressive' candidate and apart from not enjoying the experience couldn't get the info they wanted, so the eventual decision was to say no to him. I still think he would have done fine, but hey, it's just another element in how broken our interviewing practices can be.
The interviewee doesn’t know what the day to day job will look like, so they may wind up talking about entirely the wrong parts of their skills & experience. The interviewer has a much better idea what they need to learn about the candidate.
Perhaps this is less of a big deal for a generic position that’s the same at every company, but if you are matching someone to a very different role than they’ve held before, you may be interested in only certain pieces of their resume.
Walter you (with your credentials) do realize you would never go through the "standard" interview loop like the rest of us nobodies would go through right? We are lucky to be forgiven for only solving 2 LC hard problems instead of 3+ in an hour whiteboard coding interview!
I didn't have any credentials at the time, which is why I felt the need to be proactive in the job interview.
I also interviewed at Microsoft (just down the street) a few years later, and got the standard interview treatment.
I know what quicksort is, and I've implemented it more than once over the years. But I could not implement it off the top of my head at the moment without doing a quick refresher on the algorithm.
I was once assigned the task of speeding up a program written by <famous programmer you've heard of>. In perusing the code I happened to notice that there were two custom bubblesorts buried in it. I replaced them both with calls to C's qsort and got a nice boost. Was <famous programmer you've heard of> a fraud? Not at all. Anybody's code can be made better. I've been found guilty of having hidden quadratic code many times.
I have noticed that sense of "domination" come out in interviews as well. I remember long ago having one interview where the interviewer was asking me to code something which I knew would be impossible to complete in the time allotted. It just seemed so ridiculous to me that I was supposed to write this complete solution in about 30 minutes. I remember how superior he was acting at the start of the interview too, and then after I wrote up the code (unable to complete in the time given, but apparently good enough to make him happy), his tone changed completely of one where he was very friendly as though somehow I passed his "test". I was now worthy of speaking to as an equal, whereas half an hour or more earlier I was some nobody to him.
Another interview I had years later, the interviewer clearly didn't think I was worth his time because he literally got up and started playing with a yo-yo as he threw challenging question after challenging question at me and pressing me harder and harder for solutions ("Imagine you were given task X and nothing more. How would you solve this? What would you do next? If that didn't work, now what would you try?") I remember it was incredibly disrespectful and left a bad taste in my mouth about the team. Thankfully I didn't get the job.
For example: The idea that there are many programmers who cannot even write fizzbuzz sounds like a very strong claim. It seems far more likely that there are many programmers who have socially-induced anxiety.
I have interviewed senior candidates (more than two, fewer than five) who couldn’t write code to sum an array of ints (ignoring overflow). *
Not in the sense of missed a bit of punctuation, but rather in the sense of “couldn’t get started with anything that vaguely smelled like an answer, didn’t understand the type system of the language they claim 5 years of experience with, etc”.
I’ve probably interviewed fewer than 400 candidates in-person and maybe 200 of those would be senior ICs. To have more than a percent of them be utterly incapable here is a signal that there are candidates out there taking their best half-court buzzer-beater shot to try to land the prize. If you’ve done 100 first round in-person interviews and never run into someone that causes you to check up on how the phone screen happened, your screening processes work pretty well.
Is it possible that these candidates are capable programmers in daily work? Sure, it’s possible, but I’d bet against it.
* We used to ask a candidate to average an array of ints. It’s a question that was designed to tease out whether the candidate would discuss/inquire about overflow, what the return type should be, etc. After a couple years, we realized that candidates would stumble on the loop and sum more than anywhere else and we got more signal/minute from asking the vastly simpler summation question. Candidates who could fluently write the loop usually knew about overflow and the fact that the average of ints isn’t mathematically an int but that many times an int is what you want and can discuss the reasons.
I love this because I agree 100%. I accepted the idea that they were "out there" but only really internalized it after I interviewed someone with a Masters in physics who couldn't make any progress on my simple problem. (The problem does assume you can understand the concept of a geometric line, but I give the two line equations most people first run into in junior high, among other things, I'm not testing recall.) Especially on the part about making a question even simpler in the name of getting what you actually care about faster. I've also found it useful to combine things from separate questions into one question that can answer both in less time.
I still wonder if we couldn't remove most of these games though if we improved our hiring, onboarding, and firing processes in other ways than just trying to make a better hiring filter. Accidentally letting in someone wholly unsuitable for the job shouldn't be a big deal if 1) you can detect that like in the first day, or at least first week 2) the amount of damage they can do before detected is small 3) they actually start very soon after the decision to hire is made 4) you can fire them immediately after deciding a mistake was made and they don't work another day.
Startups can handle this better already and I think is reflected in a lot of them still doing more casual "single lunch conversation" style interviews before a "welcome aboard, see you on Monday" decision. For big cos, work will have to be done on getting away from the mentality of "it's ok if you're not contributing much of anything for the first x days because onboarding" where x is high (I've seen over 30), and firming up access controls (no facebook style all-access of new hires (or anyone if possible) being able to just query the production db and access the private messages of exes or whoever), and removing the "ok you're hired, see you in x weeks" policies where new hires can only start after a significant delay or all on a certain date for whatever reasons, and only getting rid of someone through a long and drawn out PIP process... Maybe it can be done some day.
On the “we can take a chance here” front, I’ve sometimes struggled with hiring someone out of a stable situation and putting them into a role where they might earn a termination. If someone’s already unemployed and that happens, they’re no worse off (and likely better off). If it happens that they resigned a stable job and that happens, they’re made worse off.
As a hiring manager or interviewer, I’m not their fiduciary, but when facing a risky hire, I wish there was a way to let the candidate make a more fully-informed choice without the knowledge that we think it’s a high-risk offer influencing them negatively in any way.
Right now, they interview and get an offer or don’t. If they get an offer that we think is high-risk, they have no reasonable way to know that.
Could you indicate it through your choice of interview questions?
Things like, "How do you feel about stretching your skills in the work force?" "What do you do when you make a mistake at work?" "How do you handle a situation where you feel out of your depth?" Etc.?
I know I would take that as a signal not to go for it if I wanted to retain stability in my life.
That's a great thought; however, we most often discover the conflict during the hiring debrief when we have mixed feedback on a candidate leaving the candidate "on the bubble".
It does give me some ideas to consider when we extend the offer (as @Jach's sibling comment also suggests).
Yeah, it's trickier when they're coming in with an existing job, and the current professional quitting culture dictates that you couldn't start any time sooner than in two weeks at least. Add that to the list of difficulties involved in lower time-to-start dates.
I've been fortunate (or not) that the hiring decisions I've played a part in have all been non-urgent with no struggle to find more candidates, so in the face of risk the org could afford to wait. The moral hazard you describe is interesting though. I don't think I would worry about it, as you say the hiring side isn't their fiduciary, and you don't necessarily know that their current situation is all that stable. I might try and assure myself that even if things don't work out, they're unlikely to be put out on the streets, since if they got past us (even with our "risky" assessment) they can probably get past someone else quickly enough, and possibly even go back to where they left depending on how stable it really was.
I'd consider being honest with them about the risk if an offer is extended. If I were on the receiving end, I'd prefer to be told before I accept something along the lines of "Honestly we're a bit unsure how well you'll adapt to our work [because of risk factors x], but we see your potential and think you'll rise to the occasion. We hope you'll come join us!" I don't think having my risk factors pointed out would affect me negatively at all, even if I think they're BS things the company shouldn't worry about, though I can see it possibly affecting the company somewhat negatively in that they're potentially giving the candidate who all-in-all they think is worth taking a chance on an extra reason to decline (and thus they'll need to continue searching which sucks). Whether that decline happens really depends on the candidate. With that information they might conclude/agree it's too risky to leave their current role for you right now, or they might actually be motivated to do better and start proving themselves right from the start especially at a company that is so forthcoming about such things. And they might fail, but at least they leave with the knowledge of what went wrong, and even if what went wrong wasn't an identified risk factor they still have your list of risk factors (some possibly valid, some BS, some that can be worked on, some that can't) that other companies might be looking out for to have for future interviews.
> 1) you can detect that like in the first day, or at least first week 2) the amount of damage they can do before detected is small...
But the cost is actually minimized already by rejecting in the interview. Just like finding a bug while writing unit tests is cheaper than QA finding it after you make a build, even if QA does so in the first hour of testing.
There's a tradeoff between the costs of trying to detect possible issues at various stages, and the costs of damage if those issues aren't caught until later. This makes the caught-earlier-is-cheaper analogy not always hold, because the costs you spend to detect an issue at some stage can easily outweigh the costs of the issue itself at any later stage. Hence the industry doesn't follow NASA practices, or even more reasonably make use of TLA+ as basic professional practice like unit tests have largely become. Many bugs have such negligible impact that they remain unfixed, regardless of when they were caught, regardless even of whether the customer found them or knows about them. We also realize that some bugs still make it past everything and get to production, so it's worthwhile to spend effort trying to reduce the cost of production bugs, which might for instance involve having a faster and smoother deploy process.
So in hiring, there are costs companies pay to avoid a bad hire, and costs that result as a consequence of bad hires -- which they mostly can only estimate because of bad hires getting through anyway, everything on the consequences side is formally probabilistic. You can think of it as hiring-filter-costs + P(bad-hire) * consequences-of-bad-hire. So yes, if P(bad-hire) is 0 because of your amazing hiring filter, then you'll never have to pay for the consequences-of-bad-hire. And supposedly, increasing hiring-filter-costs should decrease P(bad-hire), though over time we've found many practices that don't actually do that, and smarter companies have stopped doing them.
My thinking boils down to we can and should reduce hiring-filter-costs (which I want to think of as including opportunity costs and negative externalities on the whole industry, but it's not necessary), but fear prevents even experimenting at many places because there's an idea that any reduction will increase P(bad-hire) too much, and consequences-of-bad-hire are too large. So, why not focus more on identifying and lowering the consequences-of-bad-hire? Do that, and we'll also be able to lower hiring-filter-costs even if they increase P(bad-hire), which they aren't guaranteed to do anyway.
I wish America would adopt what I've seen in some European countries -- a paradigm where they introduce a fixed-time contract first, then only hire you if they like you after the contract. I have seen that in some jobs in America, but just not programming jobs.
It happens in programming jobs here but in different ways. The general style I think you're thinking of seems more likely to me for startups to try experimenting with (and more informally), but we also have a different style where you work for a general contracting agency who has you work for another company for a fixed term. Some of those after the fixed term can transform into that other company hiring you directly (and giving you higher pay not cut by the contracting company, benefits, etc. that you lacked as a contractor), or if not the contracting company helping you find a new company to contract for. The final style I can think of is already institutionalized at big companies for entry level roles in the form of hiring interns almost done with college. They work for you for a few months of the summer (often at equivalent 'hourly' salaries as a senior role) and if you like them, you offer them a full time position (at a lower entry level salary) starting next summer after they graduate. It can be a smart way to get talent that doesn't yet know it's great, or will become great, without paying a premium.
The biggest problem when hiring people with experience though, and why the probation style is pretty rare in the direct form I think you're thinking of, is that it's going to filter out the best people -- just like many other too onerous hiring practices filter them. They're just not going to put up with it when they have much better options. For Europe to the extent such a practice works at all is probably related to how universal it is for employees and employers to have a symmetric long period of notice of termination/notice of quitting that often ends up being at least a month if not 60+ days. In the US, every state is at-will, you can leave at any time and be dismissed at any time (within the bounds of certain legal protections, paranoid risk of violating which has driven larger companies to an insane PIP-style process of nudging someone out the door over a period of months). I think US companies and employees both should continue leaning in to that more as an advantage.
I have screened or interviewed hundreds of people over the past three decades. That is not a strong claim, it is simple truth. There is a large minority of so-called "professional" software developers who literally don't know what they're doing and can't anything but the dead simplest code.
They may only represent 2% of the population, but they are a much larger % of candidates searching for jobs. If you think about it a bit, you'll see why.
It's entirely believable that many programmers couldn't write fizzbuzz without searching Google in the days before it was universally memorised. The modulo operator is somewhat rare in real code and is defined slightly differently in different languages.
> The modulo operator is somewhat rare in real code
Even if a candidate didn't know that modulo even existed they could still get the remainder with simple division and subtraction.
Modulo might be slightly different depending on the language, but for fizzbuzz that's not important and most likely not what you're expected to know in such an interview question. And while it is not the most common operator it is still one of the most basic ones and present in all languages you'd use during a job interview.
> Even if a candidate didn't know that modulo even existed they could still get the remainder with simple division and subtraction.
And they'd probably fail some interviews for doing so.
> And while it is not the most common operator it is still one of the most basic ones and present in all languages you'd use during a job interview.
This is true, but I also expect some otherwise-competent programmers would fail a test that expected them to remember the specifics of bitwise operators or bitshifting. It's been years since I've needed to use any of the three in a paid job - they're just not very common in my day jobs, whereas they crop up a lot in side projects.
Asking a doctor who had completed their 12 years of medical school and residency basic biology questions would be a bit absurd.
Asking someone who claims to be a doctor of natural medicine the same might yield you more than half unable to answer.
Since there is no 12 year stringent accreditation process for being a software engineer, I'm not sure why this is a good comparison. There are many people who don't know so many basics of CS who have finished degrees, bootcamps, or have been employed in those roles for years.
Doctors simply aren't able to get accreditation or employment with the same ease as software developers.
It is a strange comparison, though I don't think the "stringent process" is the issue and it's more just that they are entirely different fields with different ways of working with different kinds of feedback in the broader economy with different levels of observability. You might find that even the doctors that had years of study and paid supervised practice (residency) are unable to answer basic things. From an MD last year: "Even doctors forget the relationship between calcium and Vitamin D, and I have spoken with doctors who did not realize that 5000 units a day could cause problems. As a doctor, there is a lot to know and remember, and most doctors haven't studied calcium metabolism in many years." (https://news.ycombinator.com/item?id=24065115)
Some people avoid the dilemma by only interviewing candidates from prestigious CS programs and with previous employment at famous companies, locking out those without the "right" credentials.
If you're going to demand that you be treated as an honest estimator of your own abilities beyond doubt don't be surprised if that ends up eliminating opportunities for people who are trying to break into the field without those credentials that put their abilities at a near certainty.
I still recall an interview I did at a recently public company in New York, for a DevOps position. 4/5 of the interviews went sailingly, with heavy emphasis on networking, scripting, and AWS knowledge. Then the lead engineer sailed in, five minutes late. He gave me an algorithm question straight from the Google playbook (I don't remember the question, sadly), then after I asked for a few minutes to ponder, proceeded to smirk and interrupt me every 30 seconds to ask if I needed help. In between he was chatting on his phone with the audible keybord ticks enabled.
This happened to me at a recent Google interview. It was the last interview of 5 and the guy gave me a gnarly graph problem. I was trying to wrap my head around it, but he kept interrupting me and giving me unwanted help. I ended up getting writer's block and just mentally shutdown because he wouldn't let me have time to think. He ended the interview early because of poor performance.
It made me feel like I didn't know anything and was very embarassing. It also changed my outlook on Google, and I'll never interview with them again.
> I wish there was a way to report bad interviewers.
You can always tell the recruiter/coordinator. It's just tricky because you don't want to burn bridges, doing it before you hear back could be read as you asking for a do-over, doing it after could come across as bitter.
We are hiring at my company. We ALWAYS value feedback. Last week we heard someone had a bad experience and the feedback was taken seriously, and without resentment.
If you have a good or bad experience, PLEASE consider telling the recruiter know.
Some companies will ask for feedback after the interview and there's sites like Glassdoor.
The last place I did interviews at, we'd bring all interviewers into a quick debrief meeting and talk about whether to give the candidate an offer (going off the notes we took during the sessions). It was pretty obvious if someone had a particularly bad session and could be weighted against other peoples' input
Please go off in a corner and ponder why such casual ageism is as unwelcome as casual racism or sexism and maybe just stay there until you come up with an answer.
Substitute woman for boomer in that sentence and listen to what it sounds like.
I could substitute it for child, another group of people who are broadly less likely to know where the setting is, and it wouldn't be offensive that way either.
As someone who was born before the moon landing and saw one of the last IMPs get installed on the ARPANET, yeah that was offensive. You should apologize.
Knowing to silence your phone in meetings is a matter of having manners, not age.
See the capacitor in this closeup that kens took of a power supply module in the Apollo Guidance Computer? My dad's company made it. You are telling the wrong person that the landing was faked.
Just as no-one would begrudge a dominated race from poking fun at the dominant race, I don't see why you are so on your high horse about the dominant age group getting the same. 'Punching up' is fair game where I come from.
I'm afraid I just don't get your outrage at an innocent comment poking fun at the most dominant section of society. But, as you're the second person to have taken offence, I'm going to delete it anyway.
First of all, I'm not a particularly big fan of generational stereotypes in general, especially on a board like this that aims to do better. If I made a comment to the effect of "Zommers are the most entitled group I have ever seen," I would be (rightly) most likely downvoted to oblivion.
Secondly, I have never encountered it as a problem--at least in a way that affected me (never could really get into a dot-com company but we know how that mostly ended up). But that's mostly because I've had various roles where experience really trumped other things. But ageism is clearly an issue in tech broadly so I take issue with with the justification that it's OK to make casual cracks with the justification that it's punching up.
To say nothing that it added absolutely nothing to your comment. Rude interviewers are the issue. Not some imagined inability for older individuals to silence their phones.
Finally, I'm not sure doubling down on the oppressed race thing helps your argument--especially given what the age breakdown in tech looks like.
Ah yes, racial humor. That will never fail you in business provided you deliver it the appropriate way. Good luck with your interviews, you’re going to do great!
I recall voicing an opinion that we should record interviews we give candidates and re-review it during the hiring committee, much like we do code reviews. Have the interviewer justify certain beliefs about the candidate with hard timestamps rather than platitudes ("not confident", "wanted to see how they react under pressure", etc. etc.).
My comment always gets pushback. Privacy, respect for interviewer ("If I work here, then I myself can judge fine!"), not enough time, etc.
Hiring processes are about as contentious as CI methodologies, it seems
I don't agree with recorded video, but I am now convinced that there should always be an random "observer" (interviewer does not get to pick) who does nothing but watch and take notes along with an interviewer.
You can give whatever excuse you want "Adding a new person to the loops," "normalizing the loop level," whatever. But I am now convinced that interviewers cannot be trusted alone with candidates anymore.
Maybe I'm naiive that they ever could be, but if one of my guys did half of the stuff people keep reporting to a candidate and I found out about it, I'd have his head and maybe his badge.
Fsck you very much whichever FAANG HR department decided that "interview loop" was something to be added to corporate evaluations for points and then promulgated it to other companies. Thus making both sides of interviewing even shittier than it was before.
Isn't interviewing in pairs fairly common? When being interviewed, there's always been two people across from me for the technical rounds; anything 1-on-1 is limited to initial phone screens and final rounds with the bigwigs. When conducting interviews, I've always been one of two, with the exception of one time when no one else was available to join me, which I'd prefer not to repeat.
I would go so far as to say a random passive observer is still a great idea in addition to two active interviewers.
It wasn't at any point that I knew. Interviewing is an expensive operation--it ties up quite senior folks for a fixed amount of time. If you pair that, you double the expense.
And it seems like phone screens are exactly the point where you most need a pair from these anecdotes.
I'm not comfortable with the idea of being recorded, especially when I'm already under the pressure of an interview. I don't know how they'll use my video, if it will be used in training, leaked, etc. I might never see that company again, but they could use my video forever without me knowing.
There is mostly no such things as "optional" in those sorts of situations. It's like "Here's a take home project. It's purely optional but it would give us a better sense of what you can do."
In other words, maybe if you utterly wow us in-person you'll get in anyway. But probably not.
If I thought my interview was being recorded, I'd assume it was being used for this internally. If given the option to have it recorded, I'd assume that I'd get "minus points" for declining.
This is very unfortunate, because I love your idea. I'd just need stronger legal protections to ensure that I wasn't perpetuating part of a dystopia.
Large companies have a fairly rigid process with strict rubrics for interviewers. In general the goal as I see it is to remove as much individual decision making as possible. Doesn't stop these issues as the comments here show.
You also then encode a single group's biases (the rubric) rather than a mix of everyone's biases. In my experience that doesn't actually make things better overall just slightly more consistently bad.
I hadn't done many interviews before starting at my current job, where they seem to make a point of getting a lot more people involved in the interview process. I guess this is an idealistic view, but it should be pretty clear if you're a hiring manager whether the people you're getting involved in the interview process are likely to be jerks to a candidate.
What you describe is embarrassing, and does reflect poorly on your company. Review sites like Glassdoor do or did have interview reviews as part of their company review, and people do go look at those when deciding whether to apply at a company.
I hope you shared your feedback with your coworker, and also the hiring manager. That coworker is probably not ready to do interviews and needs some training (or maturing) first. In the future, you might interrupt the coworker and then try to help the candidate along; maybe they won't pass the interview, but you don't have to let them leave feeling like a failure, and along the way you might get some information about where they could fit in your organization. And the entire interview won't have felt like such a waste.
Not many people are gonna risk confrontation with a colleague over some candidate they don't know. Especially not during the interview, maybe afterwards when things are calm.
I'm agreeing with you here but let's stay realistic; some companies have assholes working for them, sometimes these assholes get to positions of power. Usually the company knows damn well employee X has a bit of an attitude problem but...guess he adds enough value (or acts well enough) to keep being employed.
This is very insightful. I'd also add that having such people let loose to do interviews without training and checks and balances also is a huge red flag about the org.
They could just need training and feedback. A lot of places just throw any rando engineer into an interview with no coaching on how to behave, how to set guide rails for the candidate, how to empathize and pace the interview, what the evaluation criteria and rubric is. And then after the interview, the person has no idea if he conducted the interview properly. Totally open loop. Just yeet him at candidates untrained and then wonder why he can’t interview effectively.
I have been doing React development for 4 years now and I have never set it up from scratch and only have a rough idea how it’s done. I still apply for react jobs and do fine. Installing something from scratch is one of the least important tests because it’s something that companies typically do once every few years, they can afford to flounder for a few days on that.
I reinstall Linux from scratch about once every 5 years, long enough that I totally forget how to do it. So I keep a file with all the commands to do it.
This post ought to be the final nail on the coffin of the cult of leetcode/whiteboard style inverviewing.
Just looking at this guy's repo and his writing style -- you'd be foolish to waste his time or yours on leetcode-style hazing. Your conversation should be strictly high-level. If you must, pick a file at random from one his larger projects, and ask a few straightforward quetions ("I'm new to Go - can you explain why I would use this datastructure here?") just to make sure the repo isn't a copy of someone else's, and he isn't outright trolling you. Apologizing as you're doing so, of course ("Sorry, but do you mind if I ask a silly-sounding question or two about something in your repo?").
That's all you need to do. Most candidates won't have work samples that are quite as slam-dunk, of course. But still, your own "smell test" can save you tons and tons of time (and the candidate, agony). It's really, really hard to fake one's way through questions like these -- and really easy to tell if someone is trying to do so.
Put another way, your basic mindset should be: "Trust, but verify". Not: "Joel Spolsky wrote that post 20 years ago claiming that 99.5 percent of all candidates are utterly incompetent. Therefore I need to torture this guy until he confesses."
At least as far as answering the "can he/she code?" question is concerned.
Im not a huge fan of leetcode or work samples but on the receiving end it weeded out a huge number of bad candidates. We'd get tons of copy/pasted together infra as code from top Google results that wasn't even close to runnable (usually being close to runnable with a few small bugs was a good submission and runnable with no bugs was a great submission)
> You let one bad, he will hire more in the future.
When your organization can recognize a poor investment but cannot act on it, you have an organizational problem. Why would someone get to hire more "bad" (for whatever that means) if you can recognize it? Even if you're doing all the hiring yourself, it's unlikely you'll have a 100% record over time.
For one thing, in many countries it's hard to get rid of a poor employee unless they're malicious or really exceptionally terrible.
For another, even if "an organisational problem" or inconvenient labour laws don't get in your way, firing employees can have a serious effect on the morale of those left behind. And for good reason! Usually companies that regularly fire people for poor performance has a different sort of culture than companies that don't. In my experience, that's the worse organisational problem.
The right solution is just to be more careful about who you hire in the first place.
When they put it in the promo committee calibration doc that X level has done Y interviews per quarter, they end up on interview loops but then feel they must say no to Z% else they’re not “raising the bar”
Isn't this why there are trial periods ( no idea if it's a thing in the US with the "right to work" thing)?. Over here in the EU a work contract can't be terminated from the company just like that, there needs to be a specific reason ( like downsizing, or a serious error), and you can't quit just like that a mandatory notice period is required ( of course negotiable depending on the situation), usually in the 1-3 months range. Therefore there's a trial period at the beginning, 3-6 months usually, during which any party can end the employment with a very small notice ( a day or two).
So if someone lied their way through interviews, in the first couple of months you should be able to detect that and terminate their trial period.
> So if someone lied their way through interviews, in the first couple of months you should be able to detect that and terminate their trial period.
Managers hates firing people, it is really uncomfortable. So instead of firing this person to look for a new, they keep the person and say they need even more headcount and hope the next one is better.
In the US you are always on a trial period. The problem is that firing people is annoying and bad for team morale. More importantly even dealing with a bad coworker is really bad for team morale. No one wants to join a company that is likely to fire them in 3 months because the company f-ed up their own hiring process. No one wants to recommend their connections join a company where that is likely to happen. Made worse due to a lot of compensation is tied to staying a year at a company so leaving early is basically getting paid 50% for that time.
What is bad for morale is keeping someone around who is a bad fit. Netflix has solved this problem by paying everyone at the top of the scale and firing everyone who doesn’t meet their bar.
The question is whether or not your interviewing practices filter out bad candidates. If your entire company has no employees then maybe but I find that hard to believe.
A lot of candidates don't have public repos and you need an equal comparison across all candidates (you can introduce a lot of bias trying to gauge two candidates against different sets of criteria)
Long before open source was a thing, and repos were even thought of -- I always had a few 1,000+ line personal projects I could dig up and show people.
right, and plenty people don't have large personal projects, or not personal projects that align with their work skillset, and the question is extremely vague what scope you're looking for.
People say this but then they reject answers that e.g correctly code the brute force solution or they don’t provide an algorithm to code for, so it’s not just “can you code,” it’s can you find the optimal algorithm and also code it, and usually the algorithm isn’t trivial or one you’d ever run into at work.
This is my experience hiring developers. If I hire without a code screen, it's a 50-50% chance the developer cannot close a ticket in their first month. With a code screen, I've had one developer not work out (personal problems). And I'm talking about "ensure that no trailing spaces are being inserted to the database in this csv import function" kind of tickets.
>usually being close to runnable with a few small bugs was a good submission and runnable with no bugs was a great submission
This is exactly what we do: check the solution, and if it's close we continue the hiring process. There is a huge difference between an almost working solution and what you get 70% of the time: code that will never, ever, ever work, even if it could run.
A lot of hiring managers have no concept of experiment design.
You can’t invent a measure (or copy it from a blogs about how other companies hire), look at who it rejects, and conclude they are bad. You don’t know if they were good or bad. You never worked with any of them. You only know that you applied your measure.
The only candidates you have any information about are the ones you hired, yet you are insulting the others on the internet today
You never worked with any of them. You only know that you applied your measure.
What's interesting is that the people doing all this "weeding out" seem so sure that they're bravely defending the fort against an onslaught of incompetent hordes... and yet they're missing this extremely basic point about the nature of the process they've chosen to use.
I'd say much has changed since he wrote that, and even for the time it contains some odd advice which he tries to force down the throat of the reader, but I'd be more interested to hear other's thoughts on the article.
To Joel Spolsky's defense, leet code questions where probably a great idea for him, when it was not an industry wide thing.
He also had this, what seems to me, nightmarish, dev. speed and progress estimation tracking app he made. I don't remember what blog post, but he described it and jokingly wrote that Lisa needed to get the speed up, or something. Fast forward to today and it is a predecessor to our agile nightmare. But I don't think that was his intent.
I remember when he made that post ("Evidence Based Scheduling"), and was also appalled. Yet, here we are today and programmer velocity is a now a measured metric in Agile.
The general problem with Scrum like velocity tracking is that devs will start to lie to hit the estimates to keep the commissars of their backs. The best liar gets the straightest graph. And as devs learns to lie, you get the impression of improved estimation.
"Some developers (like Milton in this picture) may be causing problems because their ship dates are so uncertain: they need to work on learning to estimate better."
I mean ... why God. I don't know but maybe poor Milton has the hardest tasks, does not cut corners or what not. Brandon seems to "get it" and has silly narrow error range - I wonder how ...
Hard to explain the absence of something other than it's just not there. The list of projects/accomplishments are fine but the reality is that people lie about stuff like that. I poked around their recent GitHub commits but it was mostly bug reports. Nothing really that points to being obviously great at coding.
I see that on paper they're a great candidate, but the reason that companies do these tests is that often the candidate doesn't measure up to the resume
Not gp, but obviously yes. I'm yet to meet someone who can pass a leetcode-style interview with sober and conscious interviewers, but can't code. Have you?
Leetcode and coding ability is an A -> B but B -/-> A situation: failing leetcode doesn't mean you can't program, but the reverse is absolutely the case.
Leetcode can't filter out other potential issues that may impact success (e.g. poor anger management, or poor judgement).
I'm yet to meet someone who can pass LC, but can't code.
I should perhaps be clear that by "overwhelming proof of ability" I meant "the ability to program well". Which by definition does intersect with one's capacity for judgement, on a certain level. (Though is largely orthogonal to other behavioral issues, I find).
I see LC as providing proof of, at best, minimal ability. And I've seen plenty of people who can run circles around LC-style tasks, but when it gets to the bigger picture -- you know, writing good, maintainable software -- somehow, it just never comes together right.
We agree that obtaining proof of minimal ability is important. I just think that proof can be obtained in other, far less mutually degrading (and error-prone) ways.
Your post ought to be ample proof that subjective opinions (like the one you display here) are not solid grounds for hiring decisions.
When looking at this person's Github page, I -unlike you- see a vast number of trivial projects that can best be described as regurgitations of other people's work. Even worse, if you actually look at his first pinned project (https://github.com/kevinburke/nacl), you'll see that it is nothing but a wrapper with trivial changes/updates... an anti-project.
In the end, I see code that provides the illusion of competence but, when actually inspected, is actually telling me not to hire this person. The fact that pretty much anyone can declare oneself "a software engineer" today and easily present a veneer of competence together with powerful financial incentives, makes it absolutely necessary for any serious engineering team to have objective measures in place in order to evaluate candidates.
There are so many terrible engineers out there (and some even slip through the cracks and land at places like FAANG) that doing anything less is irresponsible.
I never said you should just hire someone on the basis of their repo.
But rather, if you're looking for a basic smoke test which can answer the question "Can this person actually code? Do I want to spend more time engaging them?" -- there are better ways to do that than expecting them to cram on a list of algorithms to recite over the phone. Like looking at an actual work sample.
The fact that you looked at this work sample -- and came to your own conclusion about whether they would be worth engaging -- validates this very simple and basic point.
Which is your call of course. He may not meet your bar, but it would be absurd to say this guy can't code, and hasn't been around the block in terms of general CS concepts.
That said -- my own subjective opinion is that you're being exquisitely savage in your assessment of his NaCl implementation. Yeah, it's wrapper -- like it says in the Readme, and like a lot of working code out there. It's a pretty established way of solving a problem, in fact -- in some cases a very efficient and elegant way. It's not the same thing as writing 20k of primitives from scratch, of course. But that's not what he's presenting it as. He's not saying it's a "tour de force". He's saying "I think this shows we can talk about other things besides FizzBuzz".
And then other things you're like saying... like "an anti-project"? Wow.
Engineering is not about whether someone "can code", but whether someone can _engineer solutions to problems_. Has this person demonstrated that through the projects he chooses to put on display (and advertise as signal for his ability) on Github?
This is also what a lot of those engineering interviews are trying to ascertain, in a quantitative manner. I couldn't care less about someone's degrees or ivy league schools or Github projects (in isolation). These are all secondary -if that- considerations, problem solving ability being primary.
Has this person demonstrated that through the projects he chooses to put on display (and advertise as signal for his ability) on Github?
Using the example you disparaged (his NaCl implementation), I would say:
"On first appearances, clearly yes. The problem was to adapt the a set of standard Go functions to a foreign interface - something one does quite a lot in an engineering environment. Also, the fact that he's aware of projects like NaCl and seeks to learn from the likes of Bernstein and Lange is a positive signal."
If I wanted to, perhaps I could drill down into the code and/or the NaCl spec itself. I don't know Go, so for all I know, maybe his code is actually horseshit. But all I'm saying is - from first appearances, definitely a positive signal. Infinitely more informative that FizzBuzzing (which would insult both his intelligence and mine).
This is also what a lot of those engineering interviews are trying to ascertain.
I've already made it clear that a glance at a repo does not obviate the need for an actual interview.
So I wrote that because a client had a bunch of existing nacl messages that were signed using Python, but the existing Go library uses a different protocol to sign messages than the standard Python library. The existing Go library also didn't support all of the API's supported by nacl, I figured if I was running into this, then other people would too. Clearly based on the number of stars it seems like a lot of people _are_ finding it useful.
I guess in your eyes I should have also done the cryptography, however, I am not a cryptographer or a mathematician and one of the first things that you learn when you start doing work with cryptography is that you shouldn't roll your own crypto.
I decided to make that library prominent because in my consulting practice I saw a lot of clients using cryptographic primitives that were clunky and/or had a lot more overhead, for things like cookies or public key authentication or just sharing messages. nacl is difficult to use incorrectly, and is fast and efficient in terms of overhead. I wanted to promote use of nacl more than the work that I was doing for it in particular.
For that client (a smart lock company), I rewrote the backend, that someone else wrote in C for a specification that did not match the needs of the business. The C software was extremely difficult to read and maintain - socket writes in a callback in one part of the code and socket reads in a completely different part of the code, shelling out to curl, simple Mongo queries that would take 50 lines to set up. As a result the engineering team could hardly make any progress deploying new features. I ended up rewriting the whole backend in Go, a few thousand lines of understanding what the C code was doing, and porting it, to the point where the team could actually release patches and new features on a regular cadence, and figure out what on earth was going on with the Mongo queries and shell-to-curl.
That's not public though, if you're looking for public work, you could look at Logrole (github.com/kevinburke/logrole), a HTML client for your Twilio call, SMS, phone number, and recording data. Of course, you could say "that's just a regurgitation of whoever did the API client", which is true-ish, but I also wrote the API client (github.com/kevinburke/twilio-go). Of course, you could say "that's just a regurgitation of the Twilio API," but I also helped build and maintain the Twilio API, as well as the documentation and information architecture (https://www.youtube.com/watch?v=sQP_hUNCrcE), and the client libraries (https://www.youtube.com/watch?v=C_UJHqR_2Mo).
Most recently I was hired as the first engineer at Meter (meter.com) where I built out the entire backend API, the deployment infrastructure, the CI/CD infrastructure, a pipeline for the in-the-field servers to update themselves, and a method for access points to retrieve configuration from the backend. All in all this was probably about 30-50,000 lines of code over two years. I think the API has had one customer facing error in two years and no downtime.
It is true that anyone can call themselves "a software engineer" but not everyone can get contracts for a sole member consultancy and, crucially, get their contract renewed multiple times at every single client. If I was selling snake oil and not delivering anything, that would not have happened.
> This post ought to be the final nail on the coffin of the cult of leetcode/whiteboard style inverviewing.
Looks like he has ADHD. Maybe somebody with more time can go through his blog post line-by-line and figure out what's going on.
Most of my Coderpad stuff compiles and runs the first time, so I don't use his incremental test method, or see a need for it. Maybe he learned programming on heavy-weight IDEs and got trapped mentally in those?
I generally write some naïve code and if it doesn't work, get out the debugger and fix it from there. I can work a lot faster when everything is laid out and I don't have to guess about what the data structures are going to look like. If it's performance sensitive like some tests I've seen, I'd use a profiler. I often use the jetbrains find and replace tool, as well as the search tool, I also like that I can click into the source code for libraries that I use, so I don't have to guess about the interface or implementation. Stuff like coderpad takes away a good portion of my tools that I would use day to day, I taught myself to code in notepad, but I purchased the full suite of jetbrains tools years ago so I wouldn't have to fumble around with inferior tools.
A profiler will tell you whether a O(n log n) algorithm with a very high constant factor is better or worse for your situation than an O(n²) algorithm with a very low constant.
It's the other way around, from my experience. People who can write a program without any setup can solve it even on paper, while people that are hardwired to specific setup or IDE quickly froze when you ask them for something.
> People who can write a program without any setup can solve it even on paper,
As someone who had courses in C on paper, i can only say that it's a colossal waste of time. Tooling exists to make lives easier, gain time and productivity, why avoid it?
It'd be like saying only the farmer who can do everything by hand without any tools increasing their productivity is a real one. Or an accountant refusing to use Excel/calcuators/ERPs and writing everything on paper.
And before anyone jumps to conclusions, i often write small scripts with vim with a minimal setup, but when I'm doing anything complex i prefer to use an IDE
> It'd be like saying only the farmer who can do everything by hand without any tools increasing their productivity is a real one. Or an accountant refusing to use Excel/calcuators/ERPs and writing everything on paper.
You're trying to map manual work onto engineering.
And absolutely I'd rather work with accountnant who can write me on paper without Excel/calculator/ERP his logic.
It's like you're being obtuse just for the sake of the argument. Someone who can clearly express themselves without specific medium, be it paper, IDE or whatever is strictly superior to someone who is tied to one specific medium.
When one of the mediums actively helps and avoids small human errors, I'd prefer the person knowing using it instead of sticking an old and imperfect method out of pride.
> How does it actively helps and avoids small human errors in reasoning.
Because reasoning is only a part of everything to be done. An IDE can help with typos, best practices ( global variables, lack of error checking, etc) which are tedious things that take time. Isn't it better to the things that matter instead?
I preface my interviews saying that I'm we're going to have a conversation rather than a list of trivia questions. The goal is find out what the candidate knows, rather than if they know an answer to any specific question, or specific version of a technology. If I ask a question that is too easy, it's not a trick, it's a segue to more conversation. If I ask a question that's incomprehensible, it's because I don't have a prepped list in front of me and I'm making some questions up on the fly, and the candidate should just ask for clarification. Also, that a lot of my questions are open ended, or have multiple right answers and any of them are acceptable.
It's not really a pattern of interview I expect most people can implement immediately, but it really gives a good idea of fluency. I just have to home in on what they do know. The easy questions get some momentum going and provide context for the follow ups. When they don't know an answer I'll tell them the answer. Sometimes this leads to them adding some depth beyond that answer as they know what's going on but haven't worked with it formally, or have worked with it in passing, or maybe know it by a different name.
I've only ever done this style of interview and I've never hired someone who was downright awful.
I also suspect it makes candidates more likely to accept because you can develop a bond and have a relatively enjoyable time in interviews like this.
Several times I've had the interview process be so unpleasant I lost all interest in working for the company. I try to be considerate and withdraw so I don't waste everyone's time but once I did get an offer from a company like this.
Very much this. Leetcode is terrible, but the checklist of trivia suggested in the essay is equally terrible, because it's too easy for a genuinely good programmer to simply not have experience with topic X. Or, for more experienced candidates, to not have dealt with that topic for several years and not have the knowledge fresh.
Sure, Ruby on Rails. I literally start with something like:
"You've just created a new Rails app and you change into that directory. What are some of the directories and files you find at that top level?"
"What do you find in Gemfile? Gemfile.lock? What's the difference between the two?" Maybe some discussion of resolving dependencies/versions.
There are numerous "right" answers to which files/directories are at the top level and I don't care which ones they get, but they'll get some and it gets the flow going. Most of them can be taken to greater depth. If I want to guide it away from views into models, at least they know the level of detail we're working at.
Rails is a problem interview, too, because there is so much which is take it or leave it. There are Rails apps with no front end, and someone may have not worked with views for years, or the backend may be Api based and they haven't worked with models in the same way. I've worked on a project where 95% of the work was in the workers, and the user facing code was just nudging things between work states. So I use this to guide the conversations and draw out examples of their experiences when possible.
The best interviews are designed not to filter out bad candidates but rather to allow good candidates to demonstrate that they are good. The best companies know that and design their interviews in that spirit.
Some technical test interviews appear pointless to experienced candidates because they are designed to give an opportunity to someone without any experience to show that they know enough to get started. That's how some companies can hire people without work experience.
Many (most?) companies are simply bad at hiring and use poorly designed cargo-cult test interviews. They have a vague notion that more successful companies use test interviews so they try to do the same, only without designing an actual effective hiring process. As a result, their hiring is essentially random - they interview some people, which they put through a test interview ritual, and then hire a random selection of these people. Even if you can get a job at such a company, would you want to work somewhere where you'll end up with randomly selected colleagues? These also tend to be the companies that have high employee turnover - they have to let underperformers go to compensate for their random hiring, and the best people leave because they don't like their randomly selected team.
If you have a realistic view of your own skill and fit for the role and the interviews don't make sense then either you're overqualified or it's just a bad company or both, and you wouldn't want that job anyway.
Heh. I have 25 years experience and I fail phone screens too. :-D
It's fine, though. TFA goes into ways to improve the hiring process such that this sort of thing doesn't happen. But I don't mind it at all. I want to fail fast; if the company decides I'm not the right candidate, great, I'm glad it happened before we wasted time with more extensive interviews.
There are two possibilities. If they're right and I'm not a good fit, I don't want that job because I won't be successful there. If they're wrong, I don't want that job because the company isn't being successful at hiring me. The hiring process is looking for the wrong attributes, or the interviewer isn't properly trained, or... well, something, somewhere has gone wrong. There are no false negatives.
A company that's firing on all cylinders does a lot of hiring, and is good at it. They source solid candidates, and interview them thoroughly but quickly. If they're not hiring effectively, well, they're probably dysfunctional in other areas too.
Totally agree with this. One may want a Job at a prestigious FAANG or a hot startup but may not clear for culture fit whatever even tough you answered all technical questions. I say don't despair. just as you mentioned even if you got in it may not have been a good team/good manager. We should strive to find teams/managers who can appreciate what we are and do and vice versa.
It's important to remember that it's also YOU interviewing THEM.
The coding interview is an opportunity to get to know the company culture in a very feet-on-the-ground fashion. How the interviewer behaves and how they expect you to do things reflects upon them as managers, mentors, and in general people you'll be working with for potentially a long time.
- If they're forcing you to go against your natural flow, that's a red flag.
- If they're preventing you from building or explaining things as you'd normally do on the job, that's a red flag.
- If they don't allow you to consult the resources you'd normally do, that's a red flag.
- If they prevent you from testing your work and assumptions as you'd normally do, that's a red flag.
- If they're only looking for work and no conversation (all questions to you and none to them), that's a red flag.
I've had times during an interview where I've had to pause everything and explain that this is how I work, and how I would work if I were hired. Usually the interviewer will stop, reassess, apologize and we move on no harm no foul. I've also had occasion to stop the interview and say that we're not going to be a good fit, sorry and goodbye.
It's on YOU, the interviewee, to make sure that you're not getting bullied or coerced (either accidentally or deliberately). This is not a one-way street. If you don't push back, at best you get hired into an abusive or incompetent environment, and at worst they reject you after toying with you, wasting even more of your time and sapping your morale. Yeah, it's a bad time when one hires a bad employee, but it's MUCH MUCH worse to get hired into a bad company! You're taking a bigger risk than they are, so make sure you interview them well!
Breaking off an interview is not a bad thing; you're assessing THEM as much as they are assessing you, and eliminating the deadwood early is in your best interests, because YOUR time and resources are precious too.
> It's important to remember that it's also YOU interviewing THEM.
I agree with what you said in the rest of the comment, but I'd like to go one step further with the idea of "YOU interviewing THEM". Let's add some symmetry!
Let's make the interview double the usual time, and half of it I get to ask the other side to solve LC problems that I pick at random, or design systems I made up the day before, or behavioural questions about their past experiences. According to companies, that is the only way to assess if a person is good enough for the job, so I'd like to use the same method to find out if I want to work there.
I've always thought it would be fun do say, "Ok, this sounds like a fun game. It's only fair that if you get to ask me a question I get to ask you one." I'd be willing to bet 90% of the interviewers would crumble on the first question. I've noticed that most of the interviewers not only select their questions ahead of time but also have to do research so they are sure they know the answers too.
I hate interviewing but the very few opportunities I've had to have a technical discussion back-and-forth Q&A with a candidate have been helpful. A lot of candidates don't even ask about the day-to-day expectations or stack, and I think that's unfortunate for them.
Today's interview process seems eminently gameable by grinding LeetCode and watching YouTube videos on system design. In the last 4 months of job searching I've gone from barely being able to get past the question (reading under time pressure stresses me out), to convincingly passing about half of my recent phone screens. All I've done is one LeetCode per day and interviewed a couple of times per week. I'm not fundamentally any better of a software engineer than 4 months ago. The current system is great for zero-knowledge people with the time and mettle to study, and awful for accomplished but time-poor senior engineers.
And, why all companies have decided upon almost the exact same methodology, I can't understand that either. It seems to be the same double edged sword as the Common Application for college: you get more applicants because the incremental effort required to apply to college n+1 is low, but it drives down college admit rates as well as matriculation rates: the signal to noise ratio probably goes way down.
> Today's interview process seems eminently gameable by grinding LeetCode and watching YouTube videos on system design.
That is a feature. You look at what they worked on before to see if they are senior material. Then you do leetcode and system design to see if they are smart. A smart person would most likely learn whatever tech and problems they worked on, so if they do well on leetcode you just assume they did well in their past jobs. If they can't learn leetcode then you assume they did poorly in their past jobs. If they didn't practice leetcode then you got no signal on their smartness, so no-hire due to unpredictability.
So we get these possible results and what you do with it:
- Good leetcode, great and relevant experience: Hire at high level, this guy is smart so he likely learned a lot of things during his past experiences.
- Bad leetcode, great and relevant experience: no-hire, he might be smart but we don't know so we don't know if he actually performed in his past jobs.
- Good leetcode, no/minor expereince: hire at junior level, this guy will likely learn fast
- bad leetocde, no experience: don't hire, there are no positive signals.
This works really well actually. The only bad part is that some experts needs to brush up on this when they want to find a new job, but for a well paying company that isn't a problem as millions of candidates gladly brushes up for the pay bump.
As a former director of engineering who has hired many great developers and many terrible ones, I can safely say that grinding leetcode isn't a valid signal for smart, just for conformity, ego, and time. Some of the worst developers I know could crush leetcode all day.
Yes, it's an easy signal, it makes an unpredictable and messy process feel neat and binary. However, it is a false signal, and no amount of personal comfort changes that it has about as much effect on future employee productivity as hiring by height or favorite color.
Unfortunately, software productivity is never going to be measurable. Therefore we will never ever have a reliable metric to even prove to ourselves that we are more or less productive than our coworkers, much less worth hiring. This fact should sober our prideful arrogance about hiring and introduce some humility. Our best practices are a farce with absolutely zero data to back up their validity. Ask yourselves where are the studies that prove candidates who crush leetcode perform better than those who couldn't? Where is the double blind research?
We need a heaping dose of recognition that most of our best practices are there to validate the egos of the interviewers and give a neat binary decision without any mess. We have a toxic fraternity hazing ritual to make candidates bend the knee, with exactly zero evidence it works.
We have the gall to call ourselves scientists, and we don't even measure a single thing.
Want to add agreement that grinding leetcode is a terrible metric. The ideal candidate probably already has a full time job and if you add in a family and other responsibilities, they don’t have time to focus on leetcode. Heck, they probably are tired of thinking at work all day and don’t have the mental capacity to task switch into leetcode after work (I know I don’t!)
> Unfortunately, software productivity is never going to be measurable. Therefore we will never ever have a reliable metric to even prove to ourselves that we are more or less productive than our coworkers, much less worth hiring. This fact should sober our prideful arrogance about hiring and introduce some humility. Our best practices are a farce with absolutely zero data to back up their validity. Ask yourselves where are the studies that prove candidates who crush leetcode perform better than those who couldn't? Where is the double blind research?
I think there is something similar to art in GOOD software development, as somebody who does both. The primary reason I don't like coding for a living is the same reason I don't like writing fiction for a living: I really dislike being creative on demand and on a schedule. I find it stressful. I can't make myself get a story idea any more than I can make myself find an elegant solution to a coding issue. Sitting and staring at the screen more isn't going to solve the issue for me.
Software development is to math/CS what painting is to color theory. There is science, but there's also creation. That's why it's so hard to measure and hire for. Imagine trying to judge 1000 newly graduated oil painters who had all taken the exact same classes. How would you decide who was 'best'?
> Our best practices are a farce with absolutely zero data to back up their validity. Ask yourselves where are the studies that prove candidates who crush leetcode perform better than those who couldn't? Where is the double blind research?
Google has data on this, when I worked there anyone working there could view it. Performance on their technical interviews is the strongest signal they have that someone will perform well at the job. It is stronger than every other signal. They have tried a bunch of different things, Google HR hates these technical interviews since it is hard for them to game so they work hard to remove it, but they can't since they can't come up with anything with a stronger signal.
Performance on a single technical interview isn't a strong signal, but the combined performance on 5 interviews is a strong signal. If you only run a single of these instead of many with many different people then you will hire a lot of bad people anyway. Also if you pay less than Google and similar then you will get their rejects, there is probably something wrong with the people who are good at algorithms but get rejected by FAANG, not sure why you would want to hire those.
I'm very interested in what else they tried. Also very interested to see the research, is there somewhere it is published?
I do believe that performance in a technical interview corresponds closely with overall performance, but I'm very curious how they measure job performance, considering tech job performance is mostly a subjective popularity contest. I'm not convinced that is a valuable signal either. This is what I was getting at, we can't measure productivity _of employees_ with any rigor, so how can we know if our interviewing process works. Since job performance is so subjective, it's not very surprising to me that people hired with a different process aren't as well liked. The one thing I know about people is they like people like them. People who can and will grind leetcode are a very similar group indeed, so a person who didn't do that likely sticks out like a sore thumb.
Perhaps all this is accounted for, in which case I'd love to see it.
They base it on performance reviews by the managers. The mangers aren't the person rating the candidate interviews in the first place or deciding if they get hired, so it isn't just managers confirming their own hiring ratings. It isn't perfect but that is about as much as you can do, it is really hard to create a better model for evaluating hiring processes.
I do believe you can do better than Google. But I don't believe you can do much better than Google at Google's scale with that many people if the goal is to hire people who can solve hard technical problems. In a small company you can have the founder or equivalent do the hiring, like early Google did, Larry Page reviewed every single candidate until they had thousands of employees. This is what they came up with after that. Also now instead of having Larry review everyone, they have the vice presidents review every hire in their org as an extra last step to ensure the hiring process doesn't turn bad or get gamed.
Note: If you don't pay like Google, or your goal isn't to solve hard technical problems like Google has great need for, then you probably shouldn't try to hire like Google. Also if you just run a single algorithm question that is easy to look up online, then you are not copying their process, since Google forbids such questions and require multiple questions. Also if you just run an automatic algorithm challenge online then you miss all the soft skills part that comes up when you talk to a person when solving the problem, so not their process either. Also a person who isn't at least decent at algorithms cannot administer an algorithm interview properly. They need to come up with an original problem, and they need to identify when the candidate has solved the problem even if the candidate did it in a way you didn't expect. Most engineers at Google can do that since they hired for that from the start. Most engineers at most companies can't. Also if you have managers make the hiring decision instead of an independent committee then you aren't copying their process. And if your executives are not constantly checking that the process is in line with expectations and doesn't bring in a lot of bad candidates then you aren't doing their process. Etc etc.
They do so many things for this to work, if you do all of that and wants the same thing as Google then it will work for you as well. But almost no companies wants to do all that work to get this working. They just say "Google does algorithm interviews, so go and ask the candidate an algorithm question and that is our process".
Also some caveats about this process and why you hear so much bad things about it:
- Managers at Google don't like it. They would prefer to hire the candidates they want, but as is it can be hard to even fill headcount you have been allocated if you aren't an attractive team, they would love to lower the bar to fill their seats. And the few people unattractive teams manage to hire often leave the team as soon as they can for better teams. I did that, I was first on a dying project and then I did well and joined a cool project working on cutting edge machine learning infrastructure. So this process isn't pushed by managers. It is the executives who want a well run organisation who wants this process. Which is why the executive review is mandatory, this process is first and foremost for their sake. And from heir perspective, hiring really smart people, put them in a poor project to test their ability and then get the best of those into the teams that matter is how you get great engineers to those important teams. There is no reason to lower the bar for poor maintenance projects even if those managers would love to do it.
For Google the most important thing is that their search and advertisement algorithms become as great as possible, and that their infrastructure costs gets optimized down as much as possible. They need the best engineers in the world to work on those things because that is where the money is for Google, quantity doesn't matter there, the rest are just hobby projects but those hobby projects can work as a way to find talent to those important projects. You can't hire experts on search and ad algorithms, because Google is the world leader in both. Google has to cultivate such talent themselves, so their process focuses on finding people who maybe could be cultivated to do those things, no matter what technical role they hire for any person could potentially pivot to the teams that generate revenue.
(Nice side effect is that every single technical role is filled by smart and ambitious people. When you talk with some guy maintaining a project you need to depend on, they are really quick to respond and answers your question or fixes your issue within minutes or hours. I never had problem with others not wanting to help or dragging things out. I'm sure it happens sometimes and maybe more in some areas than others, but I've never seen such a well run technical org anywhere else)
- Engineers at Google don't like it. It forces them to spend a ton of time interviewing, also they would prefer if they could recommend their friends to get hired, but they can't since they have to go through the process as well. The exception is the meritocracy camp, which is pretty strong at Google but not very outspoken.
- Engineers interviewing at Google of course don't like it, since it is a lot of work.
/ Sorry for writing a lot. I tend to use forums like this to structure my thoughts. This helps me better understand the pros and cons of things I know as well. Google's process is good at what it does, but not good at other things. Understanding what it is good at and what its weaknesses are is important. But now I think I have a better idea how to argue in this subject in the future. Although, once I feel I understand most things in a subject I stop arguing since I no longer learn anything. I like learning about just about every general concept, like hiring, business, programming, math, physics etc, discussions like this is what I do for fun.
I think the opposite is true, we're doing the same thing doctors and lawyers do: artificially restrict the supply, driving up demand and pay. Not top down and centralized, but individually and decentralized. There's plenty of jobs, almost every company reports a 10% shortage, but the developers won't let in anyone.
"Bad leetcode, great and relevant experience" - that's me after 20 years in the industry and at least 30% of that time working on very non-trivial systems. I usually don't do actual leetcode actually, I review algo books before interviewing. In the last few years it stopped working for me even at no-name companies. But I find it nearly impossible to force myself to leetcode, it feels humiliating and stupid.
Another thing I noticed is that they stopped asking questions about concurrent code or writing recursive-decent parsers. They seem to be strangely fixated on a narrow and not particularly practical subset of CS body of knowledge. We don't discuss OOP or its alternatives anymore. Companies don't seem to care about my actual hard-won real life experience even when it directly applies to the kind of domain/system they are building.
Don't get me wrong, I expect people to know how to perform depth-first traversal or the general idea behind quick sort. But we are long past that stage even in small companies with average pay at least here in the SFBA.
> even competitive programmers says that there's probably negative correlation between good engineer and competitive programmer
They don't. I've worked with some of the best competitive programmers in the world, they do think it is a positive and look for people with a strong competitive programming background.
However, if your interview process only selects for ability to solve algorithm problems, then people with a lot of practice solving algorithm problems will over perform relative their level of talent, so when you run a dumb regression analysis it might look like competitive programming experience is a negative signal. So if someone has competitive programming experience then you want to bring them in for an interview, but you should probably be harder on them in the interview than regular candidates to account for this effect.
>They don't. I've worked with some of the best competitive programmers in the world, they do think it is a positive and look for people with a strong competitive programming background.
It's written in Polish, but as I see that was about kinda different thing
>How were SPOJ and programming skills useful in the context of your studies?
>Virtually not at all. I will say more - these skills were an obstacle in obtaining the title of engineer.
><rant about higher edu institution>
>...
>How useful are SPOJ and specific, classic algorithms at the programmer's work?
>Wandering around the labor market, I got the impression that it was easier to become a programmer with a vague idea of algorithms and extensive experience in creating specific projects than being a master of all kinds of algorithms, but without experience in coding something bigger.
I have 15 years of experience, and seem to woo teams every time we do screening calls.
Could be how he describes the problem. Most of the time during code screens they don't want you to perfectly solve it. Talk it out. Someone who feels that perfect or bust is the only way to go will fail most of the time. Idk feels like something is missing. He's right, feedback loops suck.
Question is... Can he practice doing a phone screening with a friend? Does he know anyone who interviews? Does his current job allow him to interview or sit in some? Just being part of it can give great interview experience.
Around 10 years of experience is when I learned a ton of interview skills. Interviewing is indeed a skill in itself.
I personally hate this approach. Solving programming problems requires my full focus and vocalizing while trying to think is a challenge (partially due to my stutter). I can either focus on communicating clearly and professionally or I can stfu and code, not both.
I’m happy to solve a problem silently and talk about it after I’m done, but I won’t do a phone screen that asks me to word vomit in the middle of coding.
I can solve programming problems OR carry on a conversation. I cannot do both at the same time.
It isn't just coding. The same quirk shows up in other ways. For example, I also cannot carry on a conversation and navigate. I can drive safely while conversing, but I cannot navigate. When my daughter was a teenager she exploited this quirk for laughs: get in the car with Dad; start for destination; bring up interesting topic; wait to see what random place we end up going; profit!
I can explain my thinking after the fact. I just can't do the thinking while conversing.
I've been working as a consultant for a while now. Communication is a big part of the job. My current client is one that has come back repeatedly, and that pretty much lets me set the terms I want, and gives me broad discretion to decide how things are to be done. I presume that means they're happy with the results so far and want more of the same.
I do have to communicate effectively in that job. I collaborate with a VP to set directions, vet R&D proposals, choose implementation strategies, pick tools, guide other contributors, and so on. So I have to be able to communicate effectively.
I also write the majority of working code in the current set of projects we're working on.
What I don't have to do is write code while carrying on conversations. It's a good thing I don't have to do that, because I can't.
I can pair program, as long as my collaborator understands that when I need to really think about a problem, I have to leave the session to do it. I can rejoin once I've figured out what I'm thinking about, and I can explain what I thought of. The collaborators I'm working with now are used to it by now, and it's a normal part of our collaborative cadence.
I can hunt bugs in a pair session, similar to the way that I can drive just fine while conversing; I just can't navigate, or do involved creative problem solving. I'm not exactly sure what the difference is; just that there's a certain kind of thinking I need to do that gets blocked by conversation.
"I'm trying to think, but nothing happens."
I'm useless in tech-interview coding, so I just don't do it anymore. I've told several companies that wanted me to go through their coding interviews that they should just give me the lowest possible score on that part of the interview, and if they still wanted to talk to me about the job, I'd be happy to talk to them. Nobody has taken me up on that specific proposal, but I have gotten nice offers a couple of times after I thoroughly bombed coding interviews. Those experiences made me question how seriously those companies took their own processes.
How do you feel about collaborative problem solving? A big reason talking-it-out is popular is because it provides signal on your communication, problem solving and collaboration skills.
If you sit there for 30 minutes banging out a solution in silence, I get a bit of signal. If you sit there for 30 minutes in silence doing nothing, then you're stuck and you've failed the interview as I didn't get the opportunity to assist.
In every real world work problem that is solved collaboratively had some time before where people understood the context and the problem by themselves before. At least they know the codebase, the business goals, the colleague.
There is no situation where a boss call two random engineers, throw a completely new problem at their hands and ask them to solve it in 45 minutes.
> There is no situation where a boss call two random engineers, throw a completely new problem at their hands and ask them to solve it in 45 minutes.
I don't know what to say to this beyond, absolutely yes this situation exists. The easiest scenario is you are on call and get paged and must solve a critical issue in as few minutes as possible.
What kind of shop are you working in which requires you to solve algorithms puzzles to resolve outages?
Even if we think we see a way to solve an outage with something like this, we generally consider it inappropriate for an outage mitigation scenario, always preferring to rollback / flip feature flags / add capacity / other simple and minimally invasive interventions.
Implementing a better or more scalable or more resilient system is important, yes, but is done after the issue is mitigated... not at gunpoint.
> always preferring to rollback / flip feature flags / add capacity / other simple and minimally invasive interventions.
Sure, that's nice, but you're living in a fantasy if you expect that to always be possible.
I also didn't say anything about implementing scalable or more resilient, or even algorithm puzzles. That's not what we're discussing.
I'm talking about communicating when problem solving. If solving an outage was always a mundane rollback or simple intervention, then it should've been automated. For example, say a new deploy corrupted the database subtly. How do you recover it? Perhaps you rollback the deploy, create a new database instance from a backup, perform a cutover, lock writes, write a migration to copy records from the old instance to avoid losing data written since the backup, validate the restore, unlock writes, monitor. You probably also need to work with stakeholders to draft and send comms to customers, document the steps you took, answer questions from product folk.
This is what I am trying to get signal on in an interview. How do you work on your feet, when it matters most. If you expect to be able to sit in silence for 30 minutes after you were paged while you draft a solution in your head then you're going to have a bad time.
For a junior candidate, these expectations are reduced, but this is the bar for a senior candidates. If I wanted you to solve an algorithm puzzle I'd give you the puzzle and tell you to come back tomorrow. If I didn't think you could solve an algorithm puzzle, we wouldn't be having a conversation.
I think talking through an algorithm puzzle solution during an interview gives you no signal at all of how well an employee would communicate in the scenario you described.
I agree it isn't a real job requirement to be able to switch back and forth quickly between thinking and communicating. I really have a hard time decoding what another person is thinking when I'm solving a problem. It's like I use the same part of my brain for both things, and it has to reboot to switch between them.
To mitigate this I try to constantly verbalize things that I'm thinking anyway, like, "This isn't the OO way to do it, but I think the OO way would be a poor match for the problem," and, "In real code I would model each of these item types with a class, but to save time and hopefully get further with the problem I'm going to model them as lists." The more I talk, the less they talk, and the less often I have to reboot into human mode.
Can't speak of others, but when I'm doing interview I never "take points off" for not talking while coding - as long as they're building the answer in the correct direction, that's all I want.
I think "talking while coding" is, at least in theory, for the benefit of the candidate - if they're "thinking aloud" and about to make a mistake which may take 20 minutes to recover, the interviewer can gently nudge them in the correct direction.
It's not about talk while coding. It's about letting your interviewer know what strategy you employ.
I fully expect Sr engineers to talk about why they choose the approach they do, and then implement it. I can adjust their approach while I hear about it, and then let them code it up.
This whole being silent and focus coding is bs I think. When you're doing the typing part, don't talk, just code. When you're doing the strategy part, talk. I want to see how you'll be able to express your strategy to your coworkers when a real actually challenging problem occurs.
Also. Algo problems are worthless. They only tell me you know how to cram. Don't do those, they don't actually answer anything. If I see that in an interview that's a flag for me that their eng culture might have issues. Only exception would be if you're applying to a job where you'll be doing that all day.
I'm much more pro-leetcode-style-interviews than the median HNer, but I agree with this criticism. It's a very unnatural way of working, and really dilutes the skill signal. In principle, you are learning about how they communicate, but I believe in practice what you suggest, where the candidate has a chance to produce a solution and then explain it afterwards, would be more effective at this as well.
I talked it out recently and it quickly became apparent that the interviewer considered my list of "this is how I wouldn't do it" as a list of "these are approaches I would actually consider".
It's great when senior people admit they're no good at interviewing.
I've never passed a technical interview.
I got my internship at a company that thankfully valued informal broad technical discussion over a conventional technical interview, then from there I slid into a full-time job at the same company with no additional interview. By the time I applied for my second job I was senior enough not to be asked to do a technical screen.
I've attempted conventional tech interviews in between and absolutely crashed and burned! So I'm impressed by anyone getting through these interviews, because I can't do it!
I'm on your team. up front technical "gotcha" style evaluations feel like they don't respect where I'm coming from, nor my track record of work - and are a cue that the job is not the place I want to be at.
I've gotten nearly 100% of the jobs I've gotten in the past 15 years through friends. Interview process real easy after the informal "backchannel" reference check has happened. Just hi, how you doing, when can you start.
Side effect of this is probably making much less than your counterparts. If you can ace a leetcode style interview, you are much more likely to have competing offers (or retention offers later on) that allow you to make MUCH more. It’s a horrible system but just something I noticed before I got really good at them.
I've been starting to refuse 4 hour technicals myself and they all immediately drop me. I don't have your seniority but I have some. As well as a powerful online presence across Github and LinkedIn. At what level of seniority did you start doing this and what is your success rate?
There's not really a seniority level where this happens.
The outrage around the technical interviews shows a lack of understanding for the situation companies face when hiring.
The reality is that something closer to 99% of the pool are applying for jobs they're not qualified for (not the same as saying 99% of developers aren't qualified - there's extreme survivorship bias in the pool).
At company X we would hire ~1/400 applicants who applied, and about ~1/20 referrals. At a smaller company I worked for we hired perhaps 1/200.
The thing is, many of these applicants have five to ten years experience (some at FAANG), but can't code at all. It's simply not possible to trust anything on a CV without verifying it. How do you verify this at scale? A pair programming exercise.
Now the training of most interviewers is often poor, but it's far better to go down this route than the HN alternative: hiring without technical screening.
they drop me too, but those companies are not worth working at to begin with
they’re not interested in hiring employees, their interest is to get candidates through the loop and to see you fail
this is fine when you’re applying to FAANG, but a terminal sickness when the company is barely known
rather than spending weeks on interviewing, i’d suggest making a open-source contribution or reading a book
the companies, especially startups have a bigger problem than you - they need to show the growth so they can attract more investors. Therefor, if they’re not getting people, they have harder time growing. If they see you as a potential candidate, but you refuse, they’ll bend their back over backwards
Interviewing is really tough. The OP replaced some coding questions with other questions and if he were to interview a bunch of people with them, we'd get another set of blog posts complaining how ridiculous his interview questions were.
"Why is it bad to write a file to the disk one byte at a time?"
What are they interviewing for, low level disk experts?
"Here's a stack trace from an open source project, can you tell me what went wrong?"
Oh sure, print-based debugging is awful, you just have to read stack traces from unfamiliar projects in the blink of an eye.
"Here's a CRUD app, copy existing code patterns to implement a new CRUD endpoint"
Ah yes, the reductive "everything is a CRUD app" - this interviewer doesn't understand that most applications don't fall under the simplistic "CRUD" moniker!!!!
Seriously. I feel like complaining about hiring for programmers is the #1 topic discussed on HN.
I actually feel like algorithm questions are somewhat of an improvement in the programmer hiring process over the alternatives. When interviewing at places that don't have them, the surface area of questions you can be asked is so much bigger.
Like one of the anecdotes in this thread about getting drilled on a testing framework (that the candidate probably added to their resume because they have to have all these keywords to make it through their automated resume scanner and maybe they only played with it for a couple hours one afternoon because they don't use it in their day job.)
With algorithmic questions:
- You're doing one type of preparation for a large amount of potential roles
- There's tons and tons and tons of great material out there to help you prepare
- Probably most controversial but I really believe it: It really does make you a stronger programmer.
For me, doing those kind of questions forced me to learn data structures more thoroughly and dramatically improved my understanding of how to write algorithms and think about performance. This stuff really does matter in terms of code quality and it really bothers me that it's always portrayed as these completely silly puzzles.
The other thing that I see pushed a lot in these discussions is this idea that every company (or most companies) that hires programmers does these exercises. That's just not true. If you're getting coderpad links for every phone screen, you're going after a certain type of company.
What are they interviewing for, low level disk experts?
While I agree that "disk" stuff is "low level" compared to a web developer, it's definitely a very relevant question. You might ask it and if the candidate is interviewing for a web developer position rephrase it after getting a blank stare: Why is it bad retrieving 50 users from a REST endpoint like `/api/users/{id}`, one at a time?
Oh sure, print-based debugging is awful, you just have to read stack traces from unfamiliar projects in the blink of an eye.
That's actually pretty relevant. If you want to be nice, make it something that uses the same stack as you're hiring for but even if not, it can tell you something about the candidate. Some candidates might react like you did in this reply. Not hired. Some candidates might go "oh well, that looks like python, which I'm not familiar with but let's see, ..." and then they try to figure something out from the function call naming and error messages shown. In any large enough company you will definitely be looking at unfamiliar stack traces, potentially from a service you are unfamiliar with and potentially a different stack than you're used to. Happens all the time at my place for example. I can't just throw up my arms and ignore it. Gotta dig in and use generic problem solving skills.
Ah yes, the reductive "everything is a CRUD app" - this interviewer doesn't understand that most applications don't fall under the simplistic "CRUD" moniker!!!!
Or maybe it's a simple test of "OK, so all of the other questions the candidate sort of failed on. Can he at least do 'stackoverflow copy and paste programming'?" Whether it's a CRUD endpoint or something else entirely doesn't matter. The point is: can this candidate work w/ something potentially unfamiliar and still get something done? Happens all the time in any large enough code base. Happened to me just this week where I had to implement something I needed in a old python service we have. I don't like python and I don't know any of the python frameworks used there. I copy and pasted, adjusted and google some basic python syntax and got it done in like half an hour. That's what they're testing for there. Sounds perfectly reasonable to me.
It’s interesting; I assume you’re using “mental gymnastics” in a derogatory sense but I love those questions and I would love answering them. I’m actually a bit peeved that no one is asking me them right now.
What are they interviewing for, low level disk experts?
Writing one byte at a time may be “bad” at many levels. First, on a hardware level it is inefficient because the unit is a sector/block and the access latency kicks in hard, together with a wear leveling on ssds. Embedded low level indeed. On an os/syscall level it is inefficient because of a syscall time itself. It is a system level touching C and unbuffered writes in “middle-level” languages/libs. On an application level it may be inefficient if a stream has a processing pipeline with some expensive setup/try/teardown (e.g. a jsonl parser which “tries” the buffer at every new chunk). Is this still low level?
This is a question about streaming, buffering and latency avoidance, which you may not know in hw detail, but have to have at least a vague grasp of. If a candidate seems confused, one may ask them about other situations where one-at-a-time may become “bad”: sql select one row at a time, fetch NNN items one by one, use heavy_expr().x, heavy_expr().y repeatedly or in a loop. If nothing comes to their mind, chances are you are interviewing a completely inexperienced person, because there is no way they didn’t meet that or think of that before.
I totally agree that it shouldn’t be the reason to skip a candidate, but hiring is an analyzing process, and how do you find their limit if not asking deep open questions? I think that the issue is pushing hard on these areas as if it was laughable for a professional to not know it.
I'm not sure if you are being sarcastic or not, but I think those are some pretty excellent interview questions.
The first one, it's exactly about the thing I find a lot of developers lack in: a fundamental understanding about computers and processing. I've had 'experienced' developers write algorithms where every intermediate value was written to a cache/disk only to pull it out on the next line. I feel asking some more fundamental questions can help weed ppl out.
Same with being able to look at a stack trace and see where it went wrong, how quickly are you able to hone in on a specific target. Crucial skill imo, and I've had sooooo many people at my desk -- 'it's not working' -- that did not even read the f'ing error message.
I was being sarcastic with the question responses - the point being, for any question you ask in an interview, there is a significant population of candidates that will think it is a terrible question.
That’s not their point. Point is, it’s really subjective and there is no standard answer. As much hate as leetcode style interviews get, you at least have a standard interview process as that point but yeah don’t overdo it and make sure the loop is varied in the areas it covers.
I am being misunderstood, let me elaborate, point is, there is no standard answer to the question “what to ask in an interview”. They are just giving an example of how somebody will always have problem with the questions being asked and they could be right in their perspective which is just different.
> Triplebyte has a quiz that actually does a pretty good job of this; you get asked thirty or so questions about databases, networking, and simple coding questions ("what is the value of 'a' when you get to line 17?").
I could absolutely see people complaining about multiple choice questions and how you're more likely to get tripped up by minor mistakes.
It feels like the other way round to me, can you clarify why you think that a multiple choice questionnaire is more subjective than an algorithmic interview?
But this is also common in algorithmic interviews, correct? Like if I have a sheet that says bubble sort is the answer, and you provide merge sort then you are marked wrong.
The interesting thing about mcq is that their administration is completely standardised so it's more objective in pretty much every sense.
Maybe you prefer algorithmic interviews, that's fine. But I really don't understand how you can believe that they are less subjective than algorithmic interviews.
They avoid the bias of using different interviewers which is massive, even if the questions are standard.
I have sympathy because I program in a similar way as the author. Not so much with tests but with some kind of fast feedback loop where I do lots of guess-and-check. I don't know how I'd do in this kind of interview.
However...
> If I get behind or the interviewer starts interrupting to ask about the bad code I'm writing I get very stressed and have trouble both listening to the interviewer and trying to address the issues with the code.
My sense is that 90%+ of these interviews is maintaining your composure, asking questions and generally coming off as well-adjusted and personable even if you don't know the answer or get stuck.
> Certainly there are a lot of confident, mediocre tall white men in the tech industry
The author is focused with technical review, and as such he probably fails to see just how important personal review is. Yes, you are being reviewed technically, but you are also being reviewed personally. You may know a lot of technical things, but do I want you on my team? The phone interview is what i'm going to use to make that decision. Technical we can work over later, but only if you sound like someone I want on my team.
I'm always willing to say, "They look like a great fit for the team and our goals, and let's give them a chance to learn the tech" versus, "They know a lot of technical stuff and let's give them a chance to see if they can learn to work with other people."
> "They look like a great fit for the team and our goals, and let's give them a chance to learn the tech"
I wish more were like you.
I've definitely had my fair share of interviews that bombed so I feel like I have a good bearing of when I don't leave a good impression, but it's always the worst feeling to have an interview that you think went well based on the overall mood/feeling, only to receive the eventual rejection later in the day that they went with someone more senior or that they wanted someone with more experience in x.
My mentor who didn't got to a well-known college once told me he joined a very solid company not knowing anything about the frontend framework they used but they gave him documentation and let him learn it for weeks without touching the code base. He said it was honestly one of the strongest engineering teams he's worked on in his career. The idea of managers seeing potential in someone and thinking "let's go with that person who has never done x" seems so foreign to me.
Re: your last paragraph, I feel like that's symptomatic of a change in the relationship between capital and labor that has happened to some extent at every level of the market.
Companies are increasingly unwilling to invest in the wellbeing or value of individual workers, instead treating workers as fungible assets to minimize the cost of which serve as solutions to current (rather than long-term) problems. We see this with the near-universal embracement of short-staffing in service-level jobs, lack of career mobility from starting positions, and even in the tech industry with the expectation that you usually have to switch companies to get a raise.
I worked as a sole proprietor/consultant for several years, which involved lots of “personal review” during the sales process. I am well aware of how important that is.
But the author is specifically complaining about the technical phone screens using vastly inferior tools and processes than what you work with day-to-day. Being asked to write code on demand in a notepad with no way to test or compile is not a personal review, it is a monkey show.
I have 25 years of experience and I spent a year looking for a job, getting rejected after complex, time-wasting tests and endless voice interviews that went nowhere. At one point I had to take a cognitive test which I actually failed.
I eventually found something but it was really discouraging there for a long time.
23 years here and at this point I consider interviews to be 1d100 random dice rolls. No matter how much you prepare there is no way you can predict how sensible the interviewers are. I figure just deal with it and interview more. If each interview is 1% independent chance of getting an offer, and you interview 50 times, you have a 40% chance of getting an offer.
You're right, it seems totally random. For one they had the Scrum Masters interview me and they felt I didn't have enough scrum experience. For another I had to try and compile a video encoder on a three generations out of date VM and it took all day and I got $100 for my time. Madness.
Companies are really bad at interviewing. They toss the task to programmers and engineers who have never been trained in it, and expect them to not suck.
The best way, as always, is to connect with people you already know and have them short-circuit the broken process via existing trust.
This feels encouraging to read. Thanks for posting it. I only have 5 years of experience, but I've gone through something similar and it's extremely disheartening to see yourself failing these convoluted, labyrinthine tests knowing full well that you'd be able to do the job you're interviewing for pretty well given half a chance.
Everything he said is spot on.
I suspect algorithm interviews are optimized for FAANGS - e.g if you can do the whiteboard and coderpad under pressure and interruptions you'd statistically do well in FAANG. I suppose that's true otherwise they wouldn't be doing it.
But this hiring process is extremely broken for most everyone else; FAANGs don't really care about false negatives; they have so many great people wanting to work for them its ok for them not to hire some great people.
But this whole process seems problematic to me for smaller companies who are in great need of good people and aren't drowning in great resumes. You can't really afford to miss 50% of great people if you're a startup, and this process will do just that; it will let you hire good people but also not hire good people on a really bad margin.
Companies who are not FAANGs, especially startups, are better off coming up with some hiring process like the author described; it's not perfect but it will have less false negatives.
About 11-years ago, I was in a bad shape after a disastrous Startup failure. I was looking for a job and out of many prospects, I had an interview with bookings.com. It started with a phone interview and they wanted to go technical right away; so started asking me the basics of CSS.
I was taken off-guard because I don’t remember what is the default value of float, or the list of default values for many CSS objects.
I explained that I don’t remember those as they usually pop up in the IDE when I start typing. However, if they want, I can show my open-source bare-bone drop-in grid framework I wrote sometime back or I can walk them through CSS Box-Model or even explain how I can help bridge the gap between designers and engineers.
Unfortunately, it ended as their time ran out. I got a rejected reply because I could not pass the basis of CSS on a phone call. That was the incident I promise myself that any candidates I interview in future will always get a chance to explain/prove their worth in the ways they know best.
I was in agreement with most of the article, but the final footnote came out of left field for me:
7. I have a bunch of ideas around testing how quickly and how accurately people can type, or accurately transcribe text from a second screen, that I think could be decent predictors of job performance but I'm not sure are fair to test out in a professional context without academic research that could validate them at least a little bit.
Can anyone explain how including such a thing in an interview is relevant at all?
This would be highly effective at filtering out people with physical disabilities that affect typing. For example, I have chronic issues with my joints. On my personal computer I use an ergonomic keyboard (Advantage2) which has keys in weird positions. My typing speed on this keyboard is much lower than it was earlier when I used a normal keyboard. Nowadays when I use a normal keyboard, my typing speed is also slow because I am used to typing on the ergonomic keyboard, not on a normal keyboard.
Maybe the accuracy of the final work product would be a fine filter for attention to detail, but as a sibling comment points out, “quick and accurate” transcription is likely to just discriminate against otherwise fine candidates.
I assume the author thinks that would be a good proxy for some hard-to-measure quality, and is not actually aiming to hire the best typists because of their typing skills.
As for what that quality is, I don't see any hints there. It could be anything from general tech-mindedness to even basic diligence or who knows what.
Your code on github is pretty slick. I love the documentation.
Leetcode is an exam that you can study for. For me, it's pretty damned hard and I dislike it.
I can think of three reasons why someone might be good at those kinds of problems:
• You just have a knack for it.
• You are very intelligent.
• You practiced a lot.
Which one of those can you do something about?
Every time I force myself to grok a new leetcode trick, I've truly learned something. Something irrelevant in my ten years of programming, but allegedly an indicator of someone who might be good at what we do.
> ” I'm not fast at reasoning about code, and I often make trivial mistakes just trying to get a "first draft" of a program out. If I get behind or the interviewer starts interrupting to ask about the bad code I'm writing I get very stressed and have trouble both listening to the interviewer and trying to address the issues with the code.”
That’s exactly what happens to me. The thing is, I am not as fast or experienced as the author. I don’t do well on those Triplebyte quizzes because they put data structures and algorithm question among even in the frontend quiz. And also ask trick questions that I usually run the code to see what happens, not spend 3 minutes imagining what the outcome would be. I much prefer take-home tasks even if I take 4 to 5 hours when instructions said I should take two.
So, I am just terrible at technical assessments in general. And I don’t have that solid CS knowledge that some positions require or people think is the minimum necessary for me to be called “engineer. I don’t care. I am happy being called a “developer”.
The challenge is: how do I find a job post that would hire for what I am good at?
I am good at shipping things. I am good at communicating with non-engineers in general. I am good at scoping, understanding the problem that people want to solve with code and proposing the fastest way to get there. I am good at creating simple code that it’s easy to understand and do what’s needed to be done with good performance. I am good at writing tests that make sense. I am good at focusing and organizing my time to deliver working code in the agreed deadline. I am good of delivering good work with autonomy. I am good at working as a team. I am quick to learn new tech. I am quick to understand legacy code and to improve it.
Despite my poor technical interview skills, I was never fired for poor technical skills. On the contrary, I was always complimented for what I ship, how quickly I learn and contribute when dealing with new tech/codebase, and how I quickly improve upon feedback.
So, how do I find a job whose hiring process value those, not the former?
I am considering not even trying if the process has a mandatory live-code step or leet code.
It’s hard to find those places because they’re usually small companies outside the valley without name recognition.
I work for one of those places and am the lead technical interviewer (no “a” lead, “the” lead, that’s how small we are).
We do have a very easy take home code challenge that’s very related to the kind of work we do. But we only specify the language to use because that’s what 99% of our code is written in. We don’t care anything else about structure of the project and you do it on your own time (it’s really only an hour or two of work - or at least it should be)
I’m not sure of the rate of people returning the challenge to us (I hear that it’s a bit less than 50%). I interview all of the folks who return code. And pass about 2/3 of those on to the full day interview. (No-pass is usually code that doesn’t compile or gets the wrong answer on the test case that we provide - we give them the correct answer for the test case, so they know what they’re aiming for). And we hire a pretty high percentage of the folks who do the full day. Granted, we’re in a bit of a niche field and we don’t have big-tech name recognition so we don’t have the volume of applicants that other places have to deal with.
I always try to be respectful in the interview and we only talk big picture stuff. There are no quizzes or gotcha questions, we talk about the code and just have a conversation. I hope that everyone who leaves my interview feels respected and like their time wasn’t wasted. Sometimes I include one of my junior engineers on the technical interview so they get experience with it. Usually they start off peppering questions and trying to be smart, but I just set a good example and they calm down quickly and realize it’s about a conversation and seeing if the candidate would like the kind of work we do.
I can’t think of any bad hire we’ve make in my years of doing it this way.
Good candidates do not apply if leetcode is involved; good candidates know they can code and they want to have interviews where they are asked, from a high level perspective, about past projects and the like. On the other hand, bad candidates know they cannot code nor talk about past projects in a decent way, but it’s easier for them to cheat by submitting copied pasted code than to talk about software.
By using leetcode in your interviews you are giving bad candidates an opportunity to apply for a job in your company.
Yes, but bad candidates are going to apply anyway. How many depends on the size and visibility of your company. I suspect that for FAANG, leetcode is a much better filter than random (or going by shoe size). And there must be a filter. Also, think about the cost of the time employees (developers) spend interviewing -- let's come up with something quick, standardizable, and programming-related. Leetcode.
I truly despise the leetcode style of interviewing. It comes off as completely lazy and ineffective for most companies, and is a terrible indicator of how talented individuals are. I've been in the field for about a decade and have been on both sides of the interview table more than I can remember. I'm at the point where if I get asked these kinds of pointless questions at a screening phase, it is a signal I probably don't even care to work for them. If they don't care enough to actually create a meaningful interview process rather than copy+pasting the dumb Silicon Valley company pointless leetcode interviewing template, then they're clearly not somewhere I want to work.
The worst experience I had was applying for a couple of large companies like Peloton and Compass. They outsourced the leetcode interviewing process to some arbitrary company in Russia. Not only did I not have the chance to ask any questions of the company, but I had to do absolutely pointless trick questions for an arbitrary third party. It was an absolute waste of time that I only did once and told the other company basically to fuck off. Seriously, if a company can't even run their own interviews what is the freaking point?? It's an absolute absurdity and I cannot believe that this practice is so common.
At last company we used an online tool to screen junior/intermediates and the whole point of that tool was "Here's the task, here's the tests that need to pass when you're finished, feel free to write more", and someone writing more unit tests to exercise their code was a big positive.
Asking someone to implement stuff without running it sounds counterproductive to me. About as effective as asking them to write syntactically valid code on a whiteboard.
Companies are missing out on tons of great, talented individuals who are frightened to endure Leeetcode-like interviews. Are there any firms that give candidates choices on how to demonstrate they’re a good fit for open positions? Maybe you do like grinding Leetcode algorithm questions, maybe you want to create a new service that satisfies requirements and discuss the design choices, perhaps you’re really good at contributing to existing projects.
I read about three paragraphs in and then stopped.
Coderpad tests where you're encouraged not to write tests, can't hit the run button, and can't use reference material? Worthless. Most people - not just the author - are going to fail those, and they are not representative of the work environment, so what's the point?
We use Coderpad for one of the 3 interviews we give candidates. We do a tech screen first, then the coderpad, then a panel interview; the author is right in that it's about a 5-6 hour process overall (for the candidate). The coderpad is as much about watching how you go about solving a problem as it is you getting the right answer, although that certainly counts. We want to see if you pay attention to the question and implement what we asked for (not brain teaser 'gotcha!' stuff, simple stuff like "write a function that does ... and returns a boolean" - a lot of people will have their function print an answer and return nothing, or won't write a function at all, or will ignore other key requirements), that you can explain your approach, that you can think of some test cases, and that you can accept help/coaching if you're clearly stuck. We want people to do it in a language they are comfortable with, and looking up syntax is OK.
Overall I'm not sure how much value we get out of coderpad. It's manufactured pressure, just like a whiteboard exercise in an in-person interview. You can tell candidates what I wrote above, and try to convince them that it's not about getting the right answer as much as how you get at the answer, but that doesn't really work. Some people are not going to do well in those interviews regardless of their ability.
> Overall I'm not sure how much value we get out of coderpad
Well so you're pretty much in agreement with the author it seems. He's just more convinvced than you that coderpad is actually more bad than good that's all, you have a bit more doubts.
Maybe. I know we've hired people who did objectively poorly on our coderpad and they've been effective - although in one case not as an IC, IMO - while other people who did objectively great were just not.
I think my feedback is that the coderpad is not about whether you can solve tricky brain teasers correctly. It's just one signal along with several others: can you understand directions, can you demonstrate familiarity with a coding environment, and does it seem like you can think through (again: simple) problems in a way that we can compare to more real-world stuff.
Personally, my favorite challenge stuff when I was last interviewing were take-home problems that were more representative of the type of work the position would be doing. I could take my time, and idk I felt like I was doing something meaningful and holistic instead of implementing an algorithm. I've brought this up enough to realize that it's a controversial idea though; a lot of people either want that scheduled hour coderpad because they are taking their personal time for it and that's what fits in their schedule, or I've also heard people say that if they are going to do work then they expect to get paid for it, so a take-home assignment doesn't work for them. Of course there's also the possibility that someone will just copy an answer from somewhere and submit it.
I conceptually like the idea of dropping coding screens, etc - but how do you objectively identify what you're looking for in a candidate without adding bias to the process?
I.e., at least with a coding question, you can set up objective criteria. If you ask someone about their work, it might be a good experience but you're also setting yourself up to bias towards people who talk/think like you.
If you really want to test someone's coding, let them do some coding in their own time with their own tools and send it to you. This isn't ideal, since it can be time consuming, but I think it's more humane than live coding in some weird environment.
I would argue that using less of someone's time is strictly "more humane" than using more of someone's time, because an individual's reaction to a situation is... individual
There's no reason an asynchronous coding assignment needs to be more time-intensive than a synchronous coding interview - that's the convention but it's not a necessary one. Lots of companies do 4 hours of technical interviews, it's not hard to come up with an effective programming assignment that takes less time than that.
I think that staying employed as a software developer for 10+ years is a good indication of skill, far better than any random coding assignment.
Put another way, if someone has 10+ years of experience, in an ideal world there isn't much point in quizzing them on algorithms or giving them homework. They've done all of that before, passed, were promoted, etc. You can see that on their resume, and you can verify that with a few phonecalls.
Of course, in reality, you have to quiz everyone equally, and hire based on this objective metric, because even a whiff of non-objective hiring is dangerous.
> I think that staying employed as a software developer for 10+ years is a good indication of skill, far better than any random coding assignment.
As someone who has been doing interviews for as long as 2 years, wrong. I've seen at least 15 candidates that had 5-15 years of experience and we're complete shit. As in no redeemable qualities in terms of tech assignment.
> I often make trivial mistakes just trying to get a "first draft" of a program out
In my experience, it's helpful to add simple tests as you go. I have done some live coding tests in Python recently, which I'm not super familiar with, and the first couple I did, I made a lot of trivial errors that caused my solution to fail until I went back and cleaned them up. Luckily the interviewers kept their mouths shut while I was coding, and I was able to clean up quickly afterwards, but I could tell it wasn't a great look to say, "Okay, I _think_ this solution is correct," only to have it flame out spectacularly.
After those first two, I started writing simple tests for everything I did. Not only did it avoid embarrassing fails when the solution was supposedly complete, not only was it quicker overall, but every interviewer I've done this with has commented approvingly about it.
I address this in the post and part of the problem is that this is more difficult on Coderpad, and also interviewers (n=~8 out of 10 or so) discourage writing tests.
yeah and if you are an anxious person like me, (i may be on the spectrum, i have some social issues), writing code in front of people makes me very anxious, it takes me back to my childhood when my dad would scream at me for being dyslexic, i got my b’s and d’s mixed up.
i know that is not what is happening during an interview but, it’s how i feel. and this prevents me from getting ahead in an industry i am madly passionate about
I'm really sorry to hear about those traumas and your current difficulty. Shoot me an email, maybe I can help: adamzerner@protonmail.com. https://adamzerner.bearblog.dev/ if you want to internet-stalk me first.
I think it is fortunate that great engineers like OP are rejected in Leetcode style interviews. The benefit is twofold:
- It makes fresh graduate noobs (or even just noobs in general) able to compete on equal ground on high-paying job prospects. Think about it, if every high paying jobs out there value experience over leetcode style of interviews, then young noob programmers won't be able to rake great salary because all of those jobs are filled already by experienced engineers
- Great engineers should just use their talent somewhere else in startups, create more innovation, create more wealth rather than helping the big tech giants become bigger
Not so great if you are jobless and got bills to pay. “Create more innovation” is a good thing, but sometimes people need to cash those big tech giant’s checks to care for their family.
Experienced engineers and college noobs should not be competing for the same jobs.
Tech co’s seem to prefer the college grads for rank/file for many reasons. Maybe because they are easier to exploit?
Quite honestly, if the job description says "we are looking for the best of the best" or "we are looking for a talented java developer" I know this job isn't for me.
Same with algorithm questions, if the interviewer asks me an algorithm question, I will switch off and politely put a cross that this role isn't for me. I don't have cs background and my CV is clear that I've not done anything like that.
Edit: I am not a great coder by any means. I just don't like to waste my time when it's clear I am not a fit for the company or the role.
You don't need to have someone write code to find out if they know how to code, or how well. Simply ask a series of increasingly deeper questions about a range of coding topics that elucidate both technical expertise and experience. If you don't know all of that yourself, find someone else that does and have them do the interview. I've always been able to conduct interviews and find out someone's strengths and weaknesses without seeing them write code or fix a broken thing. Maybe I'm just some magical unicorn and this is a super rare skill? But I doubt it. I'm just looking for the tell-tale signs of someone who has done real work and learned things in that process.
The other thing is, as someone hiring, my main concern is just making sure the candidate has the specific skillset or experience I need. The person may learn quickly, but I probably don't have time for them to learn the one technology I need them to use ASAP. A lot of candidates end up getting rejected because they put something on their resume that we really need but they don't know well enough. And some just seem to be trying to reach and do a job they don't have experience in. It's hard to find someone who has just done the job I need done and isn't trying to get some other job.
> In theory a phone screen is supposed to evaluate whether a) this person would be good at the job being hired for and b) whether it's worth investing another five hours in trying to hire this person.
They also screen whether or not you communicate well, your motivations for looking for work and whether that matches the role and future growth of the role, whether there are any logistical or compensation-based blockers to the role...
Frankly, getting all the deal-killers out of the way is the point of the phone screen. Figuring out whether you can actually do the job, and do it well, is for later phases of the interviews.
I feel like the hiring process is often misunderstood. The first screen is one of two things - if you are working with a company directly, it is just a screening for deal-killers. If you are working with an outside recruiter, it is them trying to understand you to know how to market you to the hiring manager.
In either situation, though, if they are getting into remote coding of algorithms on an initial phone screen, something is really off in their hiring process.
So whether you would be good at the job being hired for, then.
The whole point of the post is that the phone screen isn't just screening for deal killers. If it was you wouldn't flunk people for not nailing the optimal Big O runtime for your custom algorithm.
It's hard to get meaningful answers to these really general properties you're after:
> communicate well
This has so many elements and is really only fully visible after you work with someone for a while. Sure you'll catch some red flag issues, but such issues likely to be caught are also likely to be caught no matter what you're talking about, so this can't be the primary purpose of the screen.
> motivations for looking for work
Is not wanting to trade services in exchange for money in a valued environment enough? But sure, aspects of this for most people can be an important subgoal, it's just important to set expectations on how easy it is to game and how unreliable the signals will be. I've sometimes found it useful to probe seriousness and a sense of longevity commitment (if that matters, and can be incentivized without needing to probe), and some of their values, and whether their idea of your actual environment matches your reality and how much that matters (the candidate should be actively participating in this with their own questions though), but again reliable signals are hard. Once I helped screen a manager who was thinking of coming back (he had previously left a couple years prior) to the company to manage a new project, and got out of him when asking what might stop him from leaving again in a year the vague (if true and agreeable) "you never know what opportunities life will bring" and concluded he wasn't that committed for the role I felt needed someone willing to put at least a few years at it. It surprised another guy who talked to him, but he hadn't probed into that.
> logistical
Seems this could be handled before any conversation takes place by clear communication about whether the role is remote or not, how much office time is required, whether relocation support is offered, and so on.
> compensation
Also can be handled before any conversation by having salary ranges for the role. If they're too low, a good candidate might go through with things anyway and try to negotiate beyond the range.
> getting all the deal-killers out of the way is the point of the phone screen
Ultimately I agree with you on this, but I think the biggest deal-killer for a technical role is: the candidate cannot actually program at a level you require. I'm a fan of Yegge's process (https://sites.google.com/site/steveyegge2/five-essential-pho...) and warnings and have used my own variant for what we cared about more, but frankly if your screening doesn't require some level of coding at least on the level of fizzbuzz or easier (but adjust to requirements/expectations), sooner or later you're going to bring someone in for a full round and discover they can't program, wasting possibly considerable time (more than five hours when adding it up for everyone involved) as well as the opportunity cost.
I think you and I probably agree on much of, this so take all this as discussion, not argument... that being said --
Wanting to trade services for money is not enough. Everyone wants that. But good candidates want more nuance to it. More autonomy, more meaningful work, more leadership. Or less of all those things. If I just need a code monkey who will crank out the tasks, I don't want a visionary tech lead. I may want a code monkey who aspires to be a visionary tech lead as we grow. And vice-versa. If I need someone to come in, lead, and build a team, I don't want a code monkey who is looking to just coast doing easy corporate CRUD apps. Software dev roles have a massive variety in their levels of autonomy, leadership, and growth. So do the candidates. The phone screen may miss many things, but you can at least match up on those factors.
> If they're too low, a good candidate might go through with things anyway and try to negotiate beyond the range.
This struck me as an interesting take. I tend to envision good candidates as the one who command enough market value to simply filter low salary ranges out. Why bother negotiating past their range, instead of finding a role that is in my range. There is no lack of job opps out there right now. I've even done that just in the last few months - found jobs that sounded ideal, but declined to continue once I realized the salary was too low. I did so politely, and invited them to call me if they ever have roles at a higher salary.
I also agree that more could be done in emails and phone calls, making the phone screen less critical and better for all involved. But I believe the key to making that world is to fully lose the concept of a "full round" of interviews. Especially now that everyone has remote work experience, lets make the process more fluid - more calls and emails, less massive half-day interview marathons.
Right, I think we're probably in agreement on a lot too. I do think if you're looking for a specific thing (whether it's a code monkey, or a visionary, or someone that seems to have the potential to grow into someone amazing, or a Postgres expert who can help you solve your particular scaling issues right now and then go away) you should try and purpose your interviews for figuring out whether those things are present to sufficient degree, and ignore/deprioritize getting data on other things. Though I also think even with the large variety of roles that exist, they all need an ability to program and solve the types of problems that the business is working on and is likely to encounter, so this should be tested/verified as early as possible, and that a lot of people are fooling themselves in thinking they need much more than that or that the other things they claim to care about actually matter.
Consulting gigs (like the postgres expert) avoid a lot of the crazy interview nonsense, and a large part of that I think is that wanting to trade services for money is more than enough, but sure their limited tenure is another big factor and helps them from needing to buy into the company culture or whatever. But this precludes that the business knows what services they want to receive in exchange for money. And repeating myself, often businesses aren't even sure, or don't actually need much more beyond "can program competently for our use cases" (which doesn't mean code monkey). So much of the extra stuff that retrospectively becomes valuable and maybe even worth trying to find in the interview process is nevertheless so hard to find. Some companies claim to have found ways, e.g. here's tptacek's hiring post: https://sockpuppet.org/blog/2015/03/06/the-hiring-post/ Quoting the relevant part:
> A few years ago, Matasano couldn’t have hired Alex, because we relied on interviews and resumes to hire. Then we made some changes, and became a machine that spotted and recruited people like Alex: line of business .NET developers at insurance companies who pulled Rails core CVEs out of their first hour looking at the code. Sysadmins who hardware-reversed assembly firmware for phone chipsets. Epiphany: the talent is out there, but you can’t find it on a resume.
You also won't typically find those really bonus qualities on a phone screen. The approach of work-sample tests goes in a rather different direction than what we've been discussing, though it can also get rid of full-round interviews.
On the salary bit, I did say might, because you're right that if the compensation ranges are known upfront, and the candidate finds them too low, then it's typical they will filter themselves out before anything. So if that range is known before the phone screen, you don't really need to bring it up then. For those who don't filter themselves out, then they either don't have a compensation issue, again making it pointless to include in the phone screen time, or they do have an issue to some extent but have reason to try and go through the process anyway. The first reason I listed was because they think they can negotiate high enough beyond the range once they've secured a "we want you" spot, and they might try to further amplify this by mentioning a competing offer(s) from somewhere else. Obviously the greater the gap the less likely they are to actually entertain this, however they still might if they just plan to use you as the 'competing offer' to gain leverage elsewhere. I don't think this sort of thing can be reliably detected at phone screen time, so it's pointless to ask about it, but sure, it doesn't take much time to ask a probe like "are you interviewing elsewhere now?" which might help you be ready for some negotiation games later. But I think you should always be ready for those regardless.
For more depth into one mentality that might do this: there are all sorts of desires someone can have that aren't necessarily abundant in the broader opportunities out there (like specific niche technologies, or specific people you get to call coworkers (there's only one [insert programming idol]), or the product itself (there's only one [insert video game franchise]), or having a preference for non-remote and this being the main/only good opportunity in an unusual place you want to live). So if asked at some point before an offer is extended "is this salary range acceptable?", you might honestly answer "instant yes if it can reach higher by this much, I'll reallllly have to think about it if all I can get is the top of the range, but at the middle or low end it'd definitely be no even though I really want this rare property I can't easily find elsewhere." Such an honest answer is unwise strategically to give, though, precisely because the other side may consider such uncertainty as an immediate deal-breaker, and in the end after learning even more through the process your revealed preferences might lead you to saying yes even at the lower end of the range.
> Coderpad must work for some people since companies are able to hire a nonzero number of engineers, but it is definitely not a good fit for me.
Seems everywhere I've applied recently has been using Coderpad.
Lately I've been accepting interview requests for principle roles from on-site recruiters, and I've been consistently rejected after these Coderpad tech screens. What's bugging me, however, is the "challenges" have all been softball questions. Reverse a string. Demonstrate an understanding of closures. Write a function that tests if two strings are anagrams. Write some basic React. Create an interactive form. I breeze through well enough, then it's either an immediate rejection or else they ghost me. Not once do I get farther than that. I'm not even mad, just confused as hell.
This is coming from the perspective of someone who was getting used to offers from the first job they applied to. If there's something off-putting about me lately, I wish I knew.
I've never done this, but you might consider getting an experienced interviewer to do a mock interview with you (of the same sort you've breezed through before). It might be a coincidence that you have been rejected multiple times, or it might be some small behavioral thing that someone could tell you about.
Alternatively, you can say "screw this," and I wouldn't blame you.
Huh, I like that idea. Thank you. Possibly a good excuse to get in touch with some former colleagues as well, which I suppose couldn't hurt either.
Mercifully, I'm not quite yet desperate about leave my current role. But I do feel like my career (and salary) progression is stagnant; not seeing much opportunity where I'm at. I'm thankful for what I've got, but I think it's smart to be looking. Just found it weird to face rejection so much from people reaching out to me in the first place, in a supposedly hot market.
It's tricky to interview people lacking experience, but with >=decade of experience just do deep dives into past projects. You're basically talking to one of if not the authority on whatever it was they worked on, let them tell you all about it, down to the weeds. If they can't, then you've probably got a problem.
I have 15 years of experience and have found unless I'm contacted directly by the internal recruiter of a company it's pretty much impossible to even get a reply to a job ad.
I don't know how people who are new to this industry are managing to find work. Pretty sure I'm getting filtered near 100% of the time.
I don't ask candidates to code on a phone screen. For coding proficiency, I show them code with flaws and ask them you do a code review. To break the ice I just ask them to describe the program behavior line-by-line.
For a good candidate, it's a jumping off point for a deeper discussion.
coderpad interviews test performance anxiety. there is a set of programmers without performance anxiety and there is a set of programmers with performance anxiety. whats interesting is that the cargo culting of coderpad interviews from google on down means google takes the best programmers with acting skills (thanks to infinite money) and everyone else takes the second tier of programmer with acting skills. the best programmers without acting skills (like kevin here) are left out. the plus side, for them, is they may find the set of companies who arent cargo culting and might really have a clue.
I think I would prefer to work with someone who optimizes their strengths the way kevinburke describes in his post. However as a manager I would structure phone screens to quickly filter out candidates who would never work out, no matter how optimal they seem. This is an art and odds are candidates are never going to have the perspective to appreciate the aesthetic of the completed work…
I too can pass on-site interviews, but my problem is I don't even get a first phone screen to begin with, they discard me without even spending 5 minutes on me.
You Americans (even afroamericans) have absolutely zero clue about how racist your companies actually are.
"But Google and Microsoft CEO are Indian!" Yeah, but not really.
By the way, I'm Southern European, with a Southern European sounding name.
The secret to these is to not play the game. If you send me to a website like coderpad I'll just say thanks but no thanks. If my code portfolio and extensive experience isn't enough to get me to talk to one of your real engineers and find out what I know (that I'm not bullshitting) then we should both stop wasting our time and you can find someone else.
Yeah well, I recently flunked a role I had years of experience doing. The expectation I assume in some of these phone screens is that you will lie to get the job. I refused to tell them that I had done something I hadn't and laid it out to them why something they were asking for wasn't possible in an IC role. I refuse to play these games.
> I'm not fast at reasoning about code, and I often make trivial mistakes just trying to get a "first draft" of a program out. If I get behind or the interviewer starts interrupting to ask about the bad code I'm writing I get very stressed and have trouble both listening to the interviewer and trying to address the issues with the code.
Sounds to me that the author likes to program by "sketching". I do too! Paul Graham recommends this in Hackers and Painters.
> For example, I was taught in college that one ought to figure out a program completely on paper before even going near a computer. I found that I did not program this way. I found that I liked to program sitting in front of a computer, not a piece of paper. Worse still, instead of patiently writing out a complete program and assuring myself it was correct, I tended to just spew out code that was hopelessly broken, and gradually beat it into shape. Debugging, I was taught, was a kind of final pass where you caught typos and oversights. The way I worked, it seemed like programming consisted of debugging.
>
> For a long time I felt bad about this, just as I once felt bad that I didn't hold my pencil the way they taught me to in elementary school. If I had only looked over at the other makers, the painters or the architects, I would have realized that there was a name for what I was doing: sketching. As far as I can tell, the way they taught me to program in college was all wrong. You should figure out programs as you're writing them, just as writers and painters and architects do.
On the other hand, as the post describes, interviews are geared towards a more waterfall-y approach. I find this frustrating as well, but I think that there is something really important to keep in mind that this post misses, and that most conversations about coding interviews miss. The question is whether something is _predictive_ of job performance.
For example, consider the question "What is your favorite number?". Imagine that the larger the answer to that question, the more likely you are to perform well as a developer. You might object "But I'm never doing anything relevant to my favorite number on the job!". IMHO, it doesn't matter. The goal is to predict who will perform well on the job, and if having a large favorite number does this, it should be incorporated.
Now consider the question of how well you perform on waterfall-style interview questions. Like "What's your favorite number?", it isn't something that you will find yourself doing on the job. But that isn't the right question. The right question is whether or not it is predictive of job performance.
Of course, it seems unlikely that "What is your favorite number?" would predict job performance. And it seems unlikely that if you never code in a waterfall-style on the job, that coding in that style during an interview would predict job performance. But maybe it does. I'm skeptical, but who knows. More importantly, I think that this is the question that needs to be asked. And without strong data, it's hard to be too confident in the answer.
>IMHO, it doesn't matter. The goal is to predict who will perform well on the job, and if having a large favorite number does this, it should be incorporated.
What if it was skin color, how ethnic your name is, or your gender? Did you know the majority of developers are men? Should we throw all the female applications in the trash because they're statistically not as likely to be developers?
Yeesh. Sorry if I sound hostile but... jeez. Correlation is not causation, etc etc.
Things like skin color are different from things like leetcode ability. Looking through a veil of ignorance, I definitely would not want hiring decisions to be made based on the former. But looking through the same veil of ignorance, I would be ok with hiring decisions being made based on the latter.
Well, actually, thinking about it again now, I'm not so sure. But this now becomes an ethical question. Which is certainly important. But I want to be clear that the ethical question is a different question from the question of what a purely self-interested company should do. To that self-interest question, I think the OP and lots of people argue that the answer is "leetcode-style questions having nothing to do with the job and are thus a bad idea".
My claims are that 1) this is the wrong way to think about that self-interest question. The right way is to ask whether it is predictive of job performance. And 2) although I also suspect that leetcode-style questions aren't predictive, it's hard to be confident in that without good data.
Didn't open this post/article because I assumed it was about mobile phone display technology. Couldn't have made just a tiny bit more effort on the title wording?
As Jensson points out, smart people can grind leetcode, so if you have good experience, grinding leetcode shows you're smart enough to get the job. If you don't grind, there's no signal of how good you are at coding, so it's a no hire.
Really, it's on you if you don't do what needs to be done to get the job. So get grinding! Or remain unemployed.
Recently, we had a candidate come through that wasn’t a good fit for us (he claimed to be a cypress expert so we asked him to set cypress up in a dummy react repo and he floundered the entire time). One of my coworkers took some pot shots at the candidate, being deeply critical and rude, cutting the candidate off mid-sentence, etc. I was honestly embarrassed to be on the call and I thought it reflected poorly on my company. I’d never seen my coworker act like this before and it changed the way I felt about him.
I wonder how much of the leetcode interviewing culture is rooted in this desire to dominate candidates, since ICs rarely have opportunities to wield such power over others.