A decent way to normalize this effect is to take relatively detailed notes of the exchanges, and make the decision a few hours or a day later rereading the notes.
One of the issues we had interviewing very young candidates (like fresh out of college) was how they all looked awkward, even in their outfit, and few had any kind of confidence during the 15~20 min we spent with them, with some saying really weird things (stuff like "I'm really good at the internet"). But obviously non of that matters long term, we assumed they'd probably all fit in fine once in. Putting some distance and picking up the good and bad stuff from the transcript helped a lot to get past the weird impressions, including the order we saw them and how tired we were when we saw them. To some point.
>Recent research suggests that judgmental anchoring is mediated by a selective increase in the accessibility of knowledge about the judgmental target. Anchoring thus constitutes one instance of the judgmental effects of increased knowledge accessibility. Such knowledge accessibility effects have repeatedly been demonstrated to be fairly durable, which suggests that the effects of judgmental anchoring may also persist over time. Consistent with this assumption, three experiments demonstrate that judgmental anchors influence judgment even if they were present one week before the critical judgment is made. In fact, the magnitude of anchoring was found to remain undiminished over this period of time.
I didn't get access to the full text, but had a look at other papers from the same researcher [0] on what kind of methodology they use.
In the case of recruiting, I think the main factor when moving the decision further down the line is the change in information ("a selective increase in the accessibility of knowledge about the judgmental target"), in two specific ways:
- we actually remember less about the subject, for better or worse. A candidate might have had a weird look, and the notes are probably impacted by that bias, but we can look back at their coding test without that impression and come out with a slightly different conclusion.
- we get to compare to other subjects in a different order. In particular, that helps catching weird expectations. For instance if every candidates has been falling through the same trap, it helps give them a pass and assume the question was at fault. If we had to do that in real time, only the last few ones would get a kinder judgement.
I've never had such a thing but many years ago, not long out of university, in my previous career as an electronics engineer I was asked to design a simple amplifier before the interview proper. The interviewer explained, slightly apologetically, at the end of the interview that he did this just to sort out those who were good at talking but didn't have a thorough grounding in the basics from those who were well grounded but perhaps not so good at blowing their own trumpet. I was pleased to find that I passed that part with flying colours :-)
But I would not want such things to be taken very seriously unless you trying to fill a very narrowly defined post because it is all to easy to create a test that a good candidate would fail.
I think they're very valuable if the position requires any coding at all.
In particular very simple tests (like an API interface, or reversing a string etc.) done in any language they feel comfortable is are usually a trove of info about the candidate. The result doesn't really matter, it doesn't need to run, it doesn't need to be complete, as long as you got to hear a lot about how the candidate thinks, how he moves through the problem, and how much they can write something basic, what they're confident in and what they're not used to do etc.
which makes sense, if someone made an impression on you that impression doesn't disappear in just a few days. At best it may be fuzzier, which could be good or bad.
Interestingly enough, yes, but you also understand it reading the notes. For instance they become sparser and sparser, or tendencies arise.
I would compare it to reviewing one's code a few hours later. We're still in a similar mindset, but there's a bit more distance, and we also catch the bits that don't make sense when reading back afterwards. That works even better when exchanging notes afterwards.
Yes, we hired a few that stood out. They were indeed kinda weird for a few months, some got blander afterwards and some stood a lot more, but all in all they were meeting the bar we had in mind, and the ones that really grew weren't those we expected at first.
In particular we had people who's surface personality were completely different from what we perceived during the interview. Not in any way that made it hard to work with them, but moving from university to a corporate environment was just enough of a gap to change their behavior in significant ways. I think hiring fresh graduates is way harder in that respect, and we were happy to have some flexibility in the work culture. One of the guy moved from a super rural area to be thrown into the megalopolis, and it was a real journey, we had the funniest late to work excuses ("couldn't find my bike cause I parked it near the neighbor's house and he moved it in their garage thinking it was his son's" -- later found his house had a bike parking behind the building)
One of the issues we had interviewing very young candidates (like fresh out of college) was how they all looked awkward, even in their outfit, and few had any kind of confidence during the 15~20 min we spent with them, with some saying really weird things (stuff like "I'm really good at the internet"). But obviously non of that matters long term, we assumed they'd probably all fit in fine once in. Putting some distance and picking up the good and bad stuff from the transcript helped a lot to get past the weird impressions, including the order we saw them and how tired we were when we saw them. To some point.