An honest question for HN: has anyone really been in a project whose primary reason for failure was the lack of engineering talent? I have worked in micro/small businesses and large enterprise environments, and this has never been the case. The projects I have seen fail usually do so due to poor management, poor performers are typically fired unless they can pull one over management to make themselves seem more competent than they really are (not that difficult with bad management)
I have. It's a long and interesting story, full of intrigue, backstabbing, hacking, and corporate espionage. But in summary:
Owners started tech business using trust-fund money. They were extremely non-technical, scared of computers to be honest. They hired people that were like them (also non-technical), and didn't have the ability to properly vet candidates. Tech staff was hired basically off of Craig's List with very little interviewing. Tech was built and company got large enough to get a couple contracts (heavily influenced by owners' existing personal connections).
As you'd expect, tech stack started creaking under the load. At first tech staff thought they had everything under control (strong Dunning-Kruger effect, bolstered by early successes). Eventually tech staff realized how badly things were going and started lying to management to cover for their own incompetence.
Decent tech management was hired (I think by accident) and the tech staff saw the writing on the wall. The entire tech staff just didn't show up to work one day. Left without any notice or warning (the more intriguing stuff tangents off here). New, more capable tech staff was hired (including me). At this point the system was hemorrhaging data and users. We triaged, stemmed the bleeding, and built out a decent infrastructure with a decent tech stack.
Unfortunately for them that was too late. The contracts had fairly stiff penalties for non-compliance, and the company was in non-compliance for a looooooong time. The owners had made some false statements about the assets of the company, and settlements offered were based on those statement, far too high for the company to pay. Other companies were pissed about that and took it to court. Owners ended up getting taken to the cleaners and lost pretty much everything, including a bunch of assets that they had owned before but transferred to the company (including real estate) in some fairly shady tax dodge scheme.
Ultimately the world is a better place now that this company doesn't exist. It operated in an extremely sketchy manner, and I left when I found out the gist of what was going on. I learned the earlier and later parts of the story from personal connections.
Does that count? I mean, ultimately I think if you have poor engineering talent then it's management's fault. But you can make that argument for any corporate issue: it's management's fault for not recognizing the problem and addressing it soon enough.
I don't think this is exactly the issue the interviews are trying to prevent... As mentioned by a commentator below, once somebody is hired it would be extremely hard to get them out, especially for big corps. "Project failure" is an extreme case but the loss can still be huge. Therefore the employer just wants to be rather safe than sorry with a set of standardized tests it seems.
Also, to be able to thoroughly study for and prepare for something on one's own, this on itself probably illustrates a lot about this person's qualities already. Of course it would be theoretically nice to spend all this time and energy on something that is unique to every person, but that is not totally realistic and would be hard to evaluate. One can still do that with their side projects though.
In the short term, no. Companies hire in order to support the growth they are projecting. Over the course of a year though a stressed team will achieve less, introduce more bugs, and make poor long term architectural decisions.
