The short answer is, "Yes." They are very flawed, but one of the most reliable ways to avoid "bad" hires.
The longer answer is:
Fundamentally, you need to address the fact that there exist a huge number of people in this industry declaring they have masters degrees/phds or years of industry experience, but when pressed they can't write even the simplest of functions.
While we called it out explicitly, some folks seem to miss that "Custom" questions are still fundamentally DS&A leetcode-style questions. I completely agree that "leetcode style" interviews are flawed, but most people don't have a better answer for this problem that still guarantees the person you're hiring actually can code.
We are optimizing for coders that make good choices quickly, and if they can code efficient code to toy CS problems, then you at least guarantee that 1) they can actually code and 2) they can code simple things quickly. Non-coding interviews allow you to hire people who can't do these basic things and therefore guarantee their performance is worse in a production environment.
It's important to note here that leetcode questions doesn't mean "LeetCode Hard" where the only solution is an obscure algorithm that is on average never used in production code (except deep in libraries you didn't write).
Leetcode questions can be fairly basic algorithmic/data structures questions, like "how do you store an ordered collection with O(1) lookup (ordered dict, i.e. hashtable + list)" or "check if this list contains duplicates" (use a set, not a list!), or "calculate the square root of this number" (bisecion, i.e. looping).
These are not production code questions, but the benefit is that they're easy to explain and understand in the limited time you have available for an interview.
Unfortunately there are more than expected number of people who fail these basic questions. You really don't want to hire them.
Let's keep it civil, my friend. I totally get the frustration with leetcode-style questions and acknowledge that they aren't perfect. They can indeed overlook the diverse strengths that experienced engineers bring to the table. My perspective is that understanding the basics, like the efficiency of different data structures, is crucial, not just for interviews but for making informed decisions in our work.
I'm not advocating for spot tests on complex algorithms without context. However, I believe a conversation about fundamental concepts like big-O notation reflects on one's approach to problem-solving and software design. It's not about dismissing anyone's experience or capability but ensuring a solid foundation that benefits all aspects of engineering work.
I understand senior engineers being out of practice and not being able to derive things like topological sort on the fly, but an inability to talk about the basics of big-O or the simplest of data structures is more commonly a red flag than "rust." If they never learned the material, that's a red flag. If they learned the material and they've been building software for the last decade without taking the basics of the material into account while coding, then that is a red flag as well.
Again, happy to admit that this is a flawed approach that will lose some great engineers, but most engineers that have your line of thinking are ones I actively avoid hiring. These concepts are practical, available to learn for free, and easy to understand. If an engineer feels that testing these concepts is beneath them, then they definitely aren't a good fit for any team I'm on. Big-O is to software engineers what Ohm's Law is to electricians. Imagine a world where electricians thought it was demeaning to talk about Ohm's law in an interview. ¯\_(ツ)_/¯
> Big-O is to software engineers what Ohm's Law is to electricians.
Really? Most of the work that exists is CRUD, and I've seen people who grinded leetcode algorithms, do 1+N SQL queries which is way, way worse, than forgetting what kind of time complexity a sorting algorithm has that you never implement yourself anyway.
At the same time these leetcode grinders will over engineer a UI solution to make sure they use the correct data structures to avoid looping 300 times in the browser at the cost of readability writing 30 lines of code instead of 4.
I don't remember a single time where I've used Big O in practice.
I ask myself - what will happen with performance if there's N amount of traffic or data, but I don't think of it in terms of Big O. I consider perhaps a time it takes or load it takes at upper limit numerically.
N might in some cases be terrible, and N times N performance in other cases totally fine.
I think actually thinking in terms of Big O might make you worse engineer as you start to obsess over the wrong things.
I prefer someone who's spending all that time on building things instead. This time of leetcode grinding and memorising data structures, algorithms could be used on coding side projects.
You learn way more and more practical things when you build something.
The longer answer is: Fundamentally, you need to address the fact that there exist a huge number of people in this industry declaring they have masters degrees/phds or years of industry experience, but when pressed they can't write even the simplest of functions.
While we called it out explicitly, some folks seem to miss that "Custom" questions are still fundamentally DS&A leetcode-style questions. I completely agree that "leetcode style" interviews are flawed, but most people don't have a better answer for this problem that still guarantees the person you're hiring actually can code.
We are optimizing for coders that make good choices quickly, and if they can code efficient code to toy CS problems, then you at least guarantee that 1) they can actually code and 2) they can code simple things quickly. Non-coding interviews allow you to hire people who can't do these basic things and therefore guarantee their performance is worse in a production environment.