I want to second this style of question. My company, when we still conducted in-person interviews pre-Covid, gave a similar interview to this. We'd give someone a broken toy webapp, have them fix some bugs, and then discuss what they'd do to improve it.
It had a few benefits:
First, this is much closer to the type of work that we'd ask a new employee to do, as compared to the typical industry interview question: "did you pay attention in computer science? You did? Now prove it by solving this problem that's a thinly-veiled algorithm exercise. Using a computer? Oh heck no, you're going to use a marker! There's no trick to it, we don't ask interview questions that have tricks, but if it leaks externally, it's 100% useless and we cannot ever ask it again"
On that topic, who cares if the premise leaks? If the code somehow got out, we'd just write another one.
We get to hear them talk, out loud, about their expectations and how the app's organization differs from them. This isn't interesting by itself, but the candidate gets a chance to articulate their experiences and how well they notice patterns. For more senior engineers, you get to see how quickly they can pattern match and explain the structure of the client and server, and explain how it could evolve under different circumstances. "You're in a startup that just got a ton of VC based on this hackathon project. Your friends hire you and give you this codebase. How do you spend your next 2 months?"
Also, you'd be surprised how many people disqualify themselves on attitude. People will read sections of the toy app and say out loud, "who the hell wrote this? This is terrible." And you'll ask them, "What do you mean by that?" and they'll rant about some section of it, and you're like, "wow, I really don't want to work with this person." If these engineers can't even keep it together during an EXERCISE whose PREMISE is that the code doesn't work well and can be improved, it's a negative signal that they're going to demonstrate any empathy when they're working with you or any of your colleagues over the years.
It had a few benefits:
First, this is much closer to the type of work that we'd ask a new employee to do, as compared to the typical industry interview question: "did you pay attention in computer science? You did? Now prove it by solving this problem that's a thinly-veiled algorithm exercise. Using a computer? Oh heck no, you're going to use a marker! There's no trick to it, we don't ask interview questions that have tricks, but if it leaks externally, it's 100% useless and we cannot ever ask it again"
On that topic, who cares if the premise leaks? If the code somehow got out, we'd just write another one.
We get to hear them talk, out loud, about their expectations and how the app's organization differs from them. This isn't interesting by itself, but the candidate gets a chance to articulate their experiences and how well they notice patterns. For more senior engineers, you get to see how quickly they can pattern match and explain the structure of the client and server, and explain how it could evolve under different circumstances. "You're in a startup that just got a ton of VC based on this hackathon project. Your friends hire you and give you this codebase. How do you spend your next 2 months?"
Also, you'd be surprised how many people disqualify themselves on attitude. People will read sections of the toy app and say out loud, "who the hell wrote this? This is terrible." And you'll ask them, "What do you mean by that?" and they'll rant about some section of it, and you're like, "wow, I really don't want to work with this person." If these engineers can't even keep it together during an EXERCISE whose PREMISE is that the code doesn't work well and can be improved, it's a negative signal that they're going to demonstrate any empathy when they're working with you or any of your colleagues over the years.