I am glad this discussion is being brought to the fore. In my experience, far too many managers are clueless about what engineering actually means. They see it like warehouses or factories where the number of units shipped == output. But in engineering, number of PRs or commits or lines of code are meaningless. In fact, setting these output targets take away time from engineers in doing things such as meaningful design choices, system health investigations, cross-team situations.
I see software engineering more as a team sport than as an individual sport. In team sports, each individual sacrifices some metrics for the good of the team.
This was a great writeup. And shows that this management style is not just about programming, but any activity at all. The article mentions sales as having just such maddening performance metrics and just brushes past it as ok/obvious/normal while it's just as crazy as your football example.
> I feel like you're stating this as "Look how silly KPIs are!", but your CEO is correct. The issue isn't the KPIs; it's the team members who are willing to intentionally game a system in conflict with overarching company goals.
The real issue is - KPIs are used to incentivize or PIP employees. Why are you setting the employees in a game that is in conflict with your company's overarching goals?
What about things like "Stuck on XYZ because no one fixes it because it doesn't show any good metrics and management doesn't care about it. So I fixed it at the expense of my own time."
>What about things like "Stuck on XYZ because no one fixes it because it doesn't show any good metrics and management doesn't care about it.
I'm confused by your comment. How did you decide this was something worthy of fixing if it "doesn't show any good metrics"? If you can't quantify the issue in any manner, how do you determine it's worth doing?
This is especially important when metrics would not be expected to be available -- for example, if you're designing a nuclear reactor, you need to think hard about ways to prevent a meltdown in advance, rather than collecting meltdown statistics and then fixing the piping problems that correlated with the most nuclear meltdowns.
This is also necessary when the true metric that matters is very hard to evaluate counterfactually. For example, perhaps your real task is "maximize profit for the company", but you can't actually evaluate how your actions have influenced that metric, even though you can see the number going up and down.
And necessary as well when a goal is too abstract to directly capture by metrics, resulting in bad surrogate metrics: for example, "improve user experience" is hard to measure directly, so "increase time spent interacting with website" might be measured as a substitute, with predictable outcomes that bad UI design can force users to waste more time on a page trying to find what they came for.
All of these problems are faced by metric designers, who need to pick directly-measurable metric B (UX design metric) in order to maximize metric A (long-term profits) that the shareholders actually care about, but they cannot evaluate the quality of their own metrics by a metric, for the same reason that they were not using metric A directly to begin with.
There is also the classic story about returning war planes. You can try to make sense of the damage, the bullet holes, and try to create strategies around how to improve affected areas. The problem is that the damage you actually want to inspect and prevent is on the planes that did not make it back, the ones you do not have an abundance of data on.
Let's say you've got a logging system that sometimes drops lines. And this sometimes makes debugging things hard, because you can't say whether that log line is missing because the code didn't run, or because the log line was lost.
Impact on end users? Nothing measurable. Impact on developers? Frustrating and slows them down, but by how much? It's impossible to say. How often does it happen? Well, difficult to count what isn't there. Would fixing the issue lead to a measurable increase in stories completed per week, or lines of code written, or employee retention? Probably not, as those are very noisy measures.
Nonetheless, that is not a fault I would tolerate or ignore.
> I'm confused by your comment. How did you decide this was something worthy of fixing if it "doesn't show any good metrics"? If you can't quantify the issue in any manner, how do you determine it's worth doing?
Because sometimes some things without metrics are incidental to the actual thing you set out to do.
E.g. a large refactor that switches libraries which is necessary for your new service that give 10% lower latency. But that library refactor will need to be done and it will take 2 months.
This is a challenge often encountered by product owners. At times, what appears to be a problem may not actually be one. Conversely, you might discover that a perceived issue holds greater significance than the current circumstances or priorities suggest.
> Anything surgical is typically billed against a dozen or more CPT codes for a single procedure and often involves more than one provider.
This is the real startup gap. There are thousands of CPT codes. When a medical provider is trying to give an estimate, it should be easy for Medical providers to have packaged, template CPT codes for template procedures. Then, they should be able to add/remove CPT codes from the package (like drag and drop), and the prices should change automatically.
The template packages could even be put on a social marketplace for doctors so that the information is shared.
That was what Nuna (nuna.com) was originally trying to do ~10 years ago (put together bundled payments for value-based care models). I think they've since pivoted to more general healthcare data tools.
One of the difficulties is that in many systems medical billing is done by coding specialists based off the provider's note. They may recommend CPT codes, but that may not be what's actually billed. In addition, most providers are too swamped to do things like put together an estimate or drag and drop CPT codes. Hell, many providers will literally count the clicks they have to make in an EHR and will LOUDLY let you know if your proposal will increase their number of clicks by even one.
I don't mean to be a downer on this, and I do think there are solutions... but I think 90% of the problems in healthcare aren't technological ones but are navigating large, entrenched systems that have very little incentive to change.
> I don't mean to be a downer on this, and I do think there are solutions... but I think 90% of the problems in healthcare aren't technological ones but are navigating large, entrenched systems that have very little incentive to change.
Having led provider operations and data systems in various settings for the last decade, this is absolutely true. I find a lot of the 'healthcare is ripe for disruption' comments miss that most of the work isn't going to be fixed by some neat javascript or whatever.
To your note on CPT codes, I'd also bet that, if a given provider is seeing ACA or MA patients, their billing systems and payer interactions will also be more complicated, in terms of coding (diagnosis and procedure), in order to satisfy risk adjustment needs. It's effectively impossible to incent a provider to use two entirely different systems, depending on who the patient is.
I have seen managers who were very effective at their work. But they actually understood the software in depth and were very skilled at shielding the team from politics.
Historically we've had the opposite problem. Countries tend to move toward austerity at the wrong moment in time, due to political pressures or pressure from debtors. (Though this is much more difficult to navigate when the country's debt is denominated in a foreign currency e.g. Japan with 200%+ debt to gdp which is having a hard time meeting inflation targets vs Argentina).
Even today, the European Central Bank is signaling that it has effectively done all it can do (without permanently harming the banking sector with negative rates), and that it is time to open the doors to fiscal stimulus; but, Germany, which is going through a manufacturing recession, is loath to update their constitution to facilitate fiscal stimulus.
I see software engineering more as a team sport than as an individual sport. In team sports, each individual sacrifices some metrics for the good of the team.
Here is my rant about how soccer teams would perform if they were evaluated like the McKinsey-style nonsense becoming pervasive in the tech industry: https://medium.com/@NTDF9/if-soccer-managers-did-performance...