OK, here's my take, based on 30 years of being a programmer and "manager" and "architect" and other position buzzwords.
I hate the title software "engineer" because engineering is a set of practices and processes that software creation doesn't come close to complying with.
The primary problem is the feature backlog. It's always been that, whether it was called the "Requirements" or the "Functional Requirements" or whatever.
Working out what the product of the effort has to do is hard. It involves understanding the actual end users (not the managers of the process/product), the process/product itself, and the context and environment in which it will operate.
The stakeholders are multiple and on multiple levels. There are managers and auditors that need visibility of the ongoing operations, there are users and consumers that need the actual input or output.
I've never met a "Product Owner" that actually understands the product they are building or where and how it is expected to be deployed and operate. They rarely filter out unnecessary or dangerous "features", but just add them into the overall bucket without knowing where to draw the boundaries.
The "solution" is multi-faceted, but it needs to be directed at this area.
Domain Driven Design is one approach, actually engaging with the actual end users is vital.
Sometimes, ignoring a manager's desire (particularly for "dashboards" and "reports") is very important to keeping the feature backlog manageable.
Understanding the context/environment is also vital. Does this product deal with individual's personal information? How does SOC affect it? Is the process required to be auditable? If it is, what evidence is required for the auditors?
None of this is "software" per se or even "programming", or, even worse, "engineering". It's process and product and understanding what is being built.
This is an interesting short series of articles from 2021, interviewing 17 "real" engineers (the proper build-a-bridge type) who transitioned to software engineering to explore whether they consider software work really engineering.
And 15 of the 17 agreed that yes software engineering really is engineering.
>I've never met a "Product Owner" that actually understands the product they are building...
This is sad. It's like a professional football player saying they never played for a coach who understood the objective of football. If that's truly the case, no amount of software development best practices or skill will save you from failure.
I have seen product owners who understand the product, and clearly those people exist, as there are successful products in the market. But if you've spent your career playing for the software equivalent of the Detroit Lions, I can see how the industry could appear pretty bleak.
I agree about product owners. I have never seen one that could really guide a project from start to finish. They either don’t understand the tech sufficiently, or they don’t understand the business case or they aren’t engaged enough.
Understanding the tech sufficiently and effectively rallying the team around the actual business/user justification for work are literally the job of a product manager who owns tech product development.
While they may not be common, actual product managers do exist. There are a lot of incompetent ones, to be sure, but there are also a number of incompetent engineers.
The problem is promotion culture. Once a product manager has enough experience to understand the business and the tech they get promoted and you have to deal with a newbie.
It’s so nice to deal with experienced product managers and project managers but it’s rare to deal with them.
> The problem is promotion culture. Once a product manager has enough experience to understand the business and the tech they get promoted and you have to deal with a newbie.
Where do all these PMs get promoted to? If it is to a "manager of PMs" role, that sounds as if the company is always launching ever more products, and that promotion to a higher level with more pay while remaining a PM isn't an option.
In my personal experience, most get promoted to management of people and their skills in project management atrophy over time. Some break that pattern and instead move to consulting. Most companies I’ve worked with and for do not have a career path for a competent, highly skilled PM who wants to remain just that and further hone his/her skills. The choices are up (to people management) or out (to consulting).
Sounds like the career path for a PM is even more constrained than that of a typical technical IC. At least the latter have a few "senior", "lead", "architect", and "staff" rungs they can try to climb before being forced to go the management route.
I hate the title software "engineer" because engineering is a set of practices and processes that software creation doesn't come close to complying with.
The primary problem is the feature backlog. It's always been that, whether it was called the "Requirements" or the "Functional Requirements" or whatever.
Working out what the product of the effort has to do is hard. It involves understanding the actual end users (not the managers of the process/product), the process/product itself, and the context and environment in which it will operate.
The stakeholders are multiple and on multiple levels. There are managers and auditors that need visibility of the ongoing operations, there are users and consumers that need the actual input or output.
I've never met a "Product Owner" that actually understands the product they are building or where and how it is expected to be deployed and operate. They rarely filter out unnecessary or dangerous "features", but just add them into the overall bucket without knowing where to draw the boundaries.
The "solution" is multi-faceted, but it needs to be directed at this area.
Domain Driven Design is one approach, actually engaging with the actual end users is vital.
Sometimes, ignoring a manager's desire (particularly for "dashboards" and "reports") is very important to keeping the feature backlog manageable.
Understanding the context/environment is also vital. Does this product deal with individual's personal information? How does SOC affect it? Is the process required to be auditable? If it is, what evidence is required for the auditors?
None of this is "software" per se or even "programming", or, even worse, "engineering". It's process and product and understanding what is being built.