Sorry, tried to answer and apparently I was "posting too fast".
One may start with enumerating the features that needs development (a.k.a. "user stories"). Also enumerate external dependencies, team overhead (more people working together means overhead is greater). For each feature enumerated in step 1, try listing as much detail as possible, also listing open issues and unknowns.
For example, list all the UI elements under development. List all individual functions / use cases that need distinct functionality (classes, functions) to be developed. List all the test cases. List external APIs, enumerate different ways the said API can be used. Enumerate failures and possible recovery actions.
I'll give you an example: a login dialog for an application. This starts as a two page requirements document, which balloons to around 120 items using this method.
This alone should give a good idea at the start of the project. Keep doing it (enhancing the level or detail), while keeping the ratio of initial estimate vs real value - that will help in estimating the remaining portion of the work.
The problem (the way I see it) is that we don't really have the tools to do this kind of tracking and analysis. Jira and similar systems don't even come close.
Here is an attempt to apply the methodology using Excel:
This is roughly how I was taught to estimate. It is accurate but takes time and everybody hates the result. Management halved my estimates without telling me, made fun of my mentor’s estimates, complained about needing to know everything upfront (not very agile eh!), etc. I think you are right and estimating is more of a solved problem than we think, some people just don’t want to admit it!
And then the customer decides that they don't want to move forward with the project because it's too expensive. Oh, and they don't want to pay you for all the time you spent estimating because no one pays for estimates up front anyway.
But actually, once you show the customer what's involved, and give them a realistic estimate, it might reduce the chances of cancellation. The discussion might be steered towards what they really want, or prioritization of deliverables.
A mockup (and multiple iterations thereof) might be necessary as well.
One may start with enumerating the features that needs development (a.k.a. "user stories"). Also enumerate external dependencies, team overhead (more people working together means overhead is greater). For each feature enumerated in step 1, try listing as much detail as possible, also listing open issues and unknowns.
For example, list all the UI elements under development. List all individual functions / use cases that need distinct functionality (classes, functions) to be developed. List all the test cases. List external APIs, enumerate different ways the said API can be used. Enumerate failures and possible recovery actions.
I'll give you an example: a login dialog for an application. This starts as a two page requirements document, which balloons to around 120 items using this method.
This alone should give a good idea at the start of the project. Keep doing it (enhancing the level or detail), while keeping the ratio of initial estimate vs real value - that will help in estimating the remaining portion of the work.
The problem (the way I see it) is that we don't really have the tools to do this kind of tracking and analysis. Jira and similar systems don't even come close.
Here is an attempt to apply the methodology using Excel:
https://github.com/andy-goryachev/PasswordSafe/blob/master/F...