Great article, Patrick, but I'm a little concerned about this line: "I will likely finalize the redesign and kill the old version in the coming weeks."
Please, please tell me that if you kill the old version you will have the numbers to show that that the old one is not, say, at least 5% better than the new one. At the moment you seem to be 'leaning' towards the new version even though sales from the old version are higher!
It doesn't matter how much time and money you spent on the new version, or how much better it looks, or how much easier it is to use. What matters at the end of the day is how much money it makes, so run the split test until you are (statistically) confident that you are not throwing money away when you kill one of the designs.
Given that Patrick's tentative conclusion was that the new version makes a similar amount of money, but allows more people to get value from his software, leaving up the old design would seem a bit... hard-hearted.
The problem is that such a conclusion is too tentative at the moment. It's possible that the old version is 5% better than the new version. My guess is that 5% of revenue over the next 10 years could be around $40,000 - that's a lot of money to give up! Why not keep the split test running until you can be statistically confident that the two versions make similar amounts of money?
Please, please tell me that if you kill the old version you will have the numbers to show that that the old one is not, say, at least 5% better than the new one. At the moment you seem to be 'leaning' towards the new version even though sales from the old version are higher!
It doesn't matter how much time and money you spent on the new version, or how much better it looks, or how much easier it is to use. What matters at the end of the day is how much money it makes, so run the split test until you are (statistically) confident that you are not throwing money away when you kill one of the designs.