What killed Concord iirc is that only a few airports could handle them. A rich person would just prefer a slower private jet that can land much closer to their destination. Especially since today you can basically turn your plane into a private office.
The sonic boom limited them to only flying over the sea. They could technically fly at any airport that handles large passenger jets. But if you fly supersonic over land you will piss off everyone below you.
The outcomes were SUBSTANTIALLY worse than the US government expected. The results of this study are why even war planes rarely go supersonic over the continental US.
I remember the sonic boom over the Seattle area in the summer of 2010 when Obama was in town. A seaplane accidentally violated the 10 mile exclusion zone around Air Force 1 at Boeing Field and two F-15s were scrambled from Portland International Airport (Oregon National Guard operates the fighters tasked with protection from Northern California to the Canadian border). It was quite exciting.
In addition to the sonic boom issue it was also cost. It cost a stupidly high amount in fuel to fly. The engines were horribly inefficient.
Not only that the low capacity meant that tickets would have to be orders of magnitude higher than even first class in any other commercial jet.
It's why I cant see Boom doing what people are assuming they'll do and make a commercial airliner. Unless you fancy spending $20k for a one way trip from JFK to LHR it's not really viable.