The transaction itself is not zero sum. The existence of the transaction itself is proof that both parties value what they received more than what they gave away (otherwise they would not have transacted).
The waste of time is having to haggle to arrive at the price, when the price could have been published. It benefits nobody, because that's the price the transaction would have happened at anyway. But we have to do a whole song and dance beforehand anyway, every single time.
This of course is under the assumption that time is expensive relative to the accuracy of the market clearing price of the goods being sold. In many cultures this isn't true, people have lots of free time, but each person buying the good might assign different value to it, so by spending a bit more time the people who need the lower price can still access it, whereas the shopkeeper can still make the money to keep the shop running from those who can afford it.
Honestly, thinking that there should be fixed prices for everything, if you're a rich westerner going in, means you're trying to externalize the effort of setting a market clearing rate, and trying to piggy-back off of the locals. If it's at some multinational chain, then sure, don't have people haggle with you over the price of some McNuggets, but at a family owned business, if they can get an additional 20% of the price of the goods they might be doubling their profit margin, so why wouldn't they haggle?
The seller knows the customary price, and the lowest price they would be willing to accept. The "real" price is whatever the buyer and seller agree on for a particular trade. Price discovery is an ongoing process which isn't finished just because you've determined an average price for past trades. The buyer isn't losing out unless there's actual fraud involved, even if they could have potentially negotiated a better price. And really, we're mostly talking about trades between individuals and small businesses (who may also be individuals)—why should all the surplus value represented by the difference between the highest and lowest prices acceptable to both parties go to the buyer, and not the seller?
The waste of time is having to haggle to arrive at the price, when the price could have been published. It benefits nobody, because that's the price the transaction would have happened at anyway. But we have to do a whole song and dance beforehand anyway, every single time.