As pointed out by emarcotte, the claim is not that anyone would think that the Olympic committee was making gyros but that the gyro makers are benefitting off the Olympic trademark without paying sufficient money like Coke or McDonalds.
Good point. I can see more clearly the need to protect large corporate sponsors from actual competitors (e.g. Burger King, Pepsi) where there is likely to be a diversion of profits, but in this case, I am fairly certain that there will be no diversion of profits.
I suppose it could also be argued that the usage is detrimental to the trade mark but that point is undermined by the approved association with fast food/sugary drinks.