I’m honestly pretty paranoid that this is already happening - I treat specific product recommendations from LLMs the same way I treat ones I sit up on Reddit - they could so easily simply be paid advertisements, smuggled in under the guise as organic endorsements.
It's even worse because LLM providers don't even need to be doing anything malicious for the conclusions to be garbage.
The vast majority of information that the LLM "reads" about any given product is going to come from listicles and other poorly researched "reviews", ad placements, astroturfed comments, and marketing material. They launder all of this together, "summarize" it and present it as rigorous market research. Garbage in, garbage out.
