I think you're unreasonably pessimistic in the short term, and unreasonably optimistic in the long term.
People are getting benefit from these conversations. I know people who have uploaded chat exchanges and asked an LLM for help understanding patterns and subtext to get a better idea of what the other person is really saying - maybe more about what they're really like.
Human relationship problems tend to be quite generic and non-unique, so in fact the averageness of LLMs becomes more of a strength than a weakness. It's really very rare for people to have emotional or relationship issues that no one else has experienced before.
The problem is more that if this became common OpenAI could use the tool for mass behaviour modification and manipulation. ChatGPT could easily be given a subtle bias towards some belief system or ideology, and persuaded to subtly attack competing systems.
This could be too subtle to notice, while still having huge behavioural and psychological effects on entire demographics.
We have the media doing this already. Especially social media.
But LLMs can make it far more personal, which means conversations are far more likely to have an effect.
People are getting benefit from these conversations. I know people who have uploaded chat exchanges and asked an LLM for help understanding patterns and subtext to get a better idea of what the other person is really saying - maybe more about what they're really like.
Human relationship problems tend to be quite generic and non-unique, so in fact the averageness of LLMs becomes more of a strength than a weakness. It's really very rare for people to have emotional or relationship issues that no one else has experienced before.
The problem is more that if this became common OpenAI could use the tool for mass behaviour modification and manipulation. ChatGPT could easily be given a subtle bias towards some belief system or ideology, and persuaded to subtly attack competing systems.
This could be too subtle to notice, while still having huge behavioural and psychological effects on entire demographics.
We have the media doing this already. Especially social media.
But LLMs can make it far more personal, which means conversations are far more likely to have an effect.