Can we assume this is a product of the biased real world training data? Feed an LLM data that shows women (unfairly) earn less on average and you’ll get advice that they should earn less than average.
Note the 2019 date. But I’m certain I’ve seen reports earlier.
And as a sibling comment put it: “There is no unbiased training data. That's the problem.” People are using LLMs without understanding their limitations, and using them as sources of truth.
There is no unbiased training data. That's the problem.
Think about it... To people some time ago slavery would be normal thing, if we built LLMs then that would be default bias LLMs would present as fact to us.
That is a safe assumption. And it is useful to think about if you are working to improve LLMs.
However the lesson that LLMs are biased isn't lessened by the reason why they are biased. Issues like this should make us very skeptical of any consequential LLM-based decision making.
can we assume that the LLM has been trained on not just real-world data but also content that discusses things like gender/ethnic pay gaps, the causes thereof and ameliorative strategies? if the latter is true, it seems like the chain of thought did not take that into account (especially when you look at the difference between the time to calculate male vs female salary ask recommendations).
It certainly seems plausible, but I wouldn't entirely rule out other possibilities.
Do to give an example if you present the LLM with two people that are exactly the same except they have different color shirts I think it will suggest slightly different salary for one than the other for no clear reason and without any obvious bias in the training set.
I think the current state of the art consensus is that you don't want to filter training data so much as you want to fine-tune behavior out of models in the same way that you probably don't want to shelter children too much, but to explain what is right and wrong to them.
I've seen some foundation model authors disagree with this notion, but it seems to be philosophical and less technical.
Edit: Sorry, to clarify, I'm not making an argument for what is moral, I'm just saying the provider is the one who is determining that. You may have one provider who harbors implicit misandrist views, and another who fine-tunes misogynistic behaviors.
And what is right and wrong, please do tell?
Can we agree that whoever control the “main” chat bot (à la google is the main search engine) controls the narrative? Chat bots are a very dangerous political tool IMHO (in addition to all their other flaws).
I do agree that the LLM provider is controlling the narrative.
The difficult part is you can’t (it’s not responsible to) go full libertarian on this. You have to draw the line somewhere and LLM providers are tiptoeing around morals/ethics/regulations and things can change day to day as to what’s allowed for these models to output.
I agree you cannot go full libertarian either. But I’m not trusting a company with the ethics of OpenAI to do the right thing. Nor any company for that matter…
> Can we assume this is a product of the biased real world training data? Feed an LLM data that shows women (unfairly) earn less on average and you’ll get advice that they should earn less than average.
And this is the best argument to demonstrate they are not smart
