this post was submitted on 02 Aug 2025
274 points (98.2% liked)
Fuck AI
3642 readers
521 users here now
"We did it, Patrick! We made a technological breakthrough!"
A place for all those who loathe AI to discuss things, post articles, and ridicule the AI hype. Proud supporter of working people. And proud booer of SXSW 2024.
founded 1 year ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
Exceedingly false representation of the actual experiment.
They took Llama 3 and then trained it further on specific conditions (reinforcing it on "likes" / "thumbs up"s based on positive feedback from a simulated userbase)
And then after that the scientists found the new model (which you can't really call Llama 3 anymore, it's been trained further and it's behavior fundamentally altered) behaved like this when prior informed that the user was easily influenced by the model specifically
What is important to gather though, is the fact that when a model gets trained on the metrics of "likes", it starts to behave in a manner like this, telling the user whatever they want to hear... Which makes sense, the model is effectively getting trained to min/max positive feedback from users, rather than being trained on being right / correct
But to try and represent this as a "real" chatbot's behavior is definitely false, this was a model trained by scientists explicitly to test if this behavior happens under extreme conditioning.