They tried to set it up like how Grok checks what Musk's opinion is on a topic before answering, but it's impossible with mister-word-salad
Fuck AI
"We did it, Patrick! We made a technological breakthrough!"
A place for all those who loathe AI to discuss things, post articles, and ridicule the AI hype. Proud supporter of working people. And proud booer of SXSW 2024.
In general, one of the secrets about LLMs is that the intelligence they're displaying is largely in the mind of the person interacting with it and the person watching and interpreting. If you ever watch one of them "on its own" trying to reason its way through a problem without help, it becomes painfully clear.
In this case, there is no intelligence in the source material for them to fall back on, so they don't have the cheat available. It's actually a really instructive demo.
That's why LLMs require operator to already know the answers or can figure out when output is wrong for it to be of any practical use.