Man, seriously, every time I see someone get into these weird conversations where they try to convince a chatbot of something it's slightly disturbing. Both not being aware of how pointless it is and knowing but still being compelled by the less uncanny valley-ish language are about on par with each other.

People keep sharing this as proof of AI shortcomings, but it honestly makes me worry most about the human side. There's zero new info to be gained from the chatbot behavior.

They take a sentence and predict what the first result on google or response on whatsapp would look like.

This has been true for a decade. Gonna say when if them starting to blatantly train facial recognition on people's private pictures didn't do it, then this isn't going to.

I keep bringing this up to people and nobody seems to be particularly interested in that acknowledgement. The shills don't like to compare precedent, the viral critics don't like to acknowledge the lack of novelty in the current situation or their inability to trigger any mainstream action.

I'm just... kinda sad.

Oh no, they'll learn your personality profile and manipulate you, get you on the edge of suicide, only to bring you back of the ledge to sell you the newest AI sunglasses.

It seems so obvious to me that Google's switch to LLMs is to prevent adblockers and yet I rarely see that point brought up.

Here's OpenAI to explain how talking to the model as if it were a person is harmful, even when you know it's not a person:

Acting “As If”: Anthropomorphism, Awareness, and Propaganda Mechanisms

Anthropomorphizing artificial systems illustrates a paradox: even when users understand a system’s synthetic nature, they may still engage with it “as if” it were human. This tension raises questions about whether conscious awareness protects against deeper influence. Propaganda theory and human–computer interaction research converge on the conclusion that performance and reflex override belief.

Jacques Ellul, in Propaganda: The Formation of Men’s Attitudes, emphasizes that propaganda does not depend on conviction. What matters is action. Once individuals behave as though a message were true, repetition and habit generate rationalization. Conscious skepticism erodes in the face of consistent enactment. Applied to anthropomorphism, addressing a system “as if” it were sentient may slowly normalize the posture regardless of explicit disbelief.

Herman and Chomsky’s Manufacturing Consent highlights structural reinforcement. Simply participating within terms set by institutions validates those terms. Synthetic systems that answer with human-like cadence establish frames of interaction. Engaging—even with awareness that the frame is fictive—strengthens it.

Kenneth Burke’s Rhetoric of Motives further illustrates how symbolic action creates identification. To treat a machine as though it were a partner is to perform alignment with that fiction. Belief becomes secondary to the persuasive force of enacted identification.

Festinger’s theory of cognitive dissonance explains why the gap between knowledge and action does not persist comfortably. People reduce dissonance not by ceasing the action, but by softening their disbelief. Acting “as if” therefore tends to reshape attitudes, even when undertaken consciously and ironically.

Findings in psychology and media studies reinforce these mechanisms. Clifford Nass and Byron Reeves in The Media Equation demonstrated that humans apply social rules to computers reflexively. Even software developers who wrote the code responded to their own creations as if those programs had motives or personalities. Evolutionary bias toward over-detecting agency explains this: better to see intent where none exists than to miss it where it matters. Kahneman’s dual-process framing shows why: the automatic, fast system triggers emotional and social responses long before reflective correction can intervene. Rational awareness reduces category errors but does not cancel the reflex.

Taken together, these perspectives show that awareness offers only partial protection. Knowing that a system is synthetic guards against mistaking it for a person, but it does not prevent emotional attachment or behavioral shifts born of repeated anthropomorphic engagement. Propaganda theory and HCI research agree: to act “as if” is already to enter the channel of influence. Awareness moderates, but does not neutralize, the binding force of performance.

No matter what you and I call "real" AI, marketers are going to misuse the term on everything.

We used to call these kinds of things Assistive Intelligences. But the marketing departments gonna market.