Hacker News

But I don’t want to have to dispel wrong notions because of a dumb bot using statistics to pretend to speak.

Perfect conclusion to a piece of text addressing the main issue with practical usage of LLMs: they get things wrong so often that you can't rely on them, leading you to a state that is worse than a simple "I don't know" - the "I believe that I know, but actually I don't know".

From the comments in ~~Techddit~~ Hacker News. Content warning: I'll quote rampant stupidity. Specially given HN worships this sort of bot.

[You should take down the documentation entirely, if you want to prevent incorrect interpretations of things.]¹ [The LLMs won’t be the ones emailing you]², [the people who would get things wrong if the LLM provided some kind of confident wrong answer would probably simply not read your documentation as the vast majority of users do not]³.

1. That's a bloody strawman. Howard's claim is not that he wants to get rid of all incorrect interpretation; it boils down to GPT increasing his support workload, instead of decreasing it.
2. No shit Sherlock.
3. False dichotomy. Here's a third possibility: user asks ChatGPT about the software, ChatGPT feeds the user wrong info, user realises "it doesn't fucking work!" and contacts dev, dev tells user to RTFD, user does it while contextualising it around the wrong info that ChatGPT fed him, issue doesn't go away, user keeps nagging dev.

[This is such a technopurist take. People who use LLM’s already know they can give wrong information.]¹ [Your documentation won’t be able to cover every single possible contextual scenario that an LLM [SIC - a bot using that LLM] can help with.]²

1. They do. But knowing what is wrong often depends on knowing what is correct. And if they were informed enough to say "this piece of info is wrong, this one is right...", they wouldn't be asking the bot on first place.
2. If a source of info won't help you, odds are that the bot won't help you either, even after being fed that source of info.

I do think that chatbots can be useful in some situations, mind you; just like my tarot deck. They don't give you real answers, but they give you ideas and things to think about.