Fuck AI

I'm on the fence, but will say that if, for whatever reason, it was never actually connected to the data or the connection had some flaw, I could totally believe it would just fabricate a report that looks consistent with what the request asked for. Maybe it failed to ever convey that an error occurred. Maybe it conveyed the lack of data and the user thought he could just tell the AI to fix the problem without trying to understand it himself and triggered it to generate a narrative consistent with fixing it without actually being able to fix it.

Sure if you do a sanity check it should fall apart, but that assumes they bother. Some people have crazy confidence in LLM and didn't even check.

Writing a syntactically correct SQL statement is not the same as doing accurate data analytics.

I am reminded of this story:

https://retractionwatch.com/2024/02/05/no-data-no-problem-undisclosed-tinkering-in-excel-behind-economics-paper/

Heshmati told the student he had used Excel’s autofill function to mend the data. He had marked anywhere from two to four observations before or after the missing values and dragged the selected cells down or up, depending on the case. The program then filled in the blanks. If the new numbers turned negative, Heshmati replaced them with the last positive value Excel had spit out.

Of course that guy didn't need fancy autofill to act like an idiot, he used good old fashion autofill.

Honestly, I was leaning toward "funny but probably fake" myself until I checked out OP's post history, which mentions "startups" and namedrops a few SaaS tools used heavily in marketing. If you've worked with marketers (or a fair few startup bros, honestly), you'll know this isn't beyond the bounds of reason for some of them 😂

The problem is you've got people using the tools that don't understand the output or the method to get there.

Take the Excel copilot function. You need to pass in a range of cells for the slop prompt to work on, but it's an optional parameter. If you don't pass that in, it returns results anyway. They're just complete bollocks.