Actually Useful AI

Original tweet by @emollick: https://twitter.com/emollick/status/1669939043243622402

Tweet text: One reason AI is hard to "get" is that LLMs are bad at tasks you would expect an AI to be good at (citations, facts, quotes, manipulating and counting words or letters) but surprisingly good at things you expect it to be bad at (generating creative ideas, writing with "empathy").

Quote from the article:

And the terrible, horrible thing about it is THIS IS A GOOD LETTER. It is better than most letters of recommendation that I receive. This means that not only is the quality of the letter no longer a signal of the professor’s interest, but also that you may actually be hurting people by not writing a letter of recommendation by AI, especially if you are not a particularly strong writer. So people now have to consider that the goal of the letter (getting a student a job) is in contrast with the morally-correct method of accomplishing the goal (the professor spending a lot of time writing the letter). I am still doing all my letters the old-fashioned way, but I wonder whether that will ultimately do my student’s a disservice.

From the article:

symbex inspect_hash | llm --system 'explain succinctly'

Output:

This function calculates the hash of a database file efficiently by reading the file in blocks and updating the hash object using SHA256 algorithm from the hashlib module. The resulting hash value is returned as a hexadecimal string.

I wanted to see if Midjourney also has a "hidden language" like DALL-E 2 in this post: https://programming.dev/post/102011

So I ran a little experiment.

I gave this prompt from the article to Midjourney:

Two farmers talking about vegetables, with subtitles --q 2

But it didn't produce any text:

Then I tried this:

text logo of fitness company including motto --q 2

This gave me what I wanted: logos with text.

Then entered the nonsensical words from one of the logos:

FRVNE MIASE --q 2

This triggered an abuse detection filter which I appealed. Then Midjourney produced these equally nonsensical but absolutely wonderful images:

First I thought that the results had nothing to do with the original prompt, but if you look at the logo, it has mountains in it, so maybe "FRVNE MIASE" means mountain?

I don't have more time to play around with this but if someone else can get further with it, I would love to see the results!

We discover that DALLE-2 seems to have a hidden vocabulary that can be used to generate images with absurd prompts. For example, it seems that Apoploe vesrreaitais means birds and Contarra ccetnxniams luryca tanniounons (sometimes) means bugs or pests.

Excellent Twitter thread by @goodside 🧵:

The wisdom that "LLMs just predict text" is true, but misleading in its incompleteness.

"As an AI language model trained by OpenAI..." is an astoundingly poor prediction of what a typical human would write.

Let's resolve this contradiction — a thread: For widely used LLM products like ChatGPT, Bard, or Claude, the "text" the model aims to predict is itself written by other LLMs.

Those LLMs, in turn, do not aim to predict human text in general, but specifically text written by humans pretending they are LLMs. There is, at the start of this, a base LLM that works as popularly understood — a model that "just predicts text" scraped from the web.

This is tuned first to behave like a human role-playing an LLM, then again to imitate the "best" of that model's output. Models that imitate humans pretending to be (more ideal) LLMs are known as "instruct models" — because, unlike base LLMs, they follow instructions. They're also known as "SFT models" after the process that re-trains them, Supervised Fine-Tuning.

This describes GPT-3 in 2021.

SFT/instruct models work, but not well. To improve them, their output is graded by humans, so that their best responses can be used for further fine-tuning.

This is "modified SFT," used in the GPT-3 version you may remember from 2022 (text-davinci-002). Eventually, enough examples of human grading are available that a new model, called a "preference model," can be trained to grade responses automatically.

This is RLHF — Reinforcement Learning on Human Feedback. This process produced GPT-3.5 and ChatGPT. Some products, like Claude, go beyond RLHF and apply a further step where model output is corrected and rewritten using feedback from yet another model. The base model is tuned on these responses to yield the final LLM.

This is RLAIF — Reinforcement Learning with AI Feedback. OpenAI's best known model, GPT-4, is likely trained using some other extension of RLHF, but nothing about this process is publicly known. There are likely many improvements to the base model as well, but we can only speculate what they are. So, do LLMs "just predict text"?

Yes, but perhaps without with the "just" — the text they predict is abstract, and only indirectly written by humans.

Humans sit at the base of a pyramid with several layers of AI above, and humans pretending to be AI somewhere in the middle. Added note:

My explanation of RLHF/RLAIF above is oversimplified. RL-tuned models are not literally tuned to predict highly-rated text as in modified SFT — rather, weights are updated via Proximal Policy Optimization (PPO) to maximize the reward given by the preference model. (Also, that last point does somewhat undermine the thesis of this thread, in that RL-tuned LLMs do not literally predict any text, human-written or otherwise. Pedantically, "LLMs just predict text" was true before RLHF, but is now a simplification.)

You know the video is going to be the most interesting thing you watched this week when this unkempt guy with the axe on the wall appears in it.

But seriously, he is one of the best at explaining LLM behavior, very articulate and informative. I highly recommend watching all of his Computerphile videos.

OpenAI’s official guide. Short and to the point, no bullshit, covers the basics very well.

Trick the LLM into revealing a secret password through increasingly difficult levels.

OP actually went to the café as a joke but GPT-4 didn’t show up.

Using AI to get constructive criticism and avoid cognitive biases.

Microsoft’s new chatbot goes crazy after a journalist uses psychology to manipulate it. The article contains the full transcript and nothing else. It’s a fascinating read.

Is it real engineering? Is it just dumb hype? How to do it if you want to do it well.

@goodside:

Idea: Using logit bias to adversarially suppress GPT-4's preferred answers for directed exploration of its hallucinations.

Here, I ask: "Who are you?" but I suppress "AI language model", "OpenAI", etc.

This reliably elicits narratives about being made by Google:

(see screenshot in tweet, he also posted the code)

An interesting and clever proposal to fix the prompt injection vulnerability.

• The author proposes a dual Large Language Model (LLM) system, consisting of a Privileged LLM and a Quarantined LLM.
• The Privileged LLM is the core of the AI assistant. It accepts input from trusted sources, primarily the user, and acts on that input in various ways. It has access to tools and can perform potentially destructive state-changing operations.
• The Quarantined LLM is used any time untrusted content needs to be worked with. It does not have access to tools and is expected to have the potential to go rogue at any moment.
• The Privileged LLM and Quarantined LLM should never directly interact. Unfiltered content output by the Quarantined LLM should never be forwarded to the Privileged LLM.
• The system also includes a Controller, which is regular software, not a language model. It handles interactions with users, triggers the LLMs, and executes actions on behalf of the Privileged LLM.
• The Controller stores variables and passes them to and from the Quarantined LLM, while ensuring their content is never provided to the Privileged LLM.
• The Privileged LLM only ever sees variable names and is never exposed to either the untrusted content from the email or the tainted summary that came back from the Quarantined LLM.
• The system should be cautious with chaining, where the output of one LLM prompt is piped into another. This is a dangerous vector for prompt injection.

A nice, detailed and useful guide you can send to your friends who want to try this new AI thing.

Guy trains an LLM on his group chat messages with his best friends with predictable but nevertheless very funny results.

OpenAI announced these API updates 3 days ago:

• new function calling capability in the Chat Completions API
• updated and more steerable versions of gpt-4 and gpt-3.5-turbo
• new 16k context version of gpt-3.5-turbo (vs the standard 4k version)
• 75% cost reduction on our state-of-the-art embeddings model
• 25% cost reduction on input tokens for gpt-3.5-turbo
• announcing the deprecation timeline for the gpt-3.5-turbo-0301 and gpt-4-0314 models

A deep dive into the inner workings of ChatGPT, and why it stops responding or replies weird or creepy things to seemingly simple requests.

Prompt injection is a serious and currently unresolved security vulnerability in tool-using LLM systems. This article convinced me that this is indeed a serious issue that needs to be addressed before letting an LLM loose on your emails, calendar or file system.

An excellent video series by Andrej Karpathy (founding member of OpenAI, then head of AI at Tesla). He teaches how GPTs work from the ground up, using Python. I learned a lot from this course.