AI - Artificial intelligence

• HTML.
• PDF.

In large language model (LLM) pretraining, data quality is believed to determine model quality. In this paper, we re-examine the notion of "quality" from the perspective of pre- and post-training co-design. Specifically, we explore the possibility that pre-training on more toxic data can lead to better control in post-training, ultimately decreasing a model's output toxicity. First, we use a toy experiment to study how data composition affects the geometry of features in the representation space. Next, through controlled experiments with Olmo-1B models trained on varying ratios of clean and toxic data, we find that the concept of toxicity enjoys a less entangled linear representation as the proportion of toxic data increases. Furthermore, we show that although toxic data increases the generational toxicity of the base model, it also makes the toxicity easier to remove. Evaluations on Toxigen and Real Toxicity Prompts demonstrate that models trained on toxic data achieve a better trade-off between reducing generational toxicity and preserving general capabilities when detoxifying techniques such as inference-time intervention (ITI) are applied. Our findings suggest that, with post-training taken into account, bad data may lead to good models.

“Hardmaxxing is NECESSARY. Softmaxxing alone will NEVER mog you into viability — it’s like putting a fresh coat of paint on a crumbling building,” declares a chatbot featured prominently on OpenAI’s GPTs page. It has just analyzed a photograph of a man and deemed him “subhuman”. The page, prominently linked from the sidebar in the ChatGPT interface, lists “Looksmaxxing GPT” as #6 in the “Lifestyle” section, behind bots promising astrological analysis, color analysis, and “fictional not-real therapy”.

I find this research kind of funny.

• The ‘productivity-pay gap’ has been widening for decades.
• This disparity between rising output and sluggish wages may only grow further with the spreading use of artificial intelligence.
• ‘Increasing inequality’ was among the AI-related risks flagged in the World Economic Forum’s most recent Chief Economists Outlook.
• But thinking big picture could create and nurture new areas of (well paid) human expertise.