Fuck AI

You can already self-host an open source LLM, and fine-tune it on custom datasets. Huggingface has thousands to choose from.

The largest you'll probably fit on consumer hardware is probably 32 billion parameters or so, and that's with quantization. Basically, at 8-bit quantization, you need 1GB RAM for every billion parameters. So a 32 billion parameter 8-bit model would need 32GB RAM, plus overhead. At 16-bit it would need 64GB RAM, and so on. A 24 billion parameter model with 16-bit quantization would take up 48GB RAM, etc.

The commercial LLMs that people pay subscriptions to use an API for tend to have like 130-200 billion parameters with no quantization (32-bit). So it wouldn't run on consumer hardware. But you honestly don't need one that big, and I think they actually suffer in quality by trying to overgeneralize.

For most people's purposes, a 14 billion parameter model with 16-bit architecture is probably fine. You just need 28GB of free RAM. Otherwise, on 14GB RAM you can do 14B params at 8-bit, or 7B at 16-bit. You might lose some accuracy, but with specialized fine-tuning and especially retrieval-augmented generation, it won't be severe.

Anything smaller than 7B might be pushing it, and likewise anything at 4-bit quantization would lose accuracy. 7B at 8-bit would also probably suffer on benchmarks. So realistically you'll probably need at least 16GB of RAM accounting for overhead. More if you want to run any concurrent processes.

The thing about making one from scratch though, is that it's resource-intensive. You can try generating a 1 billion parameter model with blank or randomized weights, the algorithm isn't a secret. But pre-training it could take weeks or months depending on your hardware. Maybe days if you have a high-end GPU. And that's with it running non-stop, so you can imagine the electric bill, and the task of keeping your system cool.

TL;DR, You can ask an LLM to vibe-code you a new model from scratch, but pre-training it you're gonna be limited by the resources you have available. You can already download pre-trained open source models for self-hosting though, and fine-tune them yourself if you desire.