Laughs in ddr4....
this post was submitted on 05 Nov 2025
445 points (99.1% liked)
Technology
76618 readers
2382 users here now
This is a most excellent place for technology news and articles.
Our Rules
- Follow the lemmy.world rules.
- Only tech related news or articles.
- Be excellent to each other!
- Mod approved content bots can post up to 10 articles per day.
- Threads asking for personal tech support may be deleted.
- Politics threads may be removed.
- No memes allowed as posts, OK to post as comments.
- Only approved bots from the list below, this includes using AI responses and summaries. To ask if your bot can be added please contact a mod.
- Check for duplicates before posting, duplicates may be removed
- Accounts 7 days and younger will have their posts automatically removed.
Approved Bots
founded 2 years ago
MODERATORS
People buying RAM: oh no, what do we do?
People buying GPUs: first time?
Those are the same people.
What?
You think this is the first ram crunch?
It's not even the first one in this decade...
Well, I was aware RAM prices fluctuate.
I've never been so unfortunate when buying larger RAM, or building a new system with a new DDR version.
They don't real fluctuate, it's more oscillating.
Sometimes it's "normal" but due to a wide range of issues that can change quickly and stay that way for months because everyone waits till prices go down. So as they go down, people stop waiting,
I just got a 2x64GB 6000 kit before its price skyrocketed by like $130. I saw other kits going up, but had no clue I timed it so well.
...Also, why does "AI" need so much CPU RAM?
In actual server deployments, pretty much all inference work is done in VRAM (read: HBM/GDDR); they could get by with almost no system RAM. And honestly most businesses are too dumb to train anything that extensively. ASICs that would use, say, LPDDR are super rare, and stuff like Hybrid/IGP inference is the realm of a few random folks with homelabs... Like me.
I think 'AI' might be an overly broad term for general server buildout.
Same memory production capacity can be allocated to ddr5 or to hbm and openai signed contracts with sk hynix and samsung, the two largest ram manufacturers in the world, and bought a significant percentage of next year's production.
DDR5 prices started spiking as that deals impact propagated through the supply chain. I bought a 2x32 6800 Cl30 kit for 195 euro 12 days ago. It was 330 euro 4 days later.
...Is it that interchangeable?
TBH I know little of memory fabs and HBM ICs, but I know (say) TSMC can't just switch from a power-optimized process to a high frequency one at the drop of a hat.
Slightly different part, same process. The bigger bottleneck is packaging - HBM is 3d stacked.
Ah. Yeah. And its on the fab to do that.
I always though it'd be cool for CPUs to switch to packaged RAM, too. Samsung apparently tried to do it with Wide I/O for mobile ARM stuff, but it never caught on.
load more comments
(2 replies)
There was a recentish model, qwen next that was advertised as smth that can be run entirely on RAM.
They can ALL be run on RAM, theoretically. I bought 128GB so I can run GLM 4.5 with the experts offloaded to CPU, with a custom trellis/K quant mix; but this is a 'personal use' tinkerer setup basically no one but hobbyists will touch.
Qwen Next is good at that because its very low active parameter.
...But they aren't actually deployed that way. They're basically always deployed on cloud GPU boxes that serve dozens/hundreds of people at once, in parallel.
AFAIK the only major model actually developed for CPU inference is one of the esoteric Gemma releases, aimed at mobile. And the bitnet experiments, which aren't very big so far.
(In case it's not obvious, this is my special interest, and I'm happy to ramble on about how to set up 'niche gaming rig hybrid models' for anyone interested).
I for one would enjoy triggering your unskippable cutscenes in setting up local CPU based AI if it can work on Linux with an older amd card.
Don't have funds for anything fancy, but would be interesting in playing around with it. Been wanting to get something like that setup for home assistant.
Plenty of folks do AMD. A popular homelabsetup is 32GB AMD MI50 GPUs, which are quite cheap on eBay. Even Intel is fine these days!
But what's your setup, precisely? CPU, RAM, and GPU.
I have a MI50/7900xtx gaming/ai setup at homr which in i use for learning and to test out different models. Happy to answer questions
You can use Vulkan fairly easily as long as you have 8G vram
https://blog.linux-ng.de/2025/09/27/running-llms-with-llama-cpp-using-vulkan/
The key is which model, and how.
For the really sparse MoEs, you might be better off trying ik_llama.cpp, especially if you are targeting a 'small' quant. But the dense Gemma models (as good as they are) are probably not the best choice for 8G RAM these days.
load more comments
(1 replies)
load more comments
(2 replies)
OpenAI’s “Stargate” project has recently signed an agreement with Samsung and SK hynix for up to 900,000 wafers of DRAM per month. That figure alone would account for close to 40% of global DRAM output.
High-density NAND products are effectively sold out months in advance. Samsung’s next-generation V9 NAND is already nearly booked before it's even launched. Micron has presold almost all of its High Bandwidth Memory (HBM) output through 2026. Contracts that once covered a quarter now span years, with hyperscalers buying directly at the source.
If China's going to compete on AI, it's going to be doing so with a limited supply of memory, I expect.
They're going all in on domestic chip manufacturing, and they are catching up much faster than armchair generals, and even actual generals, predicted.
Old computer blew up, had to buy a new one. Nice 128gb ddr5. Just mobo, mem, cpu. Cpu is a rhyzen 9 9700
The memory was well over 40% of the cost, wtf?
Uhm what exactly do you need 128GB for? xD
Probably VMs.
not OP, but I guess that's futureproofing, 128GB RAM should be more than enough for a long time
load more comments
(1 replies)
What are the odds SAltman and the circular investment crew are pivoting to a pump and dump scheme on server hardware (as well as just GPGPUs) after finally admitting to themselves LLMs have hit a wall not even trillions of dollars will fix (architecture failure, no AGI for you, no getting away from those pesky workers like promised). If everything pans out nicely the world will end up with a bunch of new chip fabs (a real bottleneck) and nothing to use them for, cheap computers for all. Probably won't, instead we'll get a geopolitical shitshow as China (and hopefully Europe) will have functional, actually producing valuable stuff, economies. What happens to the US is left as an exercise for the reader.
I bought 16gb of ddr4 for 110eu back in 2018. Welcome back to the DDR wars, with NAND soon to follow.
Not sure if it's exactly the same kit but you can get that for £300 in the UK.
Wonder if the guy in the article is in the US. Tariffs would probably explain the extra $200 price difference.