This is a most excellent place for technology news and articles.
This was bound to happen. Neural networks are inherently analog processes, simulating them digitally is massively expensive in terms of hardware and power.
Digital domain is good for exact computation, analog is better for approximate computation, as required by neural networks.
You might benefit from watching Hinton's lecture; much of it details technical reasons why digital is much much better than analog for intelligent systems
BTW that is the opposite of what he set out to prove He says the facts forced him to change his mind
Thank you for the link, it was very interesting.
Even though analogue neural networks have the drawback that you can't copy the neuron weights (currently, but tech may evolve to do it), they can still have use cases in lower powered edge devices.
I think we'll probably end up with hybrid designs, using digital for most parts except the calculations.
much of it details technical reasons why digital is much much better than analog for intelligent systems
For current LLMs there would be a massive gain in energy efficiency if analogue computing was used. Much of the current energy costs come from stimulating what effectively analogue processing on digital hardware. There's a lot lost in the conversation, or "emulation" of analogue.
I wish researchers like Hinton would stick to discussing the tech. Anytime he says anything about linguistics or human intelligence he sounds like a CS major smugly raising his hand in Phil 101 to a symphony of collective groans.
Hinton is a good computer scientist (with an infinitesimally narrow window of expertise). But the guy is philosophically illiterate.
That and the way companies have been building AI they have been doing so little to optimize compute to instead try to get the research out faster because that's what is expected in this bubble. I'm absolutely fully expecting to see future research finding plenty of ways to optimize these major models.
But also R&D has been entirely focused on digital chips I would not be at all surprised if there were performance and/or efficiency gains to be had in certain workloads by shifting to analog circuits
That's a good point. The model weights could be voltage levels instead of digital representations. Lots of audio tech uses analog for better fidelity.I also read that there's a startup using particle beams for lithography. Exciting times.
At least one Nobel Laureate has exactly the opposite opinion (see the Hinton lecture above)
what audio tech uses analog for better fidelity?
Vinyl records, analog tube amplifiers, a good pair of speakers 🤌
Honestly though digital compression now is so good it probably sounds the same.
cool
I hope Nvidia stock drops 10% so I can buy more.
Actually its so high up now, I think losing 10% isnt enough for it to look like a good buy.
it would need to drop a lot more than 10%
This is not a new line of research in the sense that this is not the only place looking in the mixed analog/digital computers. been articles on it for at least a year I think and when digital was taking over there was a lot of discussion around it being inferior to analog so I bet its been being thrown around to combine the two likely since digital became a thing.
Who is China? Why is it so smart?
West Taiwan. Because The Great Ruling Party said so.
