Technology

Already, the research team is working on a device that could also identify influenza and RSV.

cross-posted from: https://lemmy.world/post/1535820

I'd like to share with you Petals: decentralized inference and finetuning of large language models

• https://petals.ml/
• https://research.yandex.com/blog/petals-decentralized-inference-and-finetuning-of-large-language-models

What is Petals?

Run large language models at home, BitTorrent‑style

Run large language models like LLaMA-65B, BLOOM-176B, or BLOOMZ-176B collaboratively — you load a small part of the model, then team up with people serving the other parts to run inference or fine-tuning. Single-batch inference runs at 5-6 steps/sec for LLaMA-65B and ≈ 1 step/sec for BLOOM — up to 10x faster than offloading, enough for chatbots and other interactive apps. Parallel inference reaches hundreds of tokens/sec. Beyond classic language model APIs — you can employ any fine-tuning and sampling methods, execute custom paths through the model, or see its hidden states. You get the comforts of an API with the flexibility of PyTorch.

• Colab Link
• GitHub Docs

Overview of the Approach

On a surface level, Petals works as a decentralized pipeline designed for fast inference of neural networks. It splits any given model into several blocks (or layers) that are hosted on different servers. These servers can be spread out across continents, and anybody can connect their own GPU! In turn, users can connect to this network as a client and apply the model to their data. When a client sends a request to the network, it is routed through a chain of servers that is built to minimize the total forward pass time. Upon joining the system, each server selects the most optimal set of blocks based on the current bottlenecks within the pipeline. Below, you can see an illustration of Petals for several servers and clients running different inputs for the model.

Benchmarks

We compare the performance of Petals with offloading, as it is the most popular method for using 100B+ models on local hardware. We test both single-batch inference as an interactive setting and parallel forward pass throughput for a batch processing scenario. Our experiments are run on BLOOM-176B and cover various network conditions, from a few high-speed nodes to real-world Internet links. As you can see from the table below, Petals is predictably slower than offloading in terms of throughput but 3–25x faster in terms of latency when compared in a realistic setup. This means that inference (and sometimes even finetuning) is much faster with Petals, despite the fact that we are using a distributed model instead of a local one.

Conclusion

Our work on Petals continues the line of research towards making the latest advances in deep learning more accessible for everybody. With this work, we demonstrate that it is feasible not only to train large models with volunteer computing, but to run their inference in such a setup as well. The development of Petals is an ongoing effort: it is fully open-source (hosted at https://github.com/bigscience-workshop/petals), and we would be happy to receive any feedback or contributions regarding this project!

• You can read the full article here

TL;DR from the article: This build includes a handful of new features we’re beginning to roll out to Windows Insiders in the Dev Channel including a modernized File Explorer Home and address bar, Dynamic Lighting, and support for Emoji 15. As part of introducing these new features, we’ve also added some new known issues. We’re also releasing a new Microsoft Store update.

https://www.maginative.com/article/meta-unveils-cm3leon-a-breakthrough-ai-model-for-advanced-text-to-image-generation-and-image-understanding/

I'm a former VFX artist for Robot Chicken and a few other [adultswim] shows. These are a few of my initial thoughts on AI imagery after 1 year in the can. Excuse my vtubers model, as I'm not fully condemning AI I've had to hide my identity a bit after some nasty was thrown my way online. Won't stop me from making content though. Take care!

More than 90% of arrests based on facial recognition are Black. In state of Rio de Janeiro, unjust arrests involving Black individuals reached 81%.

Contrary to Silicon Valley wisdom, training AIs on larger data sets could worsen their tendency to replicate societal biases and racist stereotypes

New study reports companion robots with artificial intelligence may one day help alleviate loneliness epidemic. Surgeon General says loneliness may be as pernicious as cigarettes.