Technology

cross-posted from: https://lemmy.world/post/440237

The latest update of Koboldcpp v1.32 brings significant performance boosts to AI computations at home, enabling faster generation speeds and improved memory management for several AI models like MPT, GPT-2, GPT-J and GPT-NeoX, plus upgraded K-Quant matmul kernels for OpenCL.

By leveraging GPU power via OpenCL and implementing optimized programming techniques, it allows hobbyist and enthusiast users to run these advanced models more efficiently on home hardware.

LostRuin's Koboldcpp v1.32 GitHub Patch Notes

• Ported the optimized K-Quant CUDA kernels to OpenCL ! This speeds up K-Quants generation speed by about 15% with CL (Special thanks: @0cc4m)

• Implemented basic GPU offloading for MPT, GPT-2, GPT-J and GPT-NeoX via OpenCL! It still keeps a copy of the weights in RAM, but generation speed for these models should now be much faster! (50% speedup for GPT-J, and even WizardCoder is now 30% faster for me.)

• Implemented scratch buffers for the latest versions of all non-llama architectures except RWKV (MPT, GPT-2, NeoX, GPT-J), BLAS memory usage should be much lower on average, and larger BLAS batch sizes will be usable on these models.

• Merged GPT-Tokenizer improvements for non-llama models. Support Starcoder special added tokens. Coherence for non-llama models should be improved.

• Updated Lite, pulled updates from upstream, various minor bugfixes.

To use, download and run the koboldcpp.exe, which is a one-file pyinstaller. Alternatively, drag and drop a compatible ggml model on top of the .exe, or run it and manually select the model in the popup dialog.

Once loaded, you can connect like this (or use the full koboldai client):

http://localhost:5001

For more information, be sure to run the program with the --help flag.

Want to run AI models at home? Checkout Koboldcpp on Github, an inference engine made by LostRuins, or any of the other many options you can download at home for free.

there goes the ear licking ASMR 😔

I still subscribe to YouTube TV and don’t care for an alternative, but losing yet another sports channel sucks considering I’m a big baseball fan.

I wonder if higher quality datasets are the future rather than using tons of internet scraped texts. Either way, neat model!

Hi! I just got a new computer recently, and I'm concerned about browser security and security in general. Any recommendations for a good secure browser? Preferably open source.

The FTC on Wednesday sued Amazon, alleging it tricked customers into signing up for its Prime subscription program and intentionally complicated the cancellation process.

The agency claims Amazon used so-called “dark patterns” to steer users to enroll in Prime without their consent.

cross-posted from: https://sh.itjust.works/post/299438

cross-posted from: https://sh.itjust.works/post/299388

Suzuki Motor Corp. and SkyDrive Inc. agreed to cooperate in developing a “flying car” by as early as spring 2024.

Under the deal signed on June 19, Suzuki will help recruit staff for the startup’s new subsidiary dedicated to the project.

The two companies plan to produce the small electric passenger aircraft at Suzuki’s plant in Shizuoka Prefecture.

SkyDrive on June 19 also announced a design change to its next-generation flying vehicle, increasing its passenger capacity from two to three.

SkyDrive plans to start mass-producing the aircraft in 2026 following its official debut at the 2025 Osaka Kansai Expo.

Suzuki invested in SkyDrive in September 2022, six months after the two companies started their official partnership.

The auto giant offers its manufacturing expertise to the startup in exchange for the opportunity to enter a new business field.

As the technology becomes ubiquitous, a vast tasker underclass is emerging — and not going anywhere.

This version comes with a small technical challenge that we're proud to have overcome! This new feature won't be as visible as a graphical change, but it will make hosting a PeerTube platform easier, more resilient and cheaper.

OpenAI's lobbying efforts in the European Union are centered around modifying proposed AI regulations that could impact its operations. The tech firm is notably pushing for a weakening of regulations which currently classify certain AI systems, such as OpenAI's GPT-3, as "high risk."

Altman's Stance on AI Regulation:

OpenAI CEO Sam Altman has been very vocal about the need for AI regulation. However, he is advocating for a specific kind of regulation - those favoring OpenAI and its operations.

OpenAI's White Paper:

OpenAI's lobbying efforts in the EU are revealed in a document titled "OpenAI's White Paper on the European Union's Artificial Intelligence Act." The document focuses on attempting to change certain classifications in the proposed AI Act that classify certain AI systems as "high risk."

"High Risk" AI Systems:

The European Commission's "high risk" classification includes systems that could potentially harm health, safety, fundamental rights, or the environment. The Act would require legal human oversight and transparency for such systems. OpenAI, however, argues that its systems such as GPT-3 are not "high risk," but could be used in high-risk use cases. It advocates that regulation should target companies using AI models, not those providing them.

Alignment with Other Tech Giants:

OpenAI's position mirrors that of other tech giants like Microsoft and Google. These companies also lobbied for a weakening of the EU's AI Act regulations.

Outcome of Lobbying Efforts:

The lobbying efforts were successful, as the sections that OpenAI opposed were removed from the final version of the AI Act. This success may explain why Altman reversed a previous threat to pull OpenAI out of the EU over the AI Act.

Source (Mashable)

PS: I run a ML-powered news aggregator that summarizes with an AI the best tech news from 50+ media (TheVerge, TechCrunch…). If you liked this analysis, you’ll love the content you’ll receive from this tool!

What comes around is all around