Welcome to LocalLLaMA! Here we discuss running and developing machine learning models at home. Lets explore cutting edge open source neural network technology together.
Get support from the community! Ask questions, share prompts, discuss benchmarks, get hyped at the latest and greatest model releases! Enjoy talking about our awesome hobby.
As ambassadors of the self-hosting machine learning community, we strive to support each other and share our enthusiasm in a positive constructive way.
Rules:
Rule 1 - No harassment or personal character attacks of community members. I.E no namecalling, no generalizing entire groups of people that make up our community, no baseless personal insults.
Rule 2 - No comparing artificial intelligence/machine learning models to cryptocurrency. I.E no comparing the usefulness of models to that of NFTs, no comparing the resource usage required to train a model is anything close to maintaining a blockchain/ mining for crypto, no implying its just a fad/bubble that will leave people with nothing of value when it burst.
Rule 3 - No comparing artificial intelligence/machine learning to simple text prediction algorithms. I.E statements such as "llms are basically just simple text predictions like what your phone keyboard autocorrect uses, and they're still using the same algorithms since <over 10 years ago>.
Rule 4 - No implying that models are devoid of purpose or potential for enriching peoples lives.
From the Twitter post
New StarCoder coding model from @WizardLM_AI
"WizardCoder-15B-v1.0 model achieves 57.3 pass@1 on the HumanEval Benchmarks .. 22.3 points higher than the SOTA open-source Code LLMs."
My quants: https://huggingface.co/TheBloke/WizardCoder-15B-1.0-GGML https://huggingface.co/TheBloke/WizardCoder-15B-1.0-GPTQ
Original: huggingface.co WizardLM/WizardCoder-15B-V1.0 · Hugging Face
11:21 AM · Jun 14, 2023
On The Bloke's hugging face repo, it says the GGML quants are not compatible with llama.cpp, anyone know why?
It's a different type of model. llama.cpp only supports LLaMA models while GGML (the machine learning library llama.cpp is based on) has examples of various models with different architectures. WizardCoder, MPT, Bloom, probably very soon Falcon. Also some separate projects use GGML to support other models (including some of the ones I listed). For example the Rust "llm" project can support LLaMA models, MPT, BLOOM.
Looks like gpt4all supports it, thought it was based on llama for some reason going to have to give it a try
It looks like a frontend that just bundles a bunch of stuff together. Oobabooga's webui thing is similar: you can run stuff with llama.cpp, GPTQ, etc. What models and features are supported is going to depend on how the frontend manages that stuff. There are also forks of llama.cpp like koboldc++ which may support different models/features/formats (I know koboldc++ supports some older GGML file formats that llama.cpp broke compatibility with).
Oh wait does ooba support this? Nvm then I'm enjoying using that, I'm just a little lost sometimes haha
I don't know if it does or doesn't, I was just saying those two projects seemed similar: presenting a frontend for running inference on models while the user doesn't necessarily have to know/care what backend is used.
Gotcha, koboldcpp seems to be able to run it, all of it is only a tiny bit confusing :D
So if I understand correctly it is fine tuned for coding or what exactly is this Wizard model doing?
It is StarCoder and fine tuned on a new Wizard Instruct dataset optimized for coding models. So it follows the instruct formatting of prompts on top of the StarCoder base model.
That sounds great honestly! Does that work with the newest ggml yet?
Doesn’t look like it, hopefully it does someday. I am stoked to try this one out.