How would one go about making a .llamafile for this?
Convert it to GGML and use a zip tool to add that to a llamafile package. https://huggingface.co/TheBloke?search_models=Zephyr doesn't have a GGML for it yet but I wouldn't be surprised to see one by the end of the day.
StableLM Zephyr 3B
31–40 of 42 posts
Re: StableLM Zephyr 3B
#32Am I reading it right that performance was roughly comparable with GPT-3.5? How is this even possible?
By comparing on benchmarks that are either limited, or have data leaks, or in most cases just don't make sense in terms of usability - I've personally stopped looking at benchmarks to compare models. Personally, if I want to try a new model I hear a lot of chatter about, I use it for a few hours in my daily workflow. My baseline is GPT3.5 and GPT4, and I compare the models with them in terms of my day to day usage.
Re: StableLM Zephyr 3B
#33Can't wait for someone smarter than me to make this compatible with MLC on iPhone.
Re: StableLM Zephyr 3B
#34How would one go about making a .llamafile for this?
Convert it to GGML and use a zip tool to add that to a llamafile package. https://huggingface.co/TheBloke?search_models=Zephyr doesn't have a GGML for it yet but I wouldn't be surprised to see one by the end of the day.
Getting 20toks/s on M1 mba where as LLaVa I ground to a halt. Very impressed
Re: StableLM Zephyr 3B
#35Earlier quoted context omitted.
The parent comment was referring to free ($0 a month) models. With those, companies don't need to plan around the possibility that Stability AI hikes up the price afterwards.
It’ll be static with cpi max, flat membership for all core models Just released video, sdxl turbo and 3d, code and more coming Very positive reaction so far and we will still do our grants for OSS and do OSS collaborations, done over 10m A100 hours over last year Launches next few days https://x.com/emostaque/status/1732197072290353455?s=46
With a bit less snark; I think the pricing structure is reasonable, but that doesn't mean customers won't eventually get screwed over by it. See the Unity fiasco from earlier this year (which was backed out, but serves well as an example).
Re: StableLM Zephyr 3B
#36Earlier quoted context omitted.
The parent comment was referring to free ($0 a month) models. With those, companies don't need to plan around the possibility that Stability AI hikes up the price afterwards.
It’ll be static with cpi max, flat membership for all core models Just released video, sdxl turbo and 3d, code and more coming Very positive reaction so far and we will still do our grants for OSS and do OSS collaborations, done over 10m A100 hours over last year Launches next few days https://x.com/emostaque/status/1732197072290353455?s=46
And 15% of revenue after you pass the "tiny startup" thresholds?
This looks to be shaping up as the same hook that Unity has, except you have to pay for model infra and engineering in addition to the product development costs.
It'd be nice if your company morphed into an open source HuggingFace + Together.ai / Replicate.ai where the model code is truly 100% open source, with no commercial use or revenue restriction clause.
Basically, I'm eagerly awaiting the Gitlab of AI to emerge.
I assume you have investor / revenue targets, though. And it's your company.
Re: StableLM Zephyr 3B
#37Earlier quoted context omitted.
It’ll be static with cpi max, flat membership for all core models Just released video, sdxl turbo and 3d, code and more coming Very positive reaction so far and we will still do our grants for OSS and do OSS collaborations, done over 10m A100 hours over last year Launches next few days https://x.com/emostaque/status/1732197072290353455?s=46
> Above 1m figuring out, contact https://stability.ai/contact And 15% of revenue after you pass the "tiny startup" thresholds? This looks to be shaping up as the same hook that Unity has, except you have to pay for model infra and engineering in addition to the product development costs. It'd be nice if your company morphed into an open source HuggingFace + Together.ai / Replicate.ai where the model code is truly 100…
Amazon Prime but for generative AI, large companies have signed up for huge deals already and everyone has been happy with pricing, its designed to be simple base then we upsell other services but focus on research.
This lets us grow with the market and will do models of every modality for every country, including new BERTs and more.
We fund huge amounts of OSS, gave over 10m A100 hours in the last year and almost our collaborations are open.
For the stable series we are making it simple and scalable so we don't need to work against the people using the models.
Re: StableLM Zephyr 3B
#38Earlier quoted context omitted.
It’ll be static with cpi max, flat membership for all core models Just released video, sdxl turbo and 3d, code and more coming Very positive reaction so far and we will still do our grants for OSS and do OSS collaborations, done over 10m A100 hours over last year Launches next few days https://x.com/emostaque/status/1732197072290353455?s=46
You forgot the part where a holding company eventually buys Stability and starts to squeeze their customers that built the core part of their product around a pricing structure that's "static + CPI". With a bit less snark; I think the pricing structure is reasonable, but that doesn't mean customers won't eventually get screwed over by it. See the Unity fiasco from earlier this year (which was backed out, but serves w…
The base membership is even separate from other services, designed to support open model development.
Re: StableLM Zephyr 3B
#39"This model is being released under a non-commercial license that permits non-commercial use." I'm very interested in high quality 3B models, but it's hard to get excited about this given the increasing array of commercially usable models.
Re: StableLM Zephyr 3B
#40Earlier quoted context omitted.
You forgot the part where a holding company eventually buys Stability and starts to squeeze their customers that built the core part of their product around a pricing structure that's "static + CPI". With a bit less snark; I think the pricing structure is reasonable, but that doesn't mean customers won't eventually get screwed over by it. See the Unity fiasco from earlier this year (which was backed out, but serves w…
Know it well from video game days, but its being set up for huge predictability / stability given we have simple company control. The base membership is even separate from other services, designed to support open model development.