241–250 of 302 posts
Earlier quoted context omitted.
when I try to do this (MBP M1 Max, Sonoma) I get 'killed' immediately
Yep, same issue. and the error message is unhelpful
I think the best way to try this out is with LLaVA, the text+image model (like GPT-4 Vision). Here are steps to do that on macOS (which should work the same on other platforms too, I haven't tried that yet though): 1. Download the 4.26GB llamafile-server-0.1-llava-v1.5-7b-q4 file from https://huggingface.co/jartine/llava-v1.5-7B-GGUF/blob/main/... : wget https://huggingface.co/jartine/llava-v1.5-7B-GGUF/resolve/main/…
I just tried asking it a question: > User: What is the third planet from the sun? > Llama: The third planet from the sun is called Mars.
Earlier quoted context omitted.
Source on the newer GPT-4 model being worse at coding?
It’s not that it’s worse, it’s just refusing to do coding without persistent prodding and the right prompts. Some think they are trying to do something with alignment, and maybe prevent it from giving code away so that they can upsell.
If ChatGPT is using this model then it's more reasonable to assume that they are bleeding money and need to cut costs.
People really need to stop asking ChatGPT to write out complete programs in a single prompt.
Earlier quoted context omitted.
Got this: Terminating on uncaught SIGILL.
I'm assuming you're on Apple Silicon? Please follow https://github.com/Mozilla-Ocho/llamafile/issues/11 which is tracking this. We've received multiple reports even though I personally haven't figured out how to reproduce it yet.
Earlier quoted context omitted.
What is the point of wrapping absolutely portable single-file program into a Docker container, honest question? Looks like cargo cult for me.
Deploy your models into a Kubernetes cluster and let them fight for resources to death? A modern Roman Gladiators circus with Models?
Earlier quoted context omitted.
I'm assuming you're on Apple Silicon? Please follow https://github.com/Mozilla-Ocho/llamafile/issues/11 which is tracking this. We've received multiple reports even though I personally haven't figured out how to reproduce it yet.
No just old i5-2500k 16GB Ram and vega 56 GPU 8GB Vram.
Earlier quoted context omitted.
I am currently tinkering with this all, you can download a 3b parameter model and run it on your phone. Of course it isn't that great, but I had a 3b param model[1] on my potato computer (a mid ryzen cpu with onboard graphics) that does surprisingly well on benchmarks and my experience has been pretty good with it. Of course, more interesting things happen when you get to 32b and the 70b param models, which will requ…
That's a nice model that fits comfortably on Raspberry Pi. It's also only a few days old! I've just finished cherry-picking the StableLM support from the llama.cpp project upstream that you'll need in order to run these weights using llamafile. Enjoy! https://github.com/Mozilla-Ocho/llamafile/commit/865462fc465...
Earlier quoted context omitted.
I get that, my issue is when the model is coupled with the app, or the app just presumes I don't have it downloaded and doesn't ask me otherwise. This is like basic configuration stuff... What I suspect is happening is that people are cargo-culting zero-click installations. It seems rather fashionable right now.
In the README of the project (the TFA of this whole thread) there is the option to download the app without the model: "You can also also download just the llamafile software (without any weights included) from our releases page, or directly in your terminal or command prompt" There is no cargo-culting going on. Some of us do legitimately appreciate it.
Earlier quoted context omitted.
I get that, my issue is when the model is coupled with the app, or the app just presumes I don't have it downloaded and doesn't ask me otherwise. This is like basic configuration stuff... What I suspect is happening is that people are cargo-culting zero-click installations. It seems rather fashionable right now.
Is this the sentiment around? Is having everything normalized in your system that worth it? I would say having (some) duplicates in your system is mostly fine, better that having some spooky-action-at-a-distance break things when you don't expect. I expect the future is something like Windows's WinSxS, NixOS's /nix/store, pnpm's .pnpm-store where that deduping isn't "online" but it still is somewhat automated and hid…
Yes? It's right here, at the least.
And if that's the future, then the future sucks. We can teach people to be smarter, but no, instead our software has to bend over backwards to blow smoke up our ass because grandma.