11–20 of 164 posts
I tried to run in on my Arc A770, but all of the binary releases I could find were compiled without OpenVINO support enabled. I tried compiling it myself, but after two days of the compiler running it failed.
Two days sounds like a lot, both llama.cpp and openvino only takes a few minutes to compile on any decent modern cpu.
Anything that suggests curl into bash just plain sketches me out. (edit: I know, this isn't totally rational, it just seems weird to me. We download and trust a lot of software and run code from a bunch of package repositories as a regular activity...). Git clone llama.cpp and build it, it's not hard. https://github.com/ggml-org/llama.cpp/blob/master/docs/build... literally just a few steps for the basics: git clone…
Yeah, 100% and it's becoming more and more of a thing, see rust install for example.
OTOH, if you're installing llama.cpp, you're more than likely planning to run an LLM on your Linux box with an agentic harness, so a curl into bash thing might be the least of your security concerns, :-)
Anything that suggests curl into bash just plain sketches me out. (edit: I know, this isn't totally rational, it just seems weird to me. We download and trust a lot of software and run code from a bunch of package repositories as a regular activity...). Git clone llama.cpp and build it, it's not hard. https://github.com/ggml-org/llama.cpp/blob/master/docs/build... literally just a few steps for the basics: git clone…
> Anything that suggests curl into bash just plain sketches me out. Yeah, 100% and it's becoming more and more of a thing, see rust install for example. OTOH, if you're installing llama.cpp, you're more than likely planning to run an LLM on your Linux box with an agentic harness, so a curl into bash thing might be the least of your security concerns, :-)
The harness gets the openai-compatible endpoint fed into it to talk to llama-server across the network, but the VM has no access whatsoever to my personal files, mail, backups/deep storage, fileserver, Documents folder, etc.
Anything that suggests curl into bash just plain sketches me out. (edit: I know, this isn't totally rational, it just seems weird to me. We download and trust a lot of software and run code from a bunch of package repositories as a regular activity...). Git clone llama.cpp and build it, it's not hard. https://github.com/ggml-org/llama.cpp/blob/master/docs/build... literally just a few steps for the basics: git clone…
How is it different than trusting any other method of installation? If URL has https and is from an author you trust i dont see the difference.
I can understand the desire for the llama.cpp project to want to own the end user relationship, it is true that previous to this they were a tool provider and not really owning the end user experience.
Anything that suggests curl into bash just plain sketches me out. (edit: I know, this isn't totally rational, it just seems weird to me. We download and trust a lot of software and run code from a bunch of package repositories as a regular activity...). Git clone llama.cpp and build it, it's not hard. https://github.com/ggml-org/llama.cpp/blob/master/docs/build... literally just a few steps for the basics: git clone…
>Anything that suggests curl into bash just plain sketches me out. How is it different than trusting any other method of installation? If URL has https and is from an author you trust i dont see the difference.
One thing I do not do as a matter of practice is install things with a ridiculous number of recursive npm dependencies.
I tried to run in on my Arc A770, but all of the binary releases I could find were compiled without OpenVINO support enabled. I tried compiling it myself, but after two days of the compiler running it failed.
Two days sounds like a lot, both llama.cpp and openvino only takes a few minutes to compile on any decent modern cpu.
I had a great experience with llama-cpp with Nvidia backend on NixOS.
(Sorry for being that guy.)
ggerganov and the team have done a stellar job maintaining the quality while still being fast to implement new models/improvements.
Anything that suggests curl into bash just plain sketches me out. (edit: I know, this isn't totally rational, it just seems weird to me. We download and trust a lot of software and run code from a bunch of package repositories as a regular activity...). Git clone llama.cpp and build it, it's not hard. https://github.com/ggml-org/llama.cpp/blob/master/docs/build... literally just a few steps for the basics: git clone…
> Anything that suggests curl into bash just plain sketches me out. Yeah, 100% and it's becoming more and more of a thing, see rust install for example. OTOH, if you're installing llama.cpp, you're more than likely planning to run an LLM on your Linux box with an agentic harness, so a curl into bash thing might be the least of your security concerns, :-)
Yeah I recently tried the coding harness that's recommended here, Pi, in a bubble wrap sandbox and was horrified to learn that it spams multiple warnings at you if you don't give it write access to its own config/extension folder... Everyone else is rawdogging it I guess.