Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

llama.cpp

llama.app

Post reply on HN
Print view View original

51–60 of 182 posts

Re: llama.cpp

#51

Post by chii »

pplonski86 wrote: post #36

Yesterday I installed llama.cpp to test it with local AI Data Analyst that I'm building. I was also testing other open LLM providers: Ollama, Jan, vLLM, LM Studio. I had older NVIDIA card (RTX 3070) and llama.cpp instalation was smooth, contrary to vLLM which required me to reinstall CUDA drivers because by default it installed the latest one. I'm curious if there is a speed difference between the same open LLM model…

> I had older NVIDIA card (RTX 3070) and llama.cpp instalation was smooth

what model was it that you were able to run with the rtx 3070?

Re: llama.cpp

#52

Post by thecopy »

thebeardisred wrote: post #28
thecopy wrote: post #15

Earlier quoted context omitted.

>Anything that suggests curl into bash just plain sketches me out. How is it different than trusting any other method of installation? If URL has https and is from an author you trust i dont see the difference.

Package management provides cryptographic attestation over the entire process, including the scripts run. Nothing is arbitrary nor mutable in the default state. The files touched will be in a predictable place and the scope of privilege escalation is clearly defined.

>Nothing is arbitrary nor mutable in the default state.

What do you mean with this?

Re: llama.cpp

#53

Post by danielrmay »

thecopy wrote: post #15
walrus01 wrote: post #9

Anything that suggests curl into bash just plain sketches me out. (edit: I know, this isn't totally rational, it just seems weird to me. We download and trust a lot of software and run code from a bunch of package repositories as a regular activity...). Git clone llama.cpp and build it, it's not hard. https://github.com/ggml-org/llama.cpp/blob/master/docs/build... literally just a few steps for the basics: git clone…

>Anything that suggests curl into bash just plain sketches me out. How is it different than trusting any other method of installation? If URL has https and is from an author you trust i dont see the difference.

Rendered text cannot be assumed to equal the underlying text, unfortunately

Re: llama.cpp

#54

Post by halyconWays »

llama.cpp is like the ffmepg of AI, and one of the reasons I so greatly dislike ollama is that the latter completely obfuscates that they're a rebrand of the former. Georgi Gerganov and team did all the hard work; ollama is langchain-like VC-bait with a HF download wrapper.

Re: llama.cpp

#55

Post by freehorse »

walrus01 wrote: post #9

Anything that suggests curl into bash just plain sketches me out. (edit: I know, this isn't totally rational, it just seems weird to me. We download and trust a lot of software and run code from a bunch of package repositories as a regular activity...). Git clone llama.cpp and build it, it's not hard. https://github.com/ggml-org/llama.cpp/blob/master/docs/build... literally just a few steps for the basics: git clone…

You can also install it through homebrew https://formulae.brew.sh/formula/llama.cpp

Re: llama.cpp

#56

Post by cybrox »

danielrmay wrote: post #53
thecopy wrote: post #15

Earlier quoted context omitted.

>Anything that suggests curl into bash just plain sketches me out. How is it different than trusting any other method of installation? If URL has https and is from an author you trust i dont see the difference.

Rendered text cannot be assumed to equal the underlying text, unfortunately

Download and inspect it.

Re: llama.cpp

#57

Post by imrehg »

etdznots wrote: post #45
imrehg wrote: post #24

llama.cpp works pretty well for me on the Framework 13 laptop, but the current era of "move fast, break things, rarely fix" (sorry, that's how it feels), bites here quite a bit. Two examples: - https://github.com/ggml-org/llama.cpp/pull/25863 Someone's few lines change broke the native (ROCm) support for the AMD GPU inside Framework (and other integrated systems), and any rollback or proper fix is pending for almost…

The first one multiple contributors highlighted the PR as urgent andits had lots of review but it appears to be waiting for another review and/or someone that owns the affected hardware to test that the PR fixes the issue, it wpuld be easy for you to test and report whether or not it does, and the second thing is not related to llama.cpp at all Yes ideally there would be testing every hardware + software combo but th…

I did report my test results on the first one. :)

The second I didn't say it's any of llama.cpp's "fault", but it is _related_ to llama.cpp since it's being shipped in another system, aye?

Can't stick to the old hash either, because older version have different bugs. E.g. on older versions the same Qwen3.6 model reliably fails to call specific tools due to template issues, while just having the newer llama.cpp version has that fixed. So different versions - different bugs, rather than no bugs.

Why the beating you are trying to gimme, mate? :)

Re: llama.cpp

#58

Post by numpad0 »

dlcarrier wrote: post #5

I tried to run in on my Arc A770, but all of the binary releases I could find were compiled without OpenVINO support enabled. I tried compiling it myself, but after two days of the compiler running it failed.

I don't know how it works, but does Vulkan not work on Arc?

Re: llama.cpp

#59

Post by thecopy »

danielrmay wrote: post #53
thecopy wrote: post #15

Earlier quoted context omitted.

>Anything that suggests curl into bash just plain sketches me out. How is it different than trusting any other method of installation? If URL has https and is from an author you trust i dont see the difference.

Rendered text cannot be assumed to equal the underlying text, unfortunately

How so? As i understand your point, this would mean we cannot trust GitHub enough to return the same content in git clone vs curl?

Re: llama.cpp

#60

Post by larodi »

This site seems scam for not noting origins of llama.cpp and fails to quickly and clearly communicate it NOT being affiliated with GGML org.
Post reply on HN