Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

LLama.cpp now has a web interface

github.com

Post reply on HN
Print view View original

41–50 of 54 posts

Re: LLama.cpp now has a web interface

#41

Post by vitaminka »

ggerganov wrote: post #38
vitaminka wrote: post #37

Earlier quoted context omitted.

i’m curious, what’s is the approach for maintainable and decoupled various gpu backends?

It was designed in #915 (read just the OP and the linked PRs at the end) and the implementation pretty much follows it closely, at least for the Metal backend. The CUDA and OpenCL backends are currently slightly coupled in ggml as they started developing before #915, but I think we'll resolve this eventually. #915 - https://github.com/ggerganov/llama.cpp/discussions/915

interesting decoupling method, ty :)

Re: LLama.cpp now has a web interface

#42

Post by jonny_eh »

swyx wrote: post #18
Xenoamorphous wrote: post #11

Earlier quoted context omitted.

A billionare co-founder at that!

actually did tobi even have any other cofounders? i just realized i only ever hear about him and harvey finkelstein edit: google says scott lake was actually the founding CEO. TIL! https://www.linkedin.com/mwlite/profile/in/scottlake?origina...

> harvey finkelstein

Harley Finkelstein, he's a nice guy. Not a co-founder but very important to the history and running of Shopify.

Re: LLama.cpp now has a web interface

#45

Post by eclectic29 »

xal wrote: post #40
eclectic29 wrote: post #36

Can someone shed light on how does a CEO with 3 kids get time to hack on something like this? Some might even argue that all the time spent doing this would’ve been better spent on CEO activities, but thankfully this is HN and people have hobbies, so that’s that, but this can be a very time consuming hobby.

I've tried to eliminate language around being "too busy" from my vocabulary and attempt to replace it with "can't prioritize". That's sometimes a bit awkward, but really trains better habits. Sometimes I prioritize hobbies if I feel I need it, and doing this seemed fun and useful besides!

Thanks for your helpful response.

Re: LLama.cpp now has a web interface

#46

Post by aidenn0 »

ggerganov wrote: post #29
mk_stjames wrote: post #20

I'm always wondering about the, I don't even know what to call this, etiquette? of proposing PR's to projects like these that add a feature or a demo or whatnot to the main branch of a very focused project by adding something that is very different in interface, language, set and setting etc. So in this case, Tobi made this awesome little web interface that uses minimal HTML and JS as to stay in line with llama.cpp's…

My POV is that llama.cpp is primarily a playground for adding new features to the core ggml library and in the long run an interface for efficient LLM inference. The purpose of the examples in the repo is to demonstrate ways of how to use the ggml library and the LLM interface. The examples are decoupled from the primary code - i.e. you can delete all of them and the project will continue to function and build proper…

Thank you for the ggml library, by the way. It let me play around with whisper in a sane manner. To run the CUDA torch versions, I needed to shut down X to free enough GPU memory for the medium model, and the small model might require me to quit firefox. With ggml, I can use cublas and run even the large model with a huge speedup compared to CPU only torch.

Re: LLama.cpp now has a web interface

#47

Post by vdfs »

eclectic29 wrote: post #45
xal wrote: post #40

Earlier quoted context omitted.

I've tried to eliminate language around being "too busy" from my vocabulary and attempt to replace it with "can't prioritize". That's sometimes a bit awkward, but really trains better habits. Sometimes I prioritize hobbies if I feel I need it, and doing this seemed fun and useful besides!

Thanks for your helpful response.

He have this in his Twitter bio: "CEO by day, Dad in evening, hacker at night"

Re: LLama.cpp now has a web interface

#49

Post by slaymaker1907 »

xal wrote: post #34
isoprophlex wrote: post #21

> I tried to match the spirit of llama.cpp and used minimalistic js dependencies and went with the ozempic css style of ggml.ai. Ozempic? The anti-diabetes drug? That's either a glorious typo or an interesting new adjective...

Nat coined this usage https://twitter.com/natfriedman/status/1668656170645749761

Lol, I’m totally using this to describe memory optimization from now on. There’s a lot of software out there that needs to be more ozempic.

Re: LLama.cpp now has a web interface

#50

Post by sebastiennight »

I'm wondering whether the rendering of markdown through a regex that only deals with H3, strong, em and code is enough, since this implementation specifically seems to ask the bot to reply with markdown?[0]

Other than that, it's cool to see this web interface happen. It's little things like this that make new tools easier to grasp for less technical users.

I believe that people are more likely to go through the hurdles of learning difficult setup procedures (knowing the UX/UI will be easy to use) versus making the setup easier but having the daily use be too hard.

[0]: https://github.com/ggerganov/llama.cpp/pull/1998/commits/c19...

Post reply on HN