LLama.cpp now has a web interface • Hacker News Board

Re: LLama.cpp now has a web interface

#31

Post by Xenoamorphous » Wed, Jul 05, 2023, 7:37 PM UTC

swyx wrote: post #18

Xenoamorphous wrote: post #11

Earlier quoted context omitted.

A billionare co-founder at that!

actually did tobi even have any other cofounders? i just realized i only ever hear about him and harvey finkelstein edit: google says scott lake was actually the founding CEO. TIL! https://www.linkedin.com/mwlite/profile/in/scottlake?origina...

Just going by the Google snippet I got when searching for “shopify ceo”. Not sure if the downvotes I got are because somone thinks that the co-founding aspect is wrong or because they think that him being a billionaire is irrelevant, but IMO it makes the fact he’s opening PRs even cooler.

Re: LLama.cpp now has a web interface

#32

Post by isoprophlex » Wed, Jul 05, 2023, 7:40 PM UTC

jahewson wrote: post #28

isoprophlex wrote: post #21

> I tried to match the spirit of llama.cpp and used minimalistic js dependencies and went with the ozempic css style of ggml.ai. Ozempic? The anti-diabetes drug? That's either a glorious typo or an interesting new adjective...

I did a double-take when I saw that too. It’s now prescribed as a weight-loss drug and is very much in the zeitgeist so yeah I think it’s a new adjective. Personally I’m going to stick with “light weight”.

The more I think about it, the more I'm loving it

Re: LLama.cpp now has a web interface

#33

Post by xal » Wed, Jul 05, 2023, 7:51 PM UTC

mk_stjames wrote: post #20

I'm always wondering about the, I don't even know what to call this, etiquette? of proposing PR's to projects like these that add a feature or a demo or whatnot to the main branch of a very focused project by adding something that is very different in interface, language, set and setting etc. So in this case, Tobi made this awesome little web interface that uses minimal HTML and JS as to stay in line with llama.cpp's…

GG would just say no and that's that. No hard feelings, that's what makes open source so great.

Re: LLama.cpp now has a web interface

#34

Post by xal » Wed, Jul 05, 2023, 7:53 PM UTC

isoprophlex wrote: post #21

> I tried to match the spirit of llama.cpp and used minimalistic js dependencies and went with the ozempic css style of ggml.ai. Ozempic? The anti-diabetes drug? That's either a glorious typo or an interesting new adjective...

Nat coined this usage https://twitter.com/natfriedman/status/1668656170645749761

Re: LLama.cpp now has a web interface

#35

Post by xal » Wed, Jul 05, 2023, 7:54 PM UTC

ynniv wrote: post #24

i'm importing from js cdns instead of adding them here FWIW this seems counter to llama.cpp's philosophy.

I agree, I ended up getting rid of it. Only one dependency is downloaded (via bash script) and everything is baked into the binary now.

Re: LLama.cpp now has a web interface

#36

Post by eclectic29 » Wed, Jul 05, 2023, 8:21 PM UTC

Can someone shed light on how does a CEO with 3 kids get time to hack on something like this? Some might even argue that all the time spent doing this would’ve been better spent on CEO activities, but thankfully this is HN and people have hobbies, so that’s that, but this can be a very time consuming hobby.

Re: LLama.cpp now has a web interface

#37

Post by vitaminka » Wed, Jul 05, 2023, 8:29 PM UTC

ggerganov wrote: post #29

mk_stjames wrote: post #20

I'm always wondering about the, I don't even know what to call this, etiquette? of proposing PR's to projects like these that add a feature or a demo or whatnot to the main branch of a very focused project by adding something that is very different in interface, language, set and setting etc. So in this case, Tobi made this awesome little web interface that uses minimal HTML and JS as to stay in line with llama.cpp's…

My POV is that llama.cpp is primarily a playground for adding new features to the core ggml library and in the long run an interface for efficient LLM inference. The purpose of the examples in the repo is to demonstrate ways of how to use the ggml library and the LLM interface. The examples are decoupled from the primary code - i.e. you can delete all of them and the project will continue to function and build proper…

i’m curious, what’s is the approach for maintainable and decoupled various gpu backends?

Re: LLama.cpp now has a web interface

#38

Post by ggerganov » Wed, Jul 05, 2023, 8:51 PM UTC

vitaminka wrote: post #37

ggerganov wrote: post #29

Earlier quoted context omitted.

My POV is that llama.cpp is primarily a playground for adding new features to the core ggml library and in the long run an interface for efficient LLM inference. The purpose of the examples in the repo is to demonstrate ways of how to use the ggml library and the LLM interface. The examples are decoupled from the primary code - i.e. you can delete all of them and the project will continue to function and build proper…

i’m curious, what’s is the approach for maintainable and decoupled various gpu backends?

It was designed in #915 (read just the OP and the linked PRs at the end) and the implementation pretty much follows it closely, at least for the Metal backend. The CUDA and OpenCL backends are currently slightly coupled in ggml as they started developing before #915, but I think we'll resolve this eventually.

#915 - https://github.com/ggerganov/llama.cpp/discussions/915

Re: LLama.cpp now has a web interface

#39

Post by mk_stjames » Wed, Jul 05, 2023, 9:04 PM UTC

ggerganov wrote: post #29

mk_stjames wrote: post #20

I'm always wondering about the, I don't even know what to call this, etiquette? of proposing PR's to projects like these that add a feature or a demo or whatnot to the main branch of a very focused project by adding something that is very different in interface, language, set and setting etc. So in this case, Tobi made this awesome little web interface that uses minimal HTML and JS as to stay in line with llama.cpp's…

My POV is that llama.cpp is primarily a playground for adding new features to the core ggml library and in the long run an interface for efficient LLM inference. The purpose of the examples in the repo is to demonstrate ways of how to use the ggml library and the LLM interface. The examples are decoupled from the primary code - i.e. you can delete all of them and the project will continue to function and build proper…

Thanks for replying to me directly! I'm finding it fascinating to follow this project. Good luck with your company Georgi.

Re: LLama.cpp now has a web interface

#40

Post by xal » Wed, Jul 05, 2023, 9:15 PM UTC

eclectic29 wrote: post #36

Can someone shed light on how does a CEO with 3 kids get time to hack on something like this? Some might even argue that all the time spent doing this would’ve been better spent on CEO activities, but thankfully this is HN and people have hobbies, so that’s that, but this can be a very time consuming hobby.

I've tried to eliminate language around being "too busy" from my vocabulary and attempt to replace it with "can't prioritize". That's sometimes a bit awkward, but really trains better habits. Sometimes I prioritize hobbies if I feel I need it, and doing this seemed fun and useful besides!