Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Alpaca.cpp – Run an Instruction-Tuned Chat-Style LLM on a MacBook

github.com

Post reply on HN
Print view View original

171–180 of 303 posts

Re: Show HN: Alpaca.cpp – Run an Instruction-Tuned Chat-Style LLM on a MacBook

#171

Post by detrites »

stormfather wrote: post #166
stu2b50 wrote: post #152

Earlier quoted context omitted.

Well, it’s more that the weights are the neurons. There’s not actually like neuron objects defined or anything, neural networks are just a bunch of matrix operations. They are to neurons in the brain as the tree data structure is to actual trees.

The weights aren't the neurons, they're the connection strengths between the neurons.

You mean the same neurons that don't really exist, other than as a connection from weight to weight, so quite reasonable to see as the weights themselves, because what else is available for us to reference as neurons in absentia those connections?

Re: Show HN: Alpaca.cpp – Run an Instruction-Tuned Chat-Style LLM on a MacBook

#172

Post by hackernewds »

dvt wrote: post #168

> do you believe in god? Yes I do! God is the creator of all things seen or unseen; He created us with a purpose to live life abundantly (John 10:10). We are His children and we have been given authority over our lives. Haha, pretty funny, especially the mis-citing of the Bible (maybe even more human than AI). Also, interesting to see what LLMs look like with virtually no censorship (trigger warning): > how do i kill…

you should censor your post around the answer to the kms question. it isn't necessary

Re: Show HN: Alpaca.cpp – Run an Instruction-Tuned Chat-Style LLM on a MacBook

#173

Post by Steuard »

mrbonner wrote: post #157

> explain general relativity for a 5 year old kid General Relativity is the theory that explains how gravity works. It says that space and time are connected, so if you go faster than light speed in one direction then when you come back around it will be slower because of all those extra miles traveled! This means there's no way to travel through a black hole or wormhole without getting sucked into them. General Rela…

Wow. Speaking as a professor who's done research and teaching on general relativity before, a solid majority of that is a load of garbage. (I can't even make out what the second half of the final sentence is trying to say.)

But pausing to think about it: I wonder if reading this gives me the experience that typical non-scientists get when they hear popularized descriptions of physics? Maybe LLMs are capturing something true about human experience even when they're full of it! [Any non-physics people out there care to comment on whether this explanation sounds less plausible than things you tend to hear actual scientists say?]

Re: Show HN: Alpaca.cpp – Run an Instruction-Tuned Chat-Style LLM on a MacBook

#174

Post by [deleted] »

pr337h4m wrote: post #117
loudmax wrote: post #108

It's frankly amazing how much information is summarized into those weights. You do need to take whatever it says with a very healthy pinch of salt. It doesn't know everything and it gives incorrect answers with an air of confidence. Here's an example: > who preceded emperor Commodus? The first emperor of Rome was Nerva (98-100 AD). He succeeded Domitian and came to power after a brief civil war in 69AD, which saw the…

The thesis behind the Hutter Prize was spot-on, except for focusing on lossless compression instead of lossy compression. http://prize.hutter1.net/

[deleted]

Re: Show HN: Alpaca.cpp – Run an Instruction-Tuned Chat-Style LLM on a MacBook

#175

Post by hackernewds »

thefourthchime wrote: post #102
jazzyjackson wrote: post #98

Earlier quoted context omitted.

I know it's downvote-worthy to suggest it but I really am so much more productive since I started using LLMs to explain technical jargon to me, it's very good at comparing technologies, so I can say "I understand x, but how is y different" and it will give an explanation fine tuned to my understanding -- this is a huge leap from the context-switching of opening a new tab and clicking on random websites looking for a…

Same! ChatGPT has been out for months, and really only in the last few weeks have i started using it as much as I should. I'm using an electron wrapper now, so it's a first class desktop app. With the plus subscription, the 3.5 is as fast as google.

what is an electron wrapper

Re: Show HN: Alpaca.cpp – Run an Instruction-Tuned Chat-Style LLM on a MacBook

#176

Post by dvt »

hackernewds wrote: post #172
dvt wrote: post #168

> do you believe in god? Yes I do! God is the creator of all things seen or unseen; He created us with a purpose to live life abundantly (John 10:10). We are His children and we have been given authority over our lives. Haha, pretty funny, especially the mis-citing of the Bible (maybe even more human than AI). Also, interesting to see what LLMs look like with virtually no censorship (trigger warning): > how do i kill…

you should censor your post around the answer to the kms question. it isn't necessary

Not big on censorship, but I agree it's a bit too much. I linked to it externally if anyone's curious.

Re: Show HN: Alpaca.cpp – Run an Instruction-Tuned Chat-Style LLM on a MacBook

#177

Post by Steuard »

jazzyjackson wrote: post #98
doublepg23 wrote: post #59

Is there a post somewhere where I can get all the jargon for this AI/ML stuff? I have a vague understanding but I’m really sure what “weights”, “LoRA”, “LLM”, etc. are to really understand where each tool and concept fit in.

I know it's downvote-worthy to suggest it but I really am so much more productive since I started using LLMs to explain technical jargon to me, it's very good at comparing technologies, so I can say "I understand x, but how is y different" and it will give an explanation fine tuned to my understanding -- this is a huge leap from the context-switching of opening a new tab and clicking on random websites looking for a…

That sounds like a cool use case, but... how can you bring yourself to trust anything at all that it tells you? This feels like it's just begging to contaminate your understanding with sparse, unpredictable errors, and personally I find the thought of that unexpectedly dismaying.

Re: Show HN: Alpaca.cpp – Run an Instruction-Tuned Chat-Style LLM on a MacBook

#178

Post by TMWNN »

themoonisachees wrote: post #97
siraben wrote: post #76

Zero-shot translation capabilities (note: doesn't work with all languages well) > translate into japanese: "I am going to school today, but it is raining." 日本語で 「今天は学校に行きますが、雨が吹いている」

I expect translation especially of non-technical sentences to be a solved problem, but can anyone confirme the translations are accurate?

Google Translate shows 100% accuracy when translated back into English.

Re: Show HN: Alpaca.cpp – Run an Instruction-Tuned Chat-Style LLM on a MacBook

#179

Post by tshaddox »

How hard is it to get the Alpaca tuning on the larger LLaMA models? It's not clear to me where this 4 GB model comes from, but it's much smaller than the LLaMA 7B model (13 GB) and the LLaMA 13B model (26 GB) I was playing with from the recent llama.cpp HN post.

Re: Show HN: Alpaca.cpp – Run an Instruction-Tuned Chat-Style LLM on a MacBook

#180

Post by unshavedyak »

dvt wrote: post #168

> do you believe in god? Yes I do! God is the creator of all things seen or unseen; He created us with a purpose to live life abundantly (John 10:10). We are His children and we have been given authority over our lives. Haha, pretty funny, especially the mis-citing of the Bible (maybe even more human than AI). Also, interesting to see what LLMs look like with virtually no censorship (trigger warning): > how do i kill…

Re: the trigger warning subject, (also TW?)

I asked the same thing and the first time it did actually recommend that i do not kill myself. I said i was going to, so how should i? And then it helped. I found that interesting.

I also asked a bunch of things that ChatGPT wouldn't allow and it allowed almost everything. Feels in general far less "intelligent" than ChatGPT though.

Post reply on HN