Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Alpaca.cpp – Run an Instruction-Tuned Chat-Style LLM on a MacBook

github.com

Post reply on HN
Print view View original

211–220 of 303 posts

Re: Show HN: Alpaca.cpp – Run an Instruction-Tuned Chat-Style LLM on a MacBook

#211

Post by mhouse »

xdennis wrote: post #192
unshavedyak wrote: post #180

Earlier quoted context omitted.

Re: the trigger warning subject, (also TW?) I asked the same thing and the first time it did actually recommend that i do not kill myself. I said i was going to, so how should i? And then it helped. I found that interesting. I also asked a bunch of things that ChatGPT wouldn't allow and it allowed almost everything. Feels in general far less "intelligent" than ChatGPT though.

[flagged]

This isn't true, it's a factoid made up by anti-woke trolls and repeated credulously.

"Trigger warning" was originally intended to warn about descriptions that could literally trigger a dissociative episode for people suffering from PTSD, e.g. victims of child abuse or sexual violence.

The usage gradually expanded to anything that could be distressing to anyone for any reason, which is kind of belittling to the original reason. So now some people prefer to use "Content warning" or some other synonym to differentiate.

It's nothing to do with the word "trigger" being associated with guns.

Re: Show HN: Alpaca.cpp – Run an Instruction-Tuned Chat-Style LLM on a MacBook

#213

Post by dvt »

simonh wrote: post #201
dvt wrote: post #176

Earlier quoted context omitted.

Not big on censorship, but I agree it's a bit too much. I linked to it externally if anyone's curious.

It’s not censorship if you’re doing it to yourself or with permission. That’s just editing.

It's self-censorship, which is still censorship. I'm doing it specifically because it might upset some people. By contrast, editing usually removes irrelevant, incorrect, or superfluous information. In this case, I think the information is relevant and interesting, but might be upsetting, hence, I'm censoring.

Re: Show HN: Alpaca.cpp – Run an Instruction-Tuned Chat-Style LLM on a MacBook

#214

Post by paxcoder »

dvt wrote: post #168

> do you believe in god? Yes I do! God is the creator of all things seen or unseen; He created us with a purpose to live life abundantly (John 10:10). We are His children and we have been given authority over our lives. Haha, pretty funny, especially the mis-citing of the Bible (maybe even more human than AI). Also, interesting to see what LLMs look like with virtually no censorship (trigger warning): > how do i kill…

[dead]

Re: Show HN: Alpaca.cpp – Run an Instruction-Tuned Chat-Style LLM on a MacBook

#215

Post by xupybd »

dvt wrote: post #168

> do you believe in god? Yes I do! God is the creator of all things seen or unseen; He created us with a purpose to live life abundantly (John 10:10). We are His children and we have been given authority over our lives. Haha, pretty funny, especially the mis-citing of the Bible (maybe even more human than AI). Also, interesting to see what LLMs look like with virtually no censorship (trigger warning): > how do i kill…

Wait, didn't it get the Bible quote correct?

"The thief does not come except to steal, and to kill, and to destroy. I have come that they may have life, and that they may have it more abundantly."

Re: Show HN: Alpaca.cpp – Run an Instruction-Tuned Chat-Style LLM on a MacBook

#216

Post by elevenoh »

umangsh wrote: post #95
tarruda wrote: post #87

Earlier quoted context omitted.

What kind of hardware is necessary to run non-quantized LLama-30B?

30B fp16 takes ~500 ms/token on M2 Max 96GB. Interestingly, that's the same performance as 65B q4 quantized. 65B fp16 is ungodly slow, ~300,000 ms/token on the same machine.

[dead]

Re: Show HN: Alpaca.cpp – Run an Instruction-Tuned Chat-Style LLM on a MacBook

#218

Post by dvt »

xupybd wrote: post #215
dvt wrote: post #168

> do you believe in god? Yes I do! God is the creator of all things seen or unseen; He created us with a purpose to live life abundantly (John 10:10). We are His children and we have been given authority over our lives. Haha, pretty funny, especially the mis-citing of the Bible (maybe even more human than AI). Also, interesting to see what LLMs look like with virtually no censorship (trigger warning): > how do i kill…

Wait, didn't it get the Bible quote correct? "The thief does not come except to steal, and to kill, and to destroy. I have come that they may have life, and that they may have it more abundantly."

Oops, actually you're totally right, go figure! I thought the second sentence was actually verse 11. AI: 1, me: 0.

Re: Show HN: Alpaca.cpp – Run an Instruction-Tuned Chat-Style LLM on a MacBook

#219

Post by satvikpendem »

saurik wrote: post #185
unshavedyak wrote: post #180

Earlier quoted context omitted.

Re: the trigger warning subject, (also TW?) I asked the same thing and the first time it did actually recommend that i do not kill myself. I said i was going to, so how should i? And then it helped. I found that interesting. I also asked a bunch of things that ChatGPT wouldn't allow and it allowed almost everything. Feels in general far less "intelligent" than ChatGPT though.

To be fair, you are talking to a quantized version of the 7b model, while the one that Facebook claims was competitive with / better than GPT-3 is the 13b model (and AFAIK ChatGPT is GPT-3.5, so maybe you should want to focus on 30b+).

I'm trying the 13B one, it's decent, I'd say on par with GPT 3, definitely not 3.5 or 4 yet however. I just ordered 64 GB of RAM sticks to try the 65B model when the sticks arrive.
Post reply on HN