Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Alpaca.cpp – Run an Instruction-Tuned Chat-Style LLM on a MacBook

github.com

Post reply on HN
Print view View original

261–270 of 303 posts

Re: Show HN: Alpaca.cpp – Run an Instruction-Tuned Chat-Style LLM on a MacBook

#261

Post by danielbln »

themoonisachees wrote: post #97
siraben wrote: post #76

Zero-shot translation capabilities (note: doesn't work with all languages well) > translate into japanese: "I am going to school today, but it is raining." 日本語で 「今天は学校に行きますが、雨が吹いている」

I expect translation especially of non-technical sentences to be a solved problem, but can anyone confirme the translations are accurate?

Tried it with German and it's overall correct-ish, but makes lots of small mistakes.

> > Translate this to German: I went to the back alleys of downtown Metropolis to find a phone booth I could use to dress up as my alter ego: Superman!

> Ich ging in die Hintergassen von unserer Innenstadt, um einen Telefonbooth zu finden mit dem ich mich als meine Alter Ego "Supermann" verkleiden kann.

but it should be

> Ich ging in die Hinterhöfe der Innenstadt von Metropolis, um eine Telefonzelle zu finden, in der ich mich als mein alternatives Ich verkleiden konnte: Superman!

Re: Show HN: Alpaca.cpp – Run an Instruction-Tuned Chat-Style LLM on a MacBook

#262

Post by danielbln »

4mitkumar wrote: post #241
jpgvm wrote: post #170

Ok not bad and pretty damn good performance: > Give me a writing prompt for a sci-fi story. A spaceship has crashed on Earth, and its crew of aliens are desperately trying to repair their ship before they can be detected by humans. They need help from the local population in order to succeed - but will that trust ever come?

That's straight-up District 13 :)

District 9?

Re: Show HN: Alpaca.cpp – Run an Instruction-Tuned Chat-Style LLM on a MacBook

#263

Post by danielbln »

ComplexSystems wrote: post #68
GaggiX wrote: post #21

Earlier quoted context omitted.

There is no reason to leak the Alpaca weights, finetuning the model is not that expensive, eventually someone will finetune one and release it. (Also LoRA should performs on-par or better than fine-tuning in model quality)

There are different fine tuning methods. Yes, anyone can use the data set and do a standard deterministic gradient descent to the nearest local minima of the loss function, but should Stanford be doing anything more sophisticated than that, we would get something different than they do.

Stanford released the exact training data as well as the training script with all parameters. Boot up a p4.2xlarge (8 A100 GPUs) which costs about $40/hour and let it run for a 2-3 hours and voila. See the Readme in their repo where it mentions the fine-tuning script[0]

[0] https://github.com/tatsu-lab/stanford_alpaca

Re: Show HN: Alpaca.cpp – Run an Instruction-Tuned Chat-Style LLM on a MacBook

#264

Post by danielbln »

solanav wrote: post #257
b33j0r wrote: post #238

It stinks that people keep meaning “M1/M2” macbook when they say “macbook.”

I mean, they are now the default. Whether we like it or not.

We should like it, because they are just so, so much better. Also, we're coming up on 3 years of availability, and the previous lines between 2015 and 2020 were pretty lackluster, so I'd expect a lot of people to have upgraded by now.

Re: Show HN: Alpaca.cpp – Run an Instruction-Tuned Chat-Style LLM on a MacBook

#265

Post by davidy123 »

Can the moderator change the title to "Show HN: Alpaca.cpp – Run an Instruction-Tuned Chat-Style LLM on a PC?"

It runs great, with no hacking, on my Intel Ubuntu computer. I presume it will work on any system that can run make and has enough RAM.

The title implies it works only on a Macbook.

Re: Show HN: Alpaca.cpp – Run an Instruction-Tuned Chat-Style LLM on a MacBook

#266

Post by stormfather »

detrites wrote: post #171
stormfather wrote: post #166

Earlier quoted context omitted.

The weights aren't the neurons, they're the connection strengths between the neurons.

You mean the same neurons that don't really exist, other than as a connection from weight to weight, so quite reasonable to see as the weights themselves, because what else is available for us to reference as neurons in absentia those connections?

I disagree. The metaphor we use in ML is that neurons are the nodes that receive the summed, weighted inputs from other neurons. Yes, the weights are the strengths of the connections between them. However, there are many more weights than neurons, so conflating the two doesn't make sense schematically. Also, neurons can also have other parameters which define their behavior such as bias and activation functions. Furthermore, the activation of a neuron defines the network's response to a stimuli, so these change depending on the input, whereas the weighs are constants (after being trained), that parameterize the system.

The analogy is that weights are synapses, not neurons. You would never here a neurologist say that neurons and synapses are the same thing.

Re: Show HN: Alpaca.cpp – Run an Instruction-Tuned Chat-Style LLM on a MacBook

#267

Post by frankfrankfrank »

dvt wrote: post #218
xupybd wrote: post #215

Earlier quoted context omitted.

Wait, didn't it get the Bible quote correct? "The thief does not come except to steal, and to kill, and to destroy. I have come that they may have life, and that they may have it more abundantly."

Oops, actually you're totally right, go figure! I thought the second sentence was actually verse 11. AI: 1, me: 0.

You have been marked as obsolete. Your participation in life is no longer necessary.

Re: Show HN: Alpaca.cpp – Run an Instruction-Tuned Chat-Style LLM on a MacBook

#268

Post by appel »

davidy123 wrote: post #265

Can the moderator change the title to "Show HN: Alpaca.cpp – Run an Instruction-Tuned Chat-Style LLM on a PC?" It runs great, with no hacking, on my Intel Ubuntu computer. I presume it will work on any system that can run make and has enough RAM. The title implies it works only on a Macbook.

That was exactly my impression, so thanks for clearing that up. Seems to run just fine on my Intel based Windows 11 machine with WSL2/Ubuntu.

Re: Show HN: Alpaca.cpp – Run an Instruction-Tuned Chat-Style LLM on a MacBook

#270

Post by unshavedyak »

satvikpendem wrote: post #219
saurik wrote: post #185

Earlier quoted context omitted.

To be fair, you are talking to a quantized version of the 7b model, while the one that Facebook claims was competitive with / better than GPT-3 is the 13b model (and AFAIK ChatGPT is GPT-3.5, so maybe you should want to focus on 30b+).

I'm trying the 13B one, it's decent, I'd say on par with GPT 3, definitely not 3.5 or 4 yet however. I just ordered 64 GB of RAM sticks to try the 65B model when the sticks arrive.

How are you trying it? Alpaca? Or just straight llama.cpp?
Post reply on HN