Live data from Hacker News

Show HN: Alpaca.cpp – Run an Instruction-Tuned Chat-Style LLM on a MacBook

github.com

151–160 of 303 posts

Re: Show HN: Alpaca.cpp – Run an Instruction-Tuned Chat-Style LLM on a MacBook

#151

Is there a post somewhere where I can get all the jargon for this AI/ML stuff? I have a vague understanding but I’m really sure what “weights”, “LoRA”, “LLM”, etc. are to really understand where each tool and concept fit in.

This may help a little: https://daily.ginger-t.link/glossary (DISCLAIMER: my site)

Re: Show HN: Alpaca.cpp – Run an Instruction-Tuned Chat-Style LLM on a MacBook

#152
post #77

Is there a post somewhere where I can get all the jargon for this AI/ML stuff? I have a vague understanding but I’m really sure what “weights”, “LoRA”, “LLM”, etc. are to really understand where each tool and concept fit in.

I was like this a week ago. Basically, weights are numbers you feed to each neuron in the model, LoRA is a technique to adjust only some of the weights so you can fine-tune the model fast and on cheaper hardware, llm is a "large language model". I've been asking chatgpt4 these things and learning at my own pace and context: https://rentry.co/vafkn

Well, it’s more that the weights are the neurons. There’s not actually like neuron objects defined or anything, neural networks are just a bunch of matrix operations. They are to neurons in the brain as the tree data structure is to actual trees.

Re: Show HN: Alpaca.cpp – Run an Instruction-Tuned Chat-Style LLM on a MacBook

#153

It's frankly amazing how much information is summarized into those weights. You do need to take whatever it says with a very healthy pinch of salt. It doesn't know everything and it gives incorrect answers with an air of confidence. Here's an example: > who preceded emperor Commodus? The first emperor of Rome was Nerva (98-100 AD). He succeeded Domitian and came to power after a brief civil war in 69AD, which saw the…

“Closed to truth” is actually the worst property of LLM models. Because it hard for human to detect if it is wrong or not.

Re: Show HN: Alpaca.cpp – Run an Instruction-Tuned Chat-Style LLM on a MacBook

#155

Genuinely curious since I don't want to brick my mac :) - It seems that there are more than a handful of segmentation faults happening for various users running this program on their Mac's. Is there a possibility that this may cause system wide stability issues? Wondering if the MacOS user program space is well isolated that these problems are a thing of the past and/or other environments? Also, ChatGPT seems ambival…

> Wondering if the MacOS user program space is well isolated that these problems are a thing of the past and/or other environments?

If by "past" you mean "1980s" and "other environments" you mean DOS, yes. I suppose it's still a problem for FreeDOS, but I recommend against trying to run ML models in it.

Re: Show HN: Alpaca.cpp – Run an Instruction-Tuned Chat-Style LLM on a MacBook

#157
> explain general relativity for a 5 year old kid

General Relativity is the theory that explains how gravity works. It says that space and time are connected, so if you go faster than light speed in one direction then when you come back around it will be slower because of all those extra miles traveled! This means there's no way to travel through a black hole or wormhole without getting sucked into them. General Relativity also says that gravity is actually the same thing as acceleration, so if something has mass then you can think about it like an object on Earth with weight pushing down from above and pulling upwards!

Re: Show HN: Alpaca.cpp – Run an Instruction-Tuned Chat-Style LLM on a MacBook

#158

Genuinely curious since I don't want to brick my mac :) - It seems that there are more than a handful of segmentation faults happening for various users running this program on their Mac's. Is there a possibility that this may cause system wide stability issues? Wondering if the MacOS user program space is well isolated that these problems are a thing of the past and/or other environments? Also, ChatGPT seems ambival…

It's a user space process... it won't ever crash the kernel. In fact, one of the reasons for the kernel existing in the first place is to virtualize and isolate process so they won't crash the whole system.

Re: Show HN: Alpaca.cpp – Run an Instruction-Tuned Chat-Style LLM on a MacBook

#159
post #63

=> write a haiku about leaves Autumn's golden hues - Leaves of red, yellow and blue! \ #haikupoetry #leavesshort[1] > segmentation fault ./chat Interesting.

I see the segfault in one of the prompt as well.

> write a typescript program that read a text file and print to the screen

const inputFile = './input.txt'; // path to your txtfile let outputString; // string variable for printing out contents of .txt files // open up our file, if it exists try { fsReadStream(inputFile); } catch (e) {} if (!!errno) throw errnoError(); /* Throw an error on ENOENT */ // if the inputfile does not exist // read in contents of file and print out to screen [1] 51232 segmentation fault ./chat

Re: Show HN: Alpaca.cpp – Run an Instruction-Tuned Chat-Style LLM on a MacBook

#160

It's frankly amazing how much information is summarized into those weights. You do need to take whatever it says with a very healthy pinch of salt. It doesn't know everything and it gives incorrect answers with an air of confidence. Here's an example: > who preceded emperor Commodus? The first emperor of Rome was Nerva (98-100 AD). He succeeded Domitian and came to power after a brief civil war in 69AD, which saw the…

This is one of the things that LLMs seem to be struggling with. They're very good at tone, and feeling right without actually being write.

My current test is to ask them to write a three paragraph introduction to me for a departmental seminar (the kind of thing I hate doing). Thus far, none of them have managed to do it without introducing factual errors - wrong degrees (or institutions), wrong research focuses, etc. But they're all credible sounding in their wrongness.

Post reply on HN