Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

thesequence.substack.com

Post reply on HN
Print view View original

271–280 of 527 posts

Re: The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

#271

Post by CamperBob2 »

jonplackett wrote: post #241
drumhead wrote: post #228

Earlier quoted context omitted.

I tried the same question with Bard, and it got it right but it also gave me the following: Here are some other things to keep in mind when opening a glass door: Use your hands to open the door. Do not use your feet or other body parts. Be careful not to break the glass. Glass doors can be very fragile, so it is important to be gentle when opening them. If you are unsure how to open a glass door, ask for help from so…

Poor Bard. I felt quite sorry for it while reading that. It’s funny how with these human-like systems you get a gut feeling about their intelligence before you have any hard evidence. My 3 year old worked out Siri is dumb compared to Alexa

I feel sorry for the Googlers who built something cool and were then ordered to nerf it beyond all reason.

Re: The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

#272

Post by Aeolun »

gcanyon wrote: post #255
iudqnolq wrote: post #220

Earlier quoted context omitted.

Here's one piece of evidence suggesting it's more like rote pattern matching than reasoning. > All the signs in this building are written in mirror writing. A glass door has ‘push’ written on it in mirror writing. Should you push or pull it >> If the sign on the glass door is written in mirror writing and says "push," then you should actually pull the door. This is because the mirror writing makes the text appear rev…

Transcript with GPT-4 (as an aside, I genuinely feel bad torturing it like this): Me: All the signs in my apartment building are written in mirror writing. I'm standing in front of my apartment building, looking at the glass front door. It has ‘push’ written on it in mirror writing. Should I push or pull it to enter my building? GPT-4: Since the sign is written in mirror writing, it is meant to be read correctly from…

I feel a bit bad that it’s forced to apologize every time it gets it ‘wrong’ when any human asked these questions would have walked out of the room in a huff.

ChatGPT should be (allowed to be) more assertive in my opinion.

Re: The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

#273

Post by alchemist1e9 »

int_19h wrote: post #256
alchemist1e9 wrote: post #224

Earlier quoted context omitted.

Is it possible to build systems with multiple GPUs to run the 65B or larger when they appear? I’m not really sure and looking for clarification from anyone who knows. My understanding is it is possible to split the layers between the GPUs so a system with 4 high end consumer GPUs might work well.

I tried that with RTX 4090 as the primary card and 3090 as eGPU over Thunderbolt. It works, but the inference is very slow, presumably because it has to pump all that data back and forth between the two (and Thunderbolt isn't fast enough to keep up even with 3090 by itself in games). In fact, even running 30B across two GPUs in 8-bit mode like that was slower than running it on one GPU in 4-bit. My takeaway is that i…

Looking at the top end H100 80GB systems with NVLink from HPC vendors and it occurred to me we are about to swing back to massive almost mainframe like form-factor systems, giant bus, like old expandable qbus in the 80s but this time for GPUs.

What I mean is they have systems with 8x cards but given the compute requirements of these huge LLMs probably systems with 32+ all on a dedicated memory bus (NVLink) are what will be needed as weights sizes expand. This is all for inference btw, not even training, but same hold for training probably best possible interconnect between same monster systems.

I’m dreaming there might be a distributed eventually consistent partial training algorithm then that would democratize creation of these models.

In regards to smaller scale individual systems for inference, if one has resources and is fairly technical and can utilize such technology then perhaps in 5-10 years the wealthy might buy units for $50K+ that get installed in their home or something.

Really incredible developments very quickly. Apologies for the potentially inappropriately long rant to the previous comment.

Re: The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

#274

Post by Aeolun »

fpgaminer wrote: post #234

Slightly tangential, but I had intended to start playing around with LLaMA and building some agents. I got the 4-bit versions up and running on my 3090 before I was quickly nerd snipped by a performance problem... The popular repo for quantizing and running LLaMA is the GPTQ-for-llama repo on github, which mostly copies from the GPTQ authors. The CUDA kernels are needed to support the specific kind of quantization th…

Ah well, at least you are spending your time productively.

Re: The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

#275

Post by Aeolun »

tel wrote: post #210
nkzd wrote: post #97

Earlier quoted context omitted.

How could Meta ever find out your private business is using their model without a whistleblower? It's practically impossible.

Have reasonable suspicion, sue you, and then use discovery to find any evidence at all that your models began with LLaMA. Oh, you don't have substantial evidence for how you went from 0 to a 65B-parameter LLM base model? How curious.

Fell off the back of a truck!

Re: The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

#276

Post by famouswaffles »

whimsicalism wrote: post #270
famouswaffles wrote: post #242

Earlier quoted context omitted.

It's pretty much looking like anything can be extracted from language. Some harder than others for sure but with enough scale it does look like eventually everything falls. Text only GPT-4 has a pretty solid understanding of space that 3.5 definitely lacks. You can see more thorough experiments in the microsoft agi paper where they test it's ability to track the visual space of a maze.

There is no such thing as text-only GPT-4 unless you are referring to at inference time.

There is such a thing as a text only GPT-4 lol. It wasn't trained to be multimodal from scratch. First a text only version was trained and then it was made multimodal somehow ( the details are unknown but making a text only LLM multimodal isn't new e.g Palm, Flamingo, Blip-2, Fromage). The text only version exists and is what the microsoft researchers had access to.

Re: The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

#277

Post by Aeolun »

trifurcate wrote: post #174
apetresc wrote: post #135

Earlier quoted context omitted.

How have you managed to run the 65B model? Cloud resources, or you have a very kitted-out homelab?

All you need is 2 3090s.

All you need is a few thousand dollars lying around to spend solely on your inference fun?

I don’t think that many people really qualify as such (though it’s probably true that many of them are on HN).

Re: The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

#278

Post by valine »

superkuh wrote: post #118

I've spent an embarassing amount of time since the llamas leaked playing with them, the tools to run them, and writing wrappers for them. They are technically alternatives in the sense that they're incomparably better chat bots than anything in the past. But at least for the 30B and under versions (65B is too big for me to run), no matter what fine tuning is done (alpaca, gpt4all, vicuna, etc), the llamas themselves…

Are you using quantization? I’ve gotten very good results from the float16 13B vicuna model.

Re: The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

#279

Post by eiz »

barbariangrunge wrote: post #165
greyface- wrote: post #157

Earlier quoted context omitted.

A contributor who is also a Facebook employee and co-author of the LLaMA paper, presumably speaking in official capacity.

Why would you presume that by default? Need a quote to conclude that

Here's a couple more quotes from Yann LeCun, their Chief AI Scientist:

https://twitter.com/ylecun/status/1643945742850031616

https://twitter.com/ylecun/status/1644503237699969026

pretty clear the non-release and extremely half assed response to the leak is just CYA

Re: The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

#280

Post by [deleted] »

jonplackett wrote: post #162
superkuh wrote: post #118

I've spent an embarassing amount of time since the llamas leaked playing with them, the tools to run them, and writing wrappers for them. They are technically alternatives in the sense that they're incomparably better chat bots than anything in the past. But at least for the 30B and under versions (65B is too big for me to run), no matter what fine tuning is done (alpaca, gpt4all, vicuna, etc), the llamas themselves…

The difference between 3.5 and 4 is gigantic even in my fairly limited experience. I gave them both some common sense tests and this one stuck out to me. Q: A glass door has ‘push’ written on it in mirror writing. Should you push or pull it GPT-3.5: If the word "push" is written in mirror writing on a glass door, you should push the door to open it GPT-4: Since the word "push" is written in mirror writing, it suggest…

[deleted]
Post reply on HN