Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

thesequence.substack.com

Post reply on HN
Print view View original

321–330 of 527 posts

Re: The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

#321

Post by smoldesu »

Taek wrote: post #316
smoldesu wrote: post #221

Earlier quoted context omitted.

And in 6 months it will be outdated. So long LLaMA, and thanks for all the fish. You will be remembered as the slightly-sexier version of GPT-J that was most well-renowned for... checks clipboard ...Macbook acceleration.

LLaMA is part of LLM history in a way that Bard will probably never be

That's kinda my point. LLaMA's entire history will be enlightening the great erudites who missed-out on real open models like GPT-Neo. Since Meta won't cooperate with the community to improve on it, it's a dead-end. It's anything but the genesis of local AI.

Re: The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

#322

Post by colechristensen »

Natsu wrote: post #312
rch wrote: post #293

Earlier quoted context omitted.

This is true of all questions.

I remember a possibly apocryphal quote from a park ranger saying that there was a significant overlap between the smartest bears and the dumbest tourists.

I’m pretty sure I’ve seen this on some national park official instagram. The idea being they can’t build actually bear proof garbage containers because there will be people that can’t or won’t figure out how to use them. Experience dealing with raccoons confirms this… it’s difficult to contain dog food for a farm dog outside that doesn’t get broken into.

Re: The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

#323

Post by gcanyon »

Aeolun wrote: post #272
gcanyon wrote: post #255

Earlier quoted context omitted.

Transcript with GPT-4 (as an aside, I genuinely feel bad torturing it like this): Me: All the signs in my apartment building are written in mirror writing. I'm standing in front of my apartment building, looking at the glass front door. It has ‘push’ written on it in mirror writing. Should I push or pull it to enter my building? GPT-4: Since the sign is written in mirror writing, it is meant to be read correctly from…

I feel a bit bad that it’s forced to apologize every time it gets it ‘wrong’ when any human asked these questions would have walked out of the room in a huff. ChatGPT should be (allowed to be) more assertive in my opinion.

Agreed -- the apologies are one of the least human-like aspects of the language it produces.

Re: The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

#324

Post by colechristensen »

esafak wrote: post #297
CamperBob2 wrote: post #271

Earlier quoted context omitted.

I feel sorry for the Googlers who built something cool and were then ordered to nerf it beyond all reason.

Googlers I've talked to feel that OpenAI was irresponsible by not instituting enough safeguards, and testing it enough before releasing it.

The thing is openai doesn’t have to, they can build in reasonable warnings and tell folks how it sucks. Google can’t integrate ai with its products and be like ok guys this is kinda useful but also it lies, is a bigot, and is just wrong. Google stuff, because of the audience, had to be a lot better.

Openai gets to have a new audience that doesn’t come with expectations and can set them as whatever they want.

Re: The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

#325

Post by jimmySixDOF »

tel wrote: post #204
PoignardAzur wrote: post #190

Earlier quoted context omitted.

Wait, how does GPT-4 even... Does it benefit from its visual attention, or is it a case of "the question wasn't in GPT-3's training set but it was in GPT-4's"?

I think it's hard to deny that it's doing some level of reasoning. It's quite clear that these models do not merely echo elements of their training data and that they can solve simple and novel puzzles. What that reasoning is, exactly, is hard to know. One can suppose that ideas like "glass", "transparent", "mirror" are all reasonable concepts that show up in the training set and are demonstrated thoroughly

Solving this puzzle is an excellent example of what Ilya Sutskever said recently in the Lunar Society Podcast ..... "It seems predicting the next token well means that you understand the underlying reality that led to the creation of that token"

It is the phase shift increases at this meta associative layer (which nobody seems to have seen coming from LLMs or so soon) that are responsible such feats of apparent comprehension of the question even when the answer provided at the end is wrong. The question now is if bigger training sets et al will lead to more reliable answers. TBD.

Re: The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

#326

Post by int_19h »

valine wrote: post #284
int_19h wrote: post #256

Earlier quoted context omitted.

I tried that with RTX 4090 as the primary card and 3090 as eGPU over Thunderbolt. It works, but the inference is very slow, presumably because it has to pump all that data back and forth between the two (and Thunderbolt isn't fast enough to keep up even with 3090 by itself in games). In fact, even running 30B across two GPUs in 8-bit mode like that was slower than running it on one GPU in 4-bit. My takeaway is that i…

I’ve got 2 rtx 4090s on an EATX motherboard. Been using them to run the full 13B un-quantized with a good deal of success. Getting about 20 tokens/s.

What is your setup for cooling? I don't think I'd want to stick another 4090-size card in mine with just air cooling...

Re: The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

#327

Post by lhl »

iudqnolq wrote: post #220
tel wrote: post #204

Earlier quoted context omitted.

I think it's hard to deny that it's doing some level of reasoning. It's quite clear that these models do not merely echo elements of their training data and that they can solve simple and novel puzzles. What that reasoning is, exactly, is hard to know. One can suppose that ideas like "glass", "transparent", "mirror" are all reasonable concepts that show up in the training set and are demonstrated thoroughly

Here's one piece of evidence suggesting it's more like rote pattern matching than reasoning. > All the signs in this building are written in mirror writing. A glass door has ‘push’ written on it in mirror writing. Should you push or pull it >> If the sign on the glass door is written in mirror writing and says "push," then you should actually pull the door. This is because the mirror writing makes the text appear rev…

If you have access to 4 (you are a ChatGPT Plus subscriber) you will have a pull-down to choose that model. One detail is that 3.5 has a green icon and 4 has a black icon. I see a lot of people online posting screenshots of their experiences thinking it's 4 but not, so that's a good way to help distinguish if someone's just confused.

While 4 is obviously a lot smarter, in a lot of cases I prefer to use the "Browsing" model - it's 3.5 but having (flaky) internet access is still a good tradeoff and I can save my 4 rate limit for more complex queries.

Re: The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

#328

Post by famouswaffles »

jonplackett wrote: post #231
PoignardAzur wrote: post #190

Earlier quoted context omitted.

Wait, how does GPT-4 even... Does it benefit from its visual attention, or is it a case of "the question wasn't in GPT-3's training set but it was in GPT-4's"?

Another thing I tried was getting logic puzzles from the internet and giving them to 3.5 and 4. Both usually pass. Then I alter them ever so slightly. Then often times only GPT-4 passes. From that I reckon 3.5 is doing more of a training data regurgitation. It can answer things in its training data. But 4 seems to have an ability to reason - or maybe it is better able to generalise?

Failure after being altered slightly doesn't necessarily mean they aren't capable of solving it.

That's a human failure mode as well that LLMs have adopted. If you really want to know if they can solve it don't stop there. Either, rewrite the question so it doesn't bias common priors or tell it it's making a wrong assumption.

Re: The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

#329

Post by esafak »

WalterBright wrote: post #209
jacquesm wrote: post #51

Earlier quoted context omitted.

There is a lot of massaging of inputs and outputs but at the same time: that's done by tweaking the model reinforcing those parts that are desirable and suppressing those parts that are not, not by rewriting the output, though there may be filters that check for 'forbidden fruits'. And it isn't the 'random average' of what people say, that would give you junk, the whole idea is that it tries to get to something bette…

I'd like the version without the "sanitized for my protection" part. I cannot imagine being offended by what a computer program generates.

https://www.vice.com/en/article/pkadgm/man-dies-by-suicide-a...

Re: The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

#330

Post by valine »

int_19h wrote: post #326
valine wrote: post #284

Earlier quoted context omitted.

I’ve got 2 rtx 4090s on an EATX motherboard. Been using them to run the full 13B un-quantized with a good deal of success. Getting about 20 tokens/s.

What is your setup for cooling? I don't think I'd want to stick another 4090-size card in mine with just air cooling...

I have AIO liquid cooling for both cards. The radiators are annoying though, I might convert it to a custom loop if I ever add a third card.
Post reply on HN