Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

thesequence.substack.com

Post reply on HN
Print view View original

471–480 of 527 posts

Re: The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

#471

Post by tel »

steveBK123 wrote: post #287
tel wrote: post #206

Earlier quoted context omitted.

This is an interesting argument as it's easy to apply it nearly universally to any example of learning. What sort of evidence would convince you that it is learning?

That the LLMs are actually evolving before my eyes within & across sessions, without human-in-the-loop "hand tuning" iterations (sounds like injections of glorified if statements to this guy) .

You want to witness the learning firsthand, I suppose. That's reasonable. I'd also suggest that it's possible to imagine questions for the LLM that it cannot solve today and that you reasonably believe will not be available to OpenAI to "hand tune" it against. If you can come up with such a problem, it can't solve it today, but does in the future then you have some evidence, I'd think.

What's more, is we can do that today. Just think of any problem which you suspect won't be included in OpenAI's hand-tunings and check both 3.5 and 4.

Re: The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

#472

Post by nailer »

Zuiii wrote: post #416
nailer wrote: post #394

Earlier quoted context omitted.

FYI I think you made a good point and I am not silently downvoting you. I feel the same regarding people moderating comments to oblivion because they disagree on HN and have commented about this recently to dang.

I should have been more careful with how I wrote my comment. I almost always get silently downvoted when I bring this point up for discussion so I was directing that "you" at those people in particular and not at you. Sorry for coming across as attacking you and thank you for caring about hn.

It’s OK. Have a happy Easter. :)

Re: The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

#473

Post by Closi »

buryat wrote: post #307
novok wrote: post #296

Earlier quoted context omitted.

It's also an order of magnitude larger in parameter count...

unconfirmed, there’s a theory that it maybe similarly sized as gpt3.5 but with additional RLHF and because of other things it outperforms

Apparently the RLHF actually reduces it's capability, it just improves alignment/safety.

Re: The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

#474

Post by ChatGTP »

mrshadowgoose wrote: post #453
ChatGTP wrote: post #407

Earlier quoted context omitted.

I’m scared of AI, very scared, but I am way way more scared of authoritarianism + AI. If I lose my job to AI, I’ll be at least able to create new things using open source and free AI so I can hopefully be able to feed my family. If I’m locked out of it all together, I’m toast. The other thing is, OpenAI is collecting all data and using it for training, this is a disaster on many levels. I can’t be a party to it. All…

I've thought about this a ton lately. Given the unexpectedly rapid pace of development towards AGI, if progress is sustained, I don't see how this ends well in the vast majority of cases. The game theory is identical to that of nuclear weapons development, even if one's intents are good. On the road to AGI, there exists a development gap (the size of which is unknowable ahead of time) where a single actor that has ac…

Yup, it’s not looking great.

We will build terminators, they might not be as cool as what’s in the movies but you will not be able to stop them. You will be told what to do and if you don’t like it…

The government doesn’t need you anymore, you’re tax dollars are worthless and really, you’re a key driver of climate change, you can’t revolt because armies of bots without any conscience enforce “the law”, what’s next ?

This seems to be the way that OpenAI is behaving now that they have a model that is practically relevant, and I don't blame them at all for working this way. It just makes sense.

Yup, and you have a government who has no desire to reign it in.

The only hope we have is failure to get an AGI, or the AGIs are some how ultra compassionate, or we learn to augment our intelligence very quickly.

I saw this Boston Dynamics clip the other day and this nice enough looking hippy guy was like , “we just want atlas to help people…”, I felt sick and felt sorry for him because he doesn’t realise that it will very likely be used to do bad stuff by the Military and law enforcement.

All this “progress” is sold to us under the guise of helping people, “African babies need AI doctors”…

Re: The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

#475

Post by barbariangrunge »

sbierwagen wrote: post #238
barbariangrunge wrote: post #168

Earlier quoted context omitted.

I thought it needed 64gb of vram. 64gb of ram is easy to obtain

5950x is a CPU model. Integer-quantized models are generally run with CPU inference. For the larger models the problem then becomes generation time per token.

If your desktop had 256gb of ram, could you train a far larger model? Some motherboards support that

Re: The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

#476

Post by jarek83 »

lovemenot wrote: post #414
jarek83 wrote: post #392

Earlier quoted context omitted.

GPTs are parrots. v4 is much better but still just a parrot. Since humans have yet to figure out the whole process behind brain thinking/intelligence/abstractions, then there is no way they can replicate that into machines. And brain is all about chemical reactions - can that be replicated with just 0s and 1s? Sad fact is that many people will think GPT writes something meaningful, while it will always only write som…

This argument is imbalanced. Not necessarily wrong. To follow scientific method, you should specify what would be meaningful according to your parameters: not merely parroting. If and when that is achieved by LLMs, instead of moving the goalposts, accept that something genuine has happened. >> We will be flooded with content that will get harder and harder to prove wrong, but it will be wrong. I mostly agree, especia…

>> But since you wish to draw a line in the sand, define quality. Or truth.

I asked it about tourists attractions in a city I know. One of the items it listed was a Zoo. There is no Zoo there. It seemed like it just took the majority of the question into account at some point and ignored the city name itself and since most questions like that are about big cities that have zoos, it went ahead to suggest it, since it was among probable answers.

It will get better with more input and will get this kind of obvious questions right. But I believe it will never truly 'know' what it is talking about. And we will start to believe it does.

Re: The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

#477

Post by jarek83 »

fl7305 wrote: post #442
lovemenot wrote: post #414

Earlier quoted context omitted.

This argument is imbalanced. Not necessarily wrong. To follow scientific method, you should specify what would be meaningful according to your parameters: not merely parroting. If and when that is achieved by LLMs, instead of moving the goalposts, accept that something genuine has happened. >> We will be flooded with content that will get harder and harder to prove wrong, but it will be wrong. I mostly agree, especia…

Exactly. Or to put it differently: Is "GPT-4 is just a parrot" a falsifiable claim? If so, what does the test look like that would falsify it?

I guess debugability will be its most desired feature soon. Let's see what copyrights will do with it.

Re: The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

#478

Post by Gigachad »

yunohn wrote: post #375
Gigachad wrote: post #260

Earlier quoted context omitted.

I feel like there would be a good chunk of real humans who would be incapable of answering a question like this.

The point of using a computer/AI is enable predictable and/or factual output. It’s not really a useful gotcha to say that “most humans would get this wrong”…

There is no "point" it's just a technology. My point was that the OP question was actually very difficult to the point many people could not handle it. So the fact an AI can not is not entirely surprising.

Re: The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

#479

Post by travisjungroth »

nuancebydefault wrote: post #467
travisjungroth wrote: post #379

Earlier quoted context omitted.

Can you share the script? If you do, no need for the standard “I need to clean it up” and so on apologies.

Why don't you give gpt4 a chance for an hour or so and try it out with handing it a handfull of problems. I swear, you will not be disappointed.

I’ve spent a bunch of hours!

Re: The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

#480

Post by travisjungroth »

computerex wrote: post #470
travisjungroth wrote: post #379

Earlier quoted context omitted.

Can you share the script? If you do, no need for the standard “I need to clean it up” and so on apologies.

Here you go: https://gist.github.com/computerex/c3394a88098895d8731501700...

This is awesome, thank you. I’ve been building single purpose stuff and thinking about tying it together. Seeing someone’s one-off (not a framework) take on that is super helpful!
Post reply on HN