Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

thesequence.substack.com

Post reply on HN
Print view View original

21–30 of 527 posts

Re: The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

#21

Post by groffee »

moonchrome wrote: post #20
smoldesu wrote: post #7

Earlier quoted context omitted.

> It is convinced that it is always factually accurate, even though it is not. I don't think that's true. ChatGPT (or any LLM) isn't convinced much of anything. It might present something confidently (which is what most people want) but that's a side-effect of it's programming, not an indication of how good it feels on the answer. If you reply to anything ChatGPT says with "No, you're wrong." it will try to write a n…

>Everything it reads is mapped into language, not concept space Umm I'm pretty sure it's discovered concepts through compressing text - it seems perfectly capable of generalizing concepts

[dead]

Re: The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

#22

Post by lagniappe »

adeon wrote: post #10

I'm a bit worried the LLaMA leak will make the labs much more cautious about who they distribute models to for future projects, closing down things even more. I've had tons of fun implementing LLaMA, learning and playing around with variations like Vicuna. I learned a lot and probably wouldn't have got so interested in this space if the leak didn't happen.

An alternative interpretation was the LLaMa leak was an effort to shake or curtail the progress of ChatGPT's viral dominance at the time.

Re: The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

#23

Post by elcomet »

adeon wrote: post #10

I'm a bit worried the LLaMA leak will make the labs much more cautious about who they distribute models to for future projects, closing down things even more. I've had tons of fun implementing LLaMA, learning and playing around with variations like Vicuna. I learned a lot and probably wouldn't have got so interested in this space if the leak didn't happen.

They clearly expected the leak, they distributed it very widely to researchers. The important thing is the licence, not the access: you are not allowed to use it for commercial purpose.

Re: The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

#24

Post by rcxdude »

smoldesu wrote: post #7
irrational wrote: post #6

> OpenAI published a detailed blog post outlining some of the principles used to ensure safety in their models. The post emphasize in areas such as privacy, factual accuracy Am I the only one amused by the phrase “factual accuracy”? How many stories have we read like the one where it tries to ghost light the guy that this year is actually last year. “Oh, your phone must be wrong too, because there is no way I could b…

> It is convinced that it is always factually accurate, even though it is not. I don't think that's true. ChatGPT (or any LLM) isn't convinced much of anything. It might present something confidently (which is what most people want) but that's a side-effect of it's programming, not an indication of how good it feels on the answer. If you reply to anything ChatGPT says with "No, you're wrong." it will try to write a n…

There's been quite a few different iterations of ChatGPT and bing with different behaviours in this regard: it depends somewhat on the base GPT version, the fine-tuning, and the prompt. Bing very famously at one point was extremely passive aggressive when challenged on basically anything.

And while there's nothing intrinsic to the structure and training goals of LLMs which directs them towards more structured reasoning, it does seem that in the process of learning to predict language they pick up some degree of it, however flawed it may be.

Re: The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

#25

Post by musha68k »

moonchrome wrote: post #20
smoldesu wrote: post #7

Earlier quoted context omitted.

> It is convinced that it is always factually accurate, even though it is not. I don't think that's true. ChatGPT (or any LLM) isn't convinced much of anything. It might present something confidently (which is what most people want) but that's a side-effect of it's programming, not an indication of how good it feels on the answer. If you reply to anything ChatGPT says with "No, you're wrong." it will try to write a n…

>Everything it reads is mapped into language, not concept space Umm I'm pretty sure it's discovered concepts through compressing text - it seems perfectly capable of generalizing concepts

Would be interesting to read some unbiased science on that.

My hunch would be that any concept it might detect still depends on 2nd order patterns acquired through training?

The ultimate “book smart” at best, basically.

I’m not a ML scientist though, again would be interesting to read an unbiased evaluation of those aspects.

Edit: downvoted, not sure why though. Isn’t this a valid question a lot of non-ML folks would ask themselves?

Re: The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

#26

Post by echelon »

adeon wrote: post #10

I'm a bit worried the LLaMA leak will make the labs much more cautious about who they distribute models to for future projects, closing down things even more. I've had tons of fun implementing LLaMA, learning and playing around with variations like Vicuna. I learned a lot and probably wouldn't have got so interested in this space if the leak didn't happen.

If the copyright office determines model weights are uncopyrightable (huge if), then one might imagine any institutional leak would benefit everyone else in the space.

You might see hackers, employees, or contractors leaking models more frequently.

And since models are distilled functionality (no microservices and databases to deploy), they're much easier to run than a constellation of cloud infrastructure.

Re: The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

#27

Post by pingwing »

0xDEF wrote: post #5

The "leak" is being portrayed as something highly subversive done by the darn 4chan hackers. Before the "leak" Meta was sending the model to pretty much anyone who claimed to be a PhD student or researcher and had a credible college email. Meta has probably been planning to release the model sooner than later. Let's hope they release it under a true open source license.

Doesn't surprise me that Meta wants everyone to install their code onto their machines, lol.

Re: The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

#28

Post by whimsicalism »

0xDEF wrote: post #5

The "leak" is being portrayed as something highly subversive done by the darn 4chan hackers. Before the "leak" Meta was sending the model to pretty much anyone who claimed to be a PhD student or researcher and had a credible college email. Meta has probably been planning to release the model sooner than later. Let's hope they release it under a true open source license.

Feel like if they cared they would have at least watermarked the weights in some way but maybe I'm an idiot.

Re: The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

#29

Post by gleenn »

rrauenza wrote: post #9
irrational wrote: post #6

> OpenAI published a detailed blog post outlining some of the principles used to ensure safety in their models. The post emphasize in areas such as privacy, factual accuracy Am I the only one amused by the phrase “factual accuracy”? How many stories have we read like the one where it tries to ghost light the guy that this year is actually last year. “Oh, your phone must be wrong too, because there is no way I could b…

I’ve been googling trying to figure out what “ghost light” is in this context .. did you get an autocorrect for gas light?

Looks like they meant "gaslight" but I did find it on Urban Dictionary:

ghost light

Lighting in a video game that has no apparent source for the light to come from. Its like going out on a bright day, but not being able to find the sun in the sky even though the surroundings are brightly lit. Dead Rising on XBOX is a good example.

http://ghost-light.urbanup.com/2450357

Re: The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

#30

Post by drowsspa »

0xDEF wrote: post #5

The "leak" is being portrayed as something highly subversive done by the darn 4chan hackers. Before the "leak" Meta was sending the model to pretty much anyone who claimed to be a PhD student or researcher and had a credible college email. Meta has probably been planning to release the model sooner than later. Let's hope they release it under a true open source license.

It sounds like that king that wanted people to overcome their aversion for potatoes. So he put armed guards around the potato fields but instructed them to be very lax and allowed the people to rob it
Post reply on HN