Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

thesequence.substack.com

Post reply on HN
Print view View original

31–40 of 527 posts

Re: The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

#31

Post by anonzzzies »

matrix_overload wrote: post #14
whimsicalism wrote: post #11

It appears there is this genre of articles pretending that LLAMA or its RL-HF tuned variants are somehow even close to an alternative to ChatGPT. Spending more than a few moments interacting even with the larger instruct-tuned variants of these models quickly dispels that idea. Why do these takes around open-source AI remain so popular? What is the driving force?

ChatGPT being an ultra-hot topic, so every article tangentially related to it gets twice the views?

It is vastly better than anything else so far though. The rest will catch up but openai is not sleeping and they are well funded.

Re: The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

#32

Post by harrylove »

gleenn wrote: post #29
rrauenza wrote: post #9

Earlier quoted context omitted.

I’ve been googling trying to figure out what “ghost light” is in this context .. did you get an autocorrect for gas light?

Looks like they meant "gaslight" but I did find it on Urban Dictionary: ghost light Lighting in a video game that has no apparent source for the light to come from. Its like going out on a bright day, but not being able to find the sun in the sky even though the surroundings are brightly lit. Dead Rising on XBOX is a good example. http://ghost-light.urbanup.com/2450357

Agree on gaslight as the intended word. Ghost light also has a theatrical origin, still in use today. https://en.m.wikipedia.org/wiki/Ghost_light_(theatre)

Re: The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

#33

Post by jhbadger »

kmeisthax wrote: post #8
imjonse wrote: post #4

This makes is sound as if the Stanford and Berkeley teams also benefited from the leak, whereas I doubt they didn't have official access. So Alpaca/Vicuna/Koala projects would have probably happened anyway. The leak helped with popularity and demand and also somewhat positive PR for Meta, which makes me think they do not mind the leak that much.

Meta is actively trying to take down publicly available copies of LLaMA: https://github.com/github/dmca/blob/master/2023/03/2023-03-2...

Given that free alternatives like Vicuna (from the University of California and CMU) are better than LLaMA, are freely and legally available for download, and are compatible with code like llama.cpp, even if every copy of LLaMA is taken down it will have no effect on the development of chatbots. It might even improve things as people who would otherwise go for the better known LLaMA will move towards these newer, better, models.

Re: The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

#34

Post by [deleted] »

adeon wrote: post #10

I'm a bit worried the LLaMA leak will make the labs much more cautious about who they distribute models to for future projects, closing down things even more. I've had tons of fun implementing LLaMA, learning and playing around with variations like Vicuna. I learned a lot and probably wouldn't have got so interested in this space if the leak didn't happen.

[deleted]

Re: The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

#35

Post by seba_dos1 »

rcxdude wrote: post #24
smoldesu wrote: post #7

Earlier quoted context omitted.

> It is convinced that it is always factually accurate, even though it is not. I don't think that's true. ChatGPT (or any LLM) isn't convinced much of anything. It might present something confidently (which is what most people want) but that's a side-effect of it's programming, not an indication of how good it feels on the answer. If you reply to anything ChatGPT says with "No, you're wrong." it will try to write a n…

There's been quite a few different iterations of ChatGPT and bing with different behaviours in this regard: it depends somewhat on the base GPT version, the fine-tuning, and the prompt. Bing very famously at one point was extremely passive aggressive when challenged on basically anything. And while there's nothing intrinsic to the structure and training goals of LLMs which directs them towards more structured reasoni…

> Bing very famously at one point was extremely passive aggressive when challenged on basically anything.

It still wasn't an indication of how confident it "felt" with its answers. It was just role-playing a more confident and aggressive chat bot than ChatGPT does.

Re: The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

#36

Post by seydor »

lagniappe wrote: post #22
adeon wrote: post #10

I'm a bit worried the LLaMA leak will make the labs much more cautious about who they distribute models to for future projects, closing down things even more. I've had tons of fun implementing LLaMA, learning and playing around with variations like Vicuna. I learned a lot and probably wouldn't have got so interested in this space if the leak didn't happen.

An alternative interpretation was the LLaMa leak was an effort to shake or curtail the progress of ChatGPT's viral dominance at the time.

"And as long as they’re going to steal it, we want them to steal ours. They’ll get sort of addicted, and then we’ll somehow figure out how to collect sometime in the next decade".

That was ironically Bill Gates

https://www.latimes.com/archives/la-xpm-2006-apr-09-fi-micro...

Re: The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

#37

Post by jhbadger »

anonzzzies wrote: post #31
matrix_overload wrote: post #14

Earlier quoted context omitted.

ChatGPT being an ultra-hot topic, so every article tangentially related to it gets twice the views?

It is vastly better than anything else so far though. The rest will catch up but openai is not sleeping and they are well funded.

I thought that was the case before trying Vicuna. I agree that LLaMA and Alpaca are inferior to ChatGPT but I'm really not sure Vicuna is. It even (unfortunately) copies some of ChatGPT's quirks, like getting prudish when asking it to write a love scene ("It would not be appropriate for me to write...")

Re: The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

#38

Post by abstractbill »

irrational wrote: post #6

> OpenAI published a detailed blog post outlining some of the principles used to ensure safety in their models. The post emphasize in areas such as privacy, factual accuracy Am I the only one amused by the phrase “factual accuracy”? How many stories have we read like the one where it tries to ghost light the guy that this year is actually last year. “Oh, your phone must be wrong too, because there is no way I could b…

The models are a lot of fun to play with, but yeah, every time I've tried to use them for something "serious" they nearly always invent stuff (and are so convincing in how they write about it!).

Most recently I've been interested in what's happened with the 4-color theorem since the 1976 computer-assisted proof, and decided to use GPTChat instead of google+wikipedia. GPTChat had me convinced and excited that, apparently the computer-assisted part of the proof has been getting steadily smaller and smaller over the years and decades, and we're getting close to a proof that might not need computer assistance at all. It wrote really convincingly about it! And then I went and looked for the papers it had talked about. They didn't exist, and their authors either didn't exist, or worked in completely unrelated fields.

Re: The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

#39

Post by seydor »

Someone needs to legally challenge openAI on using the output of their models to train other commercial models. If web scraping is legal, then this must be legal too , even if openAI tries to curtail it. After all it was all trained on data they don't have rights to.

Re: The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

#40

Post by oliwarner »

adeon wrote: post #10

I'm a bit worried the LLaMA leak will make the labs much more cautious about who they distribute models to for future projects, closing down things even more. I've had tons of fun implementing LLaMA, learning and playing around with variations like Vicuna. I learned a lot and probably wouldn't have got so interested in this space if the leak didn't happen.

On the other side of the coin, they've distracted a huge amount of attention from OpenAI and have open source optimisations appearing for every platform they could ever consider running it on, for no extra expense.

If it was a deliberate leak, it was a good idea.

Post reply on HN