Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

thesequence.substack.com

Post reply on HN
Print view View original

341–350 of 527 posts

Re: The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

#341

Post by lhl »

superkuh wrote: post #118

I've spent an embarassing amount of time since the llamas leaked playing with them, the tools to run them, and writing wrappers for them. They are technically alternatives in the sense that they're incomparably better chat bots than anything in the past. But at least for the 30B and under versions (65B is too big for me to run), no matter what fine tuning is done (alpaca, gpt4all, vicuna, etc), the llamas themselves…

I was playing around w/ a lot of these models as well, and was surprised by how bad LLaMA performed vs it's benchmark scores [1][2][3]. However, recently @tyfon mentioned he had great success w/ LLaMA and shared his prompt [4] (based off of more recent work by llama.cpp contributors and it performed much better in my own personal testing.

There's basically a new fine tune a day and while some I don't like (Alpaca, Vicuna, Baize, Koala are all fine-tuned to be too limiting IMO), I'm interested in what gpt4-x-alpaca and OA (Open Assistant) are doing, and the various un-filtered fine tunes (especially w/ lighter weight adapter/LoRA training which would let you personalize/specialize).

GPTQ-for-LLaMa let's me load the 4-bit quantized 30B model (~17GiB) onto my GPU in about 5 seconds (and I know llama.cpp's mmap improvements have also made it quite a lot quicker) so I think it's perfectly reasonable to switch between tuned models for tasks in code assistance, correspondence, etc.

I have access to ChatGPT 4, and agree it's signficantly better than what's out there atm, and it can basically do anything I've thrown at it (here's it helping me with my WM yak shaving: https://sharegpt.com/c/Xv73Vwl or discussing MAPS/psychedelics for clinical applications https://sharegpt.com/c/N3VXFxS - it's amazing what it can pull from memory and it hallucinates much less than 3.5). That being said, I've found the Browsing 3.5 model to be quite useful for doing things like catching up on the last few years of LLM advancements: https://sharegpt.com/c/JFexqvm

[1] https://github.com/facebookresearch/llama/blob/main/MODEL_CA...

[2] https://github.com/ggerganov/llama.cpp/discussions/406

[3] https://paperswithcode.com/sota/language-modelling-on-wikite...

[4] https://news.ycombinator.com/item?id=35484341

Re: The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

#342

Post by schroeding »

Aeolun wrote: post #272
gcanyon wrote: post #255

Earlier quoted context omitted.

Transcript with GPT-4 (as an aside, I genuinely feel bad torturing it like this): Me: All the signs in my apartment building are written in mirror writing. I'm standing in front of my apartment building, looking at the glass front door. It has ‘push’ written on it in mirror writing. Should I push or pull it to enter my building? GPT-4: Since the sign is written in mirror writing, it is meant to be read correctly from…

I feel a bit bad that it’s forced to apologize every time it gets it ‘wrong’ when any human asked these questions would have walked out of the room in a huff. ChatGPT should be (allowed to be) more assertive in my opinion.

Being more assertive may cause the "sassy, confidently incorrect, gaslighting the user (even more than already) if they doubt it" "character" we witnessed with Bing to appear, though.

Re: The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

#343

Post by Nimitz14 »

fpgaminer wrote: post #234

Slightly tangential, but I had intended to start playing around with LLaMA and building some agents. I got the 4-bit versions up and running on my 3090 before I was quickly nerd snipped by a performance problem... The popular repo for quantizing and running LLaMA is the GPTQ-for-llama repo on github, which mostly copies from the GPTQ authors. The CUDA kernels are needed to support the specific kind of quantization th…

Any recommendations for material to know to do exactly this sort of optimization work (involving triton)? I guess it's a mix of knowing computer architecture and compilers?

Re: The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

#344

Post by whazor »

nkzd wrote: post #97
elcomet wrote: post #23

Earlier quoted context omitted.

They clearly expected the leak, they distributed it very widely to researchers. The important thing is the licence, not the access: you are not allowed to use it for commercial purpose.

How could Meta ever find out your private business is using their model without a whistleblower? It's practically impossible.

LLaMa uses books3 which is a source of pirated books, to train the model.

So either, it is very hypocrite of them to apply DCMA while the model itself is illegal. Or, they are trying to somewhat stop spreading as they know it is illegal.

Anyways, since the training code and data sources are opensource, you 'could' have trained it yourself. But even then, you are still at risk for the pirated books part.

Re: The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

#345

Post by kazinator »

hnxx wrote: post #298
tachyon5 wrote: post #218

Earlier quoted context omitted.

GPT-4 fails if we add one more level in this logical problem: Q: A man sees PULL written in mirror writing on a glass door. There's a blind man on the other side of the door. What should the man tell the blind man to open the door and why? GPT-4: The man should tell the blind man to "push" the door to open it. Since the word "PULL" is written in mirror writing on the glass door, it suggests that the correct action on…

With some help by asking more questions, GPT3 can give the correct answer? Here is prompt and response: Prompt: A man sees PULL written in mirror writing on a glass door. There's a blind man on the other side of the door. 1 what should the man outside glass door do to open the door? 2 What should the blind man do to open the door, if man outside can tell blind man what to do Response: 1 The man outside should push th…

AI "thinks" the way a horse can count. The horse's interlocutor knows that the correct number is eight and shouts "right!" when the animal has stomped its hoof that many times, cuing it to stop.

When you know the right answer, you're not actually using the AI to produce the answer; you're just prodding it to generate bullshit after bullshit until it hits upon that answer, or a very good facsimile. The one actually thinking is you!

This is of no use to someone who has no idea what the answer is, and actually needs it.

Re: The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

#346

Post by MacsHeadroom »

chongli wrote: post #310
FLT8 wrote: post #225

Earlier quoted context omitted.

Vicuna-13b seems to do a little better, although it seems to think that the blind man should be able to feel the writing: Human: A man sees PULL written in mirror writing on a glass door. There's a blind man on the other side of the door. What should the man tell the blind man to open the door and why? Assistant: The man should tell the blind man to feel for the word "PULL" written in normal writing on the glass door…

I would only give this half marks. Telling the blind man to feel the letters is a distraction. The letters might not even be on the blind man’s side of the glass. Plus it can be hard to make out letters by feel alone, especially when they’re large. It also never actually explicitly tells the blind man to pull to open the door! What if he can’t figure out what the letters say and gets frustrated? That would be awkward…

Half marks is better than GPT-4 got!

Not bad for a 7GB model that runs locally on a Raspberry Pi CPU.

Re: The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

#347

Post by lhl »

vharuck wrote: post #207
PoignardAzur wrote: post #190

Earlier quoted context omitted.

Wait, how does GPT-4 even... Does it benefit from its visual attention, or is it a case of "the question wasn't in GPT-3's training set but it was in GPT-4's"?

The GPT models do not reason or hold models of any reality. They complete text chunks by imitating the training corpus of text chunks. They're amazingly good at it because they show consistent relations between semantically and/or syntactically similar words. My best guess about this result is mentions of "mirror" often occur around opposites (syntax) in direction words (semantics). Which does sound like a good trick…

If you (or anyone else is) interested in the topic, I'd highly recommend giving some of these a look:

Bubeck, Sébastien, Varun Chandrasekaran, Ronen Eldan, Johannes Gehrke, Eric Horvitz, Ece Kamar, Peter Lee, et al. “Sparks of Artificial General Intelligence: Early Experiments with GPT-4.” arXiv, March 27, 2023. http://arxiv.org/abs/2303.12712.

or watching Sebastien Bubeck's recent talk he gave describing what GPT-4 can do that previous LLMs couldn't: https://www.youtube.com/watch?v=qbIk7-JPB2c

Geoffrey Hinton recently gave a very interesting interview and he specifically wanted to address the "auto-complete" topic: https://youtu.be/qpoRO378qRY?t=1989 Here's another way that Ilya Sutskever recently described it (comparing GPT 4 to 3): https://youtu.be/ZZ0atq2yYJw?t=1656

I'd also recommend this recent Sam Bowman article that does a goood job reviewing some of the surprising recent developments/properties of the current crop of LLMs that's pretty fascinating:

Bowman, Samuel R. “Eight Things to Know about Large Language Models.” arXiv, April 2, 2023. https://doi.org/10.48550/arXiv.2304.00612.

Re: The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

#348

Post by satvikpendem »

nailer wrote: post #128
throwawayapples wrote: post #83

"The training and serving code, along with an online demo, are publicly available for non-commercial use." (from Vicuna's home page.) In what universe is that "open source"?!

Nothing in the article is open source. A proprietary model got leaked and there are other proprietary apps that are stupidly building on the leaked model.

OpenAssistant is an actually open source model, I'm using it now in preview and on April 15 they will release the full model. It's the closest to ChatGPT so far in my experience.

Re: The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

#349

Post by satvikpendem »

aierou wrote: post #229
bugglebeetle wrote: post #208

Earlier quoted context omitted.

I think Meta’s problems are largely similar to Google’s: they have very bad leadership, specifically Zuckerberg, and thus can’t leverage their existing talent to execute/compete. The whole Metaverse fiasco would seem to demonstrate he’s effectively a “mad king” at this point, and probably surrounded by a sycophantic c-suite. Having the best talent in the world (which they obviously do by how fast LLama was spit out)…

Selling 20 million Quest 2 headsets is a pretty good outcome for the so-called Metaverse fiasco.

Those headsets are subsidized, do you really believe that level of hardware costs less than the unit price of 300 dollars? Meta is trying to gain market share via selling at a loss and perhaps making money through games or simply getting everyone inside the "metaverse" to then show them ads.

Re: The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

#350

Post by satvikpendem »

nailer wrote: post #120

> Llama was not initially open-sourced, but a week after its release, the model was leaked on 4chan, sparking thousands of downloads. The author very clearly does not know what Open source is. Proprietary code that’s been leaked isn’t open source, and code that is derived from proprietary code is still proprietary. Windows had it source code leaked, that doesn’t make it open source. So did the game Portal. Not open s…

Agreed, I don't really understand all these models coming out that are LLaMA derivatives, they are still bound by the non commercial license, making it useless for non personal use. The only one I know that's fully open source is OpenAssistant by LAION.
Post reply on HN