Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

thesequence.substack.com

Post reply on HN
Print view View original

421–430 of 527 posts

Re: The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

#421

Post by muyuu »

wokwokwok wrote: post #334
muyuu wrote: post #141

Earlier quoted context omitted.

his conclusion is simultaneously not warranted and correct a like-for-like comparison would be GPT-4 against the larger models like LLaMA 65B, but those cannot be run on consumer-grade hardware so one ends up comparing the stuff one can run... against the top stuff from OpenAI running on high-end GPU farms, and this technology clearly benefits a lot still from much larger scale than most people can afford the great r…

The 65B model runs fine on a Mac Studio with 64GB of memory. The output is unremarkable; it’s not significantly better than the 13B model for most uses. GPT 3.5 is an order of magnitude better at least .

You cannot heavily quantise models and assume they didn't degrade significantly.

To run it properly you need a lot more than a Mac Studio, and then comparisons need to be done more or less seriously, not just a few random prompts, because anything in a black box will "cheat" and will be fine tuned to do well at popular benchmarks.

Re: The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

#422

Post by smoldesu »

seydor wrote: post #410
smoldesu wrote: post #356

Earlier quoted context omitted.

> Llammas are showing the world that it doesn't take monopoly-level hardware to run those things. LLaMA was not necessarily the model that did that. A fairer attribution might be BERT or GPT-Neo.

it was difficult to run all those models. now gamers follow youtube tutorials

Name 1 way GPT-Neo was harder to run than LLaMA.

Re: The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

#423

Post by BeefWellington »

tel wrote: post #204
PoignardAzur wrote: post #190

Earlier quoted context omitted.

Wait, how does GPT-4 even... Does it benefit from its visual attention, or is it a case of "the question wasn't in GPT-3's training set but it was in GPT-4's"?

I think it's hard to deny that it's doing some level of reasoning. It's quite clear that these models do not merely echo elements of their training data and that they can solve simple and novel puzzles. What that reasoning is, exactly, is hard to know. One can suppose that ideas like "glass", "transparent", "mirror" are all reasonable concepts that show up in the training set and are demonstrated thoroughly

> I think it's hard to deny that it's doing some level of reasoning.

Not really, the asker is doing the reasoning here in that they are presupposing there are two operations for the door: Push or Pull. All the answer engine is doing is simply outputting what sound like believable answers (which it's really good at).

Re: The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

#424

Post by EugeneOZ »

mandmandam wrote: post #404
EugeneOZ wrote: post #383

Earlier quoted context omitted.

Because you are using results of the training.

What human wasn't trained? Even a child raised by wolves gets training.

LLM doesn't pick some results because LLM is smart, it picks results because it's trained. So analogy with a horse is valid.

Re: The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

#428

Post by Nimitz14 »

fpgaminer wrote: post #367
Nimitz14 wrote: post #343

Earlier quoted context omitted.

Any recommendations for material to know to do exactly this sort of optimization work (involving triton)? I guess it's a mix of knowing computer architecture and compilers?

Triton itself is fairly "easy", at least as far as "low level optimization languages" go. It's just (restricted) python. If you know PyTorch, you can muddle your way through Triton. They have a few tutorials. Reading up on nvidia architectures, PTX, and CUDA are likely to improve your skill at Triton.

Yeah I've been able to kinda muddle my way through but progress is frustrating as I frequently get errors I don't understand. Was thinking maybe I should study compilers a bit and then I will be able to understand the triton source code better which will help me understand why I get errors.

Re: The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

#429

Post by BeFlatXIII »

cultureswitch wrote: post #418
Gigachad wrote: post #260

Earlier quoted context omitted.

I feel like there would be a good chunk of real humans who would be incapable of answering a question like this.

There's a significant portion of all human beings who ever lived who wouldn't partake in a hypothetical. They would just respond "I don't see that door" and go away thinking your question makes no sense.

…and their votes count just as much as yours.

Re: The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

#430

Post by kazinator »

mandmandam wrote: post #404
EugeneOZ wrote: post #383

Earlier quoted context omitted.

Because you are using results of the training.

What human wasn't trained? Even a child raised by wolves gets training.

A human isn't trained to program in Python by memorizing impressions of millions of examples correlated to a verbal description, from which he or she regurgitates mash-up that statistically correlates to the prompt, without having any idea what the content means.

Ask it to write the same program in some little known programming language. A human who knows how to code could do that by studying a small amount of reference material.

Post reply on HN