Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

thesequence.substack.com

Post reply on HN
Print view View original

201–210 of 527 posts

Re: The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

#201

Post by PoignardAzur »

Method-X wrote: post #200
PoignardAzur wrote: post #190

Earlier quoted context omitted.

Wait, how does GPT-4 even... Does it benefit from its visual attention, or is it a case of "the question wasn't in GPT-3's training set but it was in GPT-4's"?

It has the ability to reason. It may not be conscious, but it is intelligent.

That's not an answer.

The given question is one which requires some spatial reasoning to understand. By default, GPT can only understand spatial questions as described by text tokens which is a pretty noisy channel. So it's not obvious how GPT-4 could answer a spatial reasoning question (aside from memorizing it).

Re: The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

#202

Post by steveBK123 »

hutzlibu wrote: post #199
PoignardAzur wrote: post #190

Earlier quoted context omitted.

Wait, how does GPT-4 even... Does it benefit from its visual attention, or is it a case of "the question wasn't in GPT-3's training set but it was in GPT-4's"?

I would suspect, that this is one of the manual fine tuned questions. Meaning in before versions people used this question to show flaws and now this specific flaw is fixed. Otherwise it would be indeed reasoning in my understanding.

The evolution of answers from version to version makes it clear there are insane amounts of manual fine tunings happening. I think this is largely overlooked by the "its learning" crowd.

Re: The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

#203

Post by lolinder »

dandongus wrote: post #156
lolinder wrote: post #154

Earlier quoted context omitted.

A token per second-ish with a Ryzen 7 5800X. If I run it for too long it gets slower as heat throttling kicks in, I need a better cooling system if I'm going to run it non-stop.

For a bit of comparison, if you've tested, how fast are 13B or 7B on the same setup?

Really fast. I didn't bother timing, but they're faster than ChatGPT by a long shot. I didn't spend very long with them because the quality is so much worse than the 65B.

I should probably go back and try again to see if it's worth it for the extra speed, now that I've played with 65B for a while.

Re: The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

#204

Post by tel »

PoignardAzur wrote: post #190
jonplackett wrote: post #162

Earlier quoted context omitted.

The difference between 3.5 and 4 is gigantic even in my fairly limited experience. I gave them both some common sense tests and this one stuck out to me. Q: A glass door has ‘push’ written on it in mirror writing. Should you push or pull it GPT-3.5: If the word "push" is written in mirror writing on a glass door, you should push the door to open it GPT-4: Since the word "push" is written in mirror writing, it suggest…

Wait, how does GPT-4 even... Does it benefit from its visual attention, or is it a case of "the question wasn't in GPT-3's training set but it was in GPT-4's"?

I think it's hard to deny that it's doing some level of reasoning. It's quite clear that these models do not merely echo elements of their training data and that they can solve simple and novel puzzles.

What that reasoning is, exactly, is hard to know. One can suppose that ideas like "glass", "transparent", "mirror" are all reasonable concepts that show up in the training set and are demonstrated thoroughly

Re: The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

#205

Post by Method-X »

PoignardAzur wrote: post #201
Method-X wrote: post #200

Earlier quoted context omitted.

It has the ability to reason. It may not be conscious, but it is intelligent.

That's not an answer. The given question is one which requires some spatial reasoning to understand. By default, GPT can only understand spatial questions as described by text tokens which is a pretty noisy channel. So it's not obvious how GPT-4 could answer a spatial reasoning question (aside from memorizing it).

This is a good explanation: https://www.youtube.com/watch?v=qbIk7-JPB2c

Re: The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

#206

Post by tel »

steveBK123 wrote: post #202
hutzlibu wrote: post #199

Earlier quoted context omitted.

I would suspect, that this is one of the manual fine tuned questions. Meaning in before versions people used this question to show flaws and now this specific flaw is fixed. Otherwise it would be indeed reasoning in my understanding.

The evolution of answers from version to version makes it clear there are insane amounts of manual fine tunings happening. I think this is largely overlooked by the "its learning" crowd.

This is an interesting argument as it's easy to apply it nearly universally to any example of learning.

What sort of evidence would convince you that it is learning?

Re: The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

#207

Post by vharuck »

PoignardAzur wrote: post #190
jonplackett wrote: post #162

Earlier quoted context omitted.

The difference between 3.5 and 4 is gigantic even in my fairly limited experience. I gave them both some common sense tests and this one stuck out to me. Q: A glass door has ‘push’ written on it in mirror writing. Should you push or pull it GPT-3.5: If the word "push" is written in mirror writing on a glass door, you should push the door to open it GPT-4: Since the word "push" is written in mirror writing, it suggest…

Wait, how does GPT-4 even... Does it benefit from its visual attention, or is it a case of "the question wasn't in GPT-3's training set but it was in GPT-4's"?

The GPT models do not reason or hold models of any reality. They complete text chunks by imitating the training corpus of text chunks. They're amazingly good at it because they show consistent relations between semantically and/or syntactically similar words.

My best guess about this result is mentions of "mirror" often occur around opposites (syntax) in direction words (semantics). Which does sound like a good trick question for these models.

Re: The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

#208

Post by bugglebeetle »

oezi wrote: post #196
bugglebeetle wrote: post #93

Earlier quoted context omitted.

I would assume so. Meta’s ML/AI team is very strong, but they probably don’t have a comparable product offering to ChatGPT ready for public use. So instead, they bought themselves some time by letting the open source community run wild with a lesser model and eat into OpenAI’s moat.

What would you think is holding back Meta and Google? Why can't they just scale up the compute they throw at the problem? What are they tinkering on?

I think Meta’s problems are largely similar to Google’s: they have very bad leadership, specifically Zuckerberg, and thus can’t leverage their existing talent to execute/compete. The whole Metaverse fiasco would seem to demonstrate he’s effectively a “mad king” at this point, and probably surrounded by a sycophantic c-suite. Having the best talent in the world (which they obviously do by how fast LLama was spit out) isn’t going to matter that much if its all serving at the behest of someone who has become deluded by their initial success and has no ability to course correct.

Re: The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

#209

Post by WalterBright »

jacquesm wrote: post #51
EGreg wrote: post #45

Earlier quoted context omitted.

I am shocked that it speaks the way it does when it was trained on random stuff it doesn’t have rights to. They say they trained it on databases they had bought access to etc. And it seems that way. Because how does ChatGPT: 1. Do what you ask instead of continuing your instructions? 2. Use such nice and helpful language as opposed to just random average of what people say? 3. And most of all — how does it have a str…

There is a lot of massaging of inputs and outputs but at the same time: that's done by tweaking the model reinforcing those parts that are desirable and suppressing those parts that are not, not by rewriting the output, though there may be filters that check for 'forbidden fruits'. And it isn't the 'random average' of what people say, that would give you junk, the whole idea is that it tries to get to something bette…

I'd like the version without the "sanitized for my protection" part. I cannot imagine being offended by what a computer program generates.

Re: The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

#210

Post by tel »

nkzd wrote: post #97
elcomet wrote: post #23

Earlier quoted context omitted.

They clearly expected the leak, they distributed it very widely to researchers. The important thing is the licence, not the access: you are not allowed to use it for commercial purpose.

How could Meta ever find out your private business is using their model without a whistleblower? It's practically impossible.

Have reasonable suspicion, sue you, and then use discovery to find any evidence at all that your models began with LLaMA. Oh, you don't have substantial evidence for how you went from 0 to a 65B-parameter LLM base model? How curious.
Post reply on HN