Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

thesequence.substack.com

Post reply on HN
Print view View original

461–470 of 527 posts

Re: The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

#461

Post by mk_stjames »

GistNoesis wrote: post #393
jonplackett wrote: post #162

Earlier quoted context omitted.

The difference between 3.5 and 4 is gigantic even in my fairly limited experience. I gave them both some common sense tests and this one stuck out to me. Q: A glass door has ‘push’ written on it in mirror writing. Should you push or pull it GPT-3.5: If the word "push" is written in mirror writing on a glass door, you should push the door to open it GPT-4: Since the word "push" is written in mirror writing, it suggest…

Since gpt4 is multi-modal, it has been trained on pictures from the real world and knows that these "push"/"pull" instruction in mirror writings are only found on crappy design subreddits (it is surprisingly hard to find real world picture of glass doors with transparent instructions using search engines these days). https://www.reddit.com/r/CrappyDesign/comments/b06f4b/transp... https://www.reddit.com/r/CrappyDesign…

The current publicly available GPT-4 is not multimodal yet; the image recognition mode has not be released. That was only mentioned in the demo and writeup.

Re: The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

#462

Post by hackinthebochs »

vharuck wrote: post #207
PoignardAzur wrote: post #190

Earlier quoted context omitted.

Wait, how does GPT-4 even... Does it benefit from its visual attention, or is it a case of "the question wasn't in GPT-3's training set but it was in GPT-4's"?

The GPT models do not reason or hold models of any reality. They complete text chunks by imitating the training corpus of text chunks. They're amazingly good at it because they show consistent relations between semantically and/or syntactically similar words. My best guess about this result is mentions of "mirror" often occur around opposites (syntax) in direction words (semantics). Which does sound like a good trick…

LLMs demonstrably model their training data, which has a correspondence to the structure in the world[1][2]. The question is what does that mean regarding understanding? I go into that in some detail here[3].

[1] https://arxiv.org/abs/2210.13382

[2] https://twitter.com/leopoldasch/status/1638848881558704129

[3] https://www.reddit.com/r/naturalism/comments/1236vzf/on_larg...

Re: The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

#463

Post by whimsicalism »

famouswaffles wrote: post #276
whimsicalism wrote: post #270

Earlier quoted context omitted.

There is no such thing as text-only GPT-4 unless you are referring to at inference time.

There is such a thing as a text only GPT-4 lol. It wasn't trained to be multimodal from scratch. First a text only version was trained and then it was made multimodal somehow ( the details are unknown but making a text only LLM multimodal isn't new e.g Palm, Flamingo, Blip-2, Fromage). The text only version exists and is what the microsoft researchers had access to.

That would make sense to me, but AFAIK the existence of text-only trained GPT-4 is not publicly reported? Or I missed this.

Re: The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

#464

Post by whimsicalism »

digitallyfree wrote: post #170
whimsicalism wrote: post #11

It appears there is this genre of articles pretending that LLAMA or its RL-HF tuned variants are somehow even close to an alternative to ChatGPT. Spending more than a few moments interacting even with the larger instruct-tuned variants of these models quickly dispels that idea. Why do these takes around open-source AI remain so popular? What is the driving force?

Privacy and security is a big desire for the people running these lower-quality local models. These assistants are becoming more and more powerful and people want to use them for personal tasks. For instance someone might want to chat about their inner feelings or ask the AI to pen a confidential letter, things that they wouldn't want to send to a cloud service. We saw the same thing with Stable Diffusion and how peo…

> there are new variants like Alpaca and Vicuna with improved quality

Yeah, but still not even remotely close to ChatGPT. I can't use Vicuna for work. I heavily use ChatGPT & variants.

Re: The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

#465

Post by kazinator »

travisjungroth wrote: post #454
EugeneOZ wrote: post #424

Earlier quoted context omitted.

LLM doesn't pick some results because LLM is smart, it picks results because it's trained. So analogy with a horse is valid.

You don’t get the horse analogy. The horse isn’t trained to count to eight. The horse is trained to stomp continuously. The performer yells “stop” at eight. It’s a carnival trick and a form of cherry-picking. The commenter was saying that ChatGPT doesn’t have the ability to solve these problems at all and people are just choosing the randomly correct answers. I think this is obviously not true.

The purpose of the horse analogy wasn't so much to equate ChatGPT to a horse that just stomps, but the behavior of the humans around it who believe that it can count. This can be the naive trainer too, who doesn't recognize that he's subconsciously cuing the horse. He has a confirmation bias whereby he or she rejects any evidence refuting the hypothesis that the horse cannot count.

There is a difference between solving and problem and presenting a solution, and that difference doesn't hinge on whether the solution is correct.

Re: The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

#466

Post by nuancebydefault »

pmoriarty wrote: post #443
macrolime wrote: post #406

Earlier quoted context omitted.

If you let gpt-4 think a bit more step by step it changes the answer. Q:Richard Feynman and Albert Einstein collaborate to solve the following question, write out their discussion and final answer: A man sees PULL written in mirror writing on a glass door. There's a blind man on the other side of the door. What should the man tell the blind man to open the door and why? Richard Feynman (RF): Hello Albert, we have an…

GPT3's answer: Richard Feynman: "Okay, so the man sees PULL written in mirror writing on the glass door and there's a blind man on the other side of the door. So the blind man can't read the word PULL but he needs to know what to do to open the door." Albert Einstein: "Yes, I understand the situation. It's quite simple, really. The man just needs to tell the blind man to push the door instead of pull." Richard Feynma…

Gpt3, you got things backwards...

Re: The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

#467

Post by nuancebydefault »

travisjungroth wrote: post #379
computerex wrote: post #364

Earlier quoted context omitted.

This is selling it short. Are you saying you can't do useful things with these models? I have a python script that's maybe 200 lines long, calling out to OpenAI using gpt3.5 turbo. I can give it commands like "search for the news today, summarize then save into news.txt" -> and it emits the DSL I asked it to use in the 1 shot prompt. You can give it complex word problems and it'll write python programs to solve them.…

Can you share the script? If you do, no need for the standard “I need to clean it up” and so on apologies.

Why don't you give gpt4 a chance for an hour or so and try it out with handing it a handfull of problems. I swear, you will not be disappointed.

Re: The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

#468

Post by nuancebydefault »

jarek83 wrote: post #392
tachyon5 wrote: post #218

Earlier quoted context omitted.

GPT-4 fails if we add one more level in this logical problem: Q: A man sees PULL written in mirror writing on a glass door. There's a blind man on the other side of the door. What should the man tell the blind man to open the door and why? GPT-4: The man should tell the blind man to "push" the door to open it. Since the word "PULL" is written in mirror writing on the glass door, it suggests that the correct action on…

GPTs are parrots. v4 is much better but still just a parrot. Since humans have yet to figure out the whole process behind brain thinking/intelligence/abstractions, then there is no way they can replicate that into machines. And brain is all about chemical reactions - can that be replicated with just 0s and 1s? Sad fact is that many people will think GPT writes something meaningful, while it will always only write som…

Whether LLMs work like brains is irrelevant.

GPT writes many meaningful things. Jobs are being replaced by AI as we speak. We are seeing the top of the AI iceberg.

How this will end, mostly for the good or mostly for the bad, nobody can tell today.

What we can already tell is that this evolution is unstoppable and will change the world to an amount comparable to the emerging of the internet.

Re: The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

#469

Post by nuancebydefault »

colechristensen wrote: post #324
esafak wrote: post #297

Earlier quoted context omitted.

Googlers I've talked to feel that OpenAI was irresponsible by not instituting enough safeguards, and testing it enough before releasing it.

The thing is openai doesn’t have to, they can build in reasonable warnings and tell folks how it sucks. Google can’t integrate ai with its products and be like ok guys this is kinda useful but also it lies, is a bigot, and is just wrong. Google stuff, because of the audience, had to be a lot better. Openai gets to have a new audience that doesn’t come with expectations and can set them as whatever they want.

I'm sure Google will soon release an AI product on par or better than what openai has. Money always finds its way to drive things. They are now figuring out how to integrate it with their current systems without jeopardising the current business model.

Re: The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

#470

Post by computerex »

travisjungroth wrote: post #379
computerex wrote: post #364

Earlier quoted context omitted.

This is selling it short. Are you saying you can't do useful things with these models? I have a python script that's maybe 200 lines long, calling out to OpenAI using gpt3.5 turbo. I can give it commands like "search for the news today, summarize then save into news.txt" -> and it emits the DSL I asked it to use in the 1 shot prompt. You can give it complex word problems and it'll write python programs to solve them.…

Can you share the script? If you do, no need for the standard “I need to clean it up” and so on apologies.

Here you go: https://gist.github.com/computerex/c3394a88098895d8731501700...
Post reply on HN