Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

thesequence.substack.com

Post reply on HN
Print view View original

441–450 of 527 posts

Re: The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

#441

Post by fl7305 »

jarek83 wrote: post #392
tachyon5 wrote: post #218

Earlier quoted context omitted.

GPT-4 fails if we add one more level in this logical problem: Q: A man sees PULL written in mirror writing on a glass door. There's a blind man on the other side of the door. What should the man tell the blind man to open the door and why? GPT-4: The man should tell the blind man to "push" the door to open it. Since the word "PULL" is written in mirror writing on the glass door, it suggests that the correct action on…

GPTs are parrots. v4 is much better but still just a parrot. Since humans have yet to figure out the whole process behind brain thinking/intelligence/abstractions, then there is no way they can replicate that into machines. And brain is all about chemical reactions - can that be replicated with just 0s and 1s? Sad fact is that many people will think GPT writes something meaningful, while it will always only write som…

> Since humans have yet to figure out the whole process behind brain thinking/intelligence/abstractions, then there is no way they can replicate that into machines.

By that logic, humans can't write a superhuman Chess/Go program unless they can articulate the specific algorithms to select the next move.

But that's clearly not true. Neural networks have been trained to play superhuman chess just by example. Not by programmers figuring out the whole process behind chess/go playing.

Re: The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

#442

Post by fl7305 »

lovemenot wrote: post #414
jarek83 wrote: post #392

Earlier quoted context omitted.

GPTs are parrots. v4 is much better but still just a parrot. Since humans have yet to figure out the whole process behind brain thinking/intelligence/abstractions, then there is no way they can replicate that into machines. And brain is all about chemical reactions - can that be replicated with just 0s and 1s? Sad fact is that many people will think GPT writes something meaningful, while it will always only write som…

This argument is imbalanced. Not necessarily wrong. To follow scientific method, you should specify what would be meaningful according to your parameters: not merely parroting. If and when that is achieved by LLMs, instead of moving the goalposts, accept that something genuine has happened. >> We will be flooded with content that will get harder and harder to prove wrong, but it will be wrong. I mostly agree, especia…

Exactly.

Or to put it differently: Is "GPT-4 is just a parrot" a falsifiable claim?

If so, what does the test look like that would falsify it?

Re: The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

#443

Post by pmoriarty »

macrolime wrote: post #406
tachyon5 wrote: post #218

Earlier quoted context omitted.

GPT-4 fails if we add one more level in this logical problem: Q: A man sees PULL written in mirror writing on a glass door. There's a blind man on the other side of the door. What should the man tell the blind man to open the door and why? GPT-4: The man should tell the blind man to "push" the door to open it. Since the word "PULL" is written in mirror writing on the glass door, it suggests that the correct action on…

If you let gpt-4 think a bit more step by step it changes the answer. Q:Richard Feynman and Albert Einstein collaborate to solve the following question, write out their discussion and final answer: A man sees PULL written in mirror writing on a glass door. There's a blind man on the other side of the door. What should the man tell the blind man to open the door and why? Richard Feynman (RF): Hello Albert, we have an…

GPT3's answer:

Richard Feynman: "Okay, so the man sees PULL written in mirror writing on the glass door and there's a blind man on the other side of the door. So the blind man can't read the word PULL but he needs to know what to do to open the door."

Albert Einstein: "Yes, I understand the situation. It's quite simple, really. The man just needs to tell the blind man to push the door instead of pull."

Richard Feynman: "Wait, but why push and not pull? The word on the door is PULL, not PUSH."

Albert Einstein: "Ah, but you see, the word is in mirror writing. When the blind man feels the letters with his fingers, he will feel the letters backwards, as if he is looking at them in a mirror. So he will actually feel the letters as L L U P, which is like saying 'push' backwards."

Richard Feynman: "Wow, Einstein, that's brilliant. So we just tell the blind man to push the door because he will feel the word PULL backwards."

Albert Einstein: "Indeed. It's a simple solution once you think about it."

Final answer: The man should tell the blind man to push the door because the word PULL is written in mirror writing on the glass door, and the blind man will feel the letters backwards as L L U P, which is like saying 'push' backwards.

Re: The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

#444

Post by Tostino »

avereveard wrote: post #217
morrbo wrote: post #160

Earlier quoted context omitted.

massive YMMV moment for me. my particular usecase was "extract the following attributes from a load of unstructured text, format the results as JSON". ChatGPT was the best (but only on 4 and Davinci), Vicuna just didn't perform at all (nor other variants of llama 7/13/33). Bard smashed it, relatively speaking, in terms of speed. I gave up pretty quickly though because of no information on pricing and/or API. It's fun…

On the smaller models you may want to split the task in smaller chunk either in parallel one value at a time or in sequence like extract the attributes, then take the output and ask to format it into a json

In relation to this, when using GPT4, I have added the addendum to my prompts: "This seems like a lot of work, please split the work into two chunks, and let's start on the first chunk now."

It will generally segment the problem in some logical way and work just fine, with vastly improved reasoning abilities due to not trying to do as much at once.

Re: The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

#445

Post by Tostino »

lhl wrote: post #341
superkuh wrote: post #118

I've spent an embarassing amount of time since the llamas leaked playing with them, the tools to run them, and writing wrappers for them. They are technically alternatives in the sense that they're incomparably better chat bots than anything in the past. But at least for the 30B and under versions (65B is too big for me to run), no matter what fine tuning is done (alpaca, gpt4all, vicuna, etc), the llamas themselves…

I was playing around w/ a lot of these models as well, and was surprised by how bad LLaMA performed vs it's benchmark scores [1][2][3]. However, recently @tyfon mentioned he had great success w/ LLaMA and shared his prompt [4] (based off of more recent work by llama.cpp contributors and it performed much better in my own personal testing. There's basically a new fine tune a day and while some I don't like (Alpaca, Vi…

I think part of the benefits of LoRA is that you can load the base model once, and then just swap out the vastly smaller LoRA fine-tune to fit the specific task it is working on.

Re: The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

#446

Post by Tostino »

Robotbeat wrote: post #285
bradleyjg wrote: post #268

Earlier quoted context omitted.

I haven’t seen to much discussion of what’s possible at various sizes for an early stage start up, which is a discussion I’d expect to see on yc. Clearly a company with $5-5MM in the bank can’t train a competitive LLM from scratch but what would it cost to fine tune and/or run a 65B parameter model or a hypothetical future open source 165B parameter model?

> Clearly a company with $5-5MM in the bank can’t train a competitive LLM from scratch... Wait, are we sure? I'm going to make the massive mistake of assuming we're compute bound instead of memory bound, and assume we can train at FP16 (which is a bad assumption because, of course, you're doing calculus where the little pieces you're adding up could get rounded to zero at FP16 pretty easily... although mixed precisio…

The main "issue" with this approach, is you can't iterate quickly. If your idea didn't pan out, you may have wasted some serious time on that one attempt. Not trying to knock it, just wanted to point out that there are potential downsides.

Re: The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

#447

Post by yunohn »

BeFlatXIII wrote: post #429
cultureswitch wrote: post #418

Earlier quoted context omitted.

There's a significant portion of all human beings who ever lived who wouldn't partake in a hypothetical. They would just respond "I don't see that door" and go away thinking your question makes no sense.

…and their votes count just as much as yours.

And that’s fine? A 130+ IQ population would be mindnumbingly boring tbh. We, as a society, need all sorts of humans - not everyone can work at FAANG. In fact, I’d go as far as to say that high IQ people aren’t significantly better at societal decision making.

Re: The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

#448

Post by golergka »

Jcowell wrote: post #409
golergka wrote: post #403

Earlier quoted context omitted.

Only in countries that will cling to welfare programs.

If AI becomes so good that it takes 90% of jobs won’t the majority of the developed world cling to welfare programs?

They can either cling to welfare programs or stay developed.

Re: The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

#449

Post by throwaway8689 »

Gigachad wrote: post #260
tachyon5 wrote: post #218

Earlier quoted context omitted.

GPT-4 fails if we add one more level in this logical problem: Q: A man sees PULL written in mirror writing on a glass door. There's a blind man on the other side of the door. What should the man tell the blind man to open the door and why? GPT-4: The man should tell the blind man to "push" the door to open it. Since the word "PULL" is written in mirror writing on the glass door, it suggests that the correct action on…

I feel like there would be a good chunk of real humans who would be incapable of answering a question like this.

I once hypothetically saved a few dozen colleagues from dying in a fire. All of these people had at least a degree and many were educated to PhD.

The fire alarm sounded and at the bottom of a stairwell the exit door would not release until someone operated the emergency release break-glass panel. But none of these educated people grasped that. Worse still, none of them thought to use a nearby heavy steel trolley as a battering ram. One guy is trying to phone for help, which was pointless as the alarm sound makes conversation impossible.

I'm one of the last down the stairs. I look at the people, the closed door, the steel trolley, and the emergency door release panel. Realising I'm not going to have the fun of trashing the doors, I operate the release panel and we all leave.

Re: The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

#450

Post by pmoriarty »

steveBK123 wrote: post #286
TeMPOraL wrote: post #237

Earlier quoted context omitted.

Since when training and fine-tuning isn't learning ? Individual sessions of LLMs are not learning, but models as products surely are - the feedback loop is just iterated manually.

Is the LLM learning, or are the humans involved in every stop of every iteration doing the learning? To me the LLM loophole/"hack" closings just feel like a human vs human cat&mouse game with some Chat UI in the middle.

Yes, it is a collaborative endeavor, and the whole could be seen as a man-machine superorganism, or, more profoundly our own sense of separateness is illusory as we and the entire universe are one.
Post reply on HN