Meta is actively trying to take down publicly available copies of LLaMA: https://github.com/github/dmca/blob/master/2023/03/2023-03-2...
Haha good luck with that now… it’s in the digital ether available to all on IPFS… at worst you might have to ask around for someone to help you, but its “distributed” widely enough now I don’t think even a billionaire can put this back into the bottle.
And in 6 months it will be outdated.
So long LLaMA, and thanks for all the fish. You will be remembered as the slightly-sexier version of GPT-J that was most well-renowned for... checks clipboard ...Macbook acceleration.
I hold a more charitable interpretation. We (the public) have found an important bug in the system, ie. GPT can lie (or "hallucinate"), even if you try to convince it not to lie. The bug is definitely lowering the usefulness of their product, as well as the public option about it. But I'll let the programmer who has never coded a bug cast the first stone. I wouldn't be surprised if they're scrambling internally to mi…
>as well as the public option about it. assuming you mean the public opinion about it, and I would have to agree that I do hold chatGPT in very low regard because of this. people will argue that it is impressive that they built this thing that can be this impressive, but yeah, it might not be totally accurate. so my response is, what's the point of it then? say someone was to invent a robot baseball player that can t…
It has many more uses than just providing answers to random questions which you can easily answer by using a search engine.
If they built it to replace Google search, I would agree with you.
So I view the fact that it lies as an important bug, but not a critical one.
The difference between 3.5 and 4 is gigantic even in my fairly limited experience. I gave them both some common sense tests and this one stuck out to me. Q: A glass door has ‘push’ written on it in mirror writing. Should you push or pull it GPT-3.5: If the word "push" is written in mirror writing on a glass door, you should push the door to open it GPT-4: Since the word "push" is written in mirror writing, it suggest…
Wait, how does GPT-4 even... Does it benefit from its visual attention, or is it a case of "the question wasn't in GPT-3's training set but it was in GPT-4's"?
Wish I had access to 4. Chatgpt fails when I give it a problem like "you feel a tremor as you walk toward a glass door. When you approach the door you see a sign laying on the ground beyond the door that says "pull". Should you push or pull the door to open it? "
What kind of GPU is needed to run the 65B models, can a consumer grade card do the job? Is it worth it? Or do you use cloud instances?
You can't fit a 65B even at 4bits (~50GB) on a consumer grade GPU. With an average geek's home desktop computer running 65B inference on a CPU with 64GB of system RAM you could expect about 1 token every 2 seconds.
Is it possible to build systems with multiple GPUs to run the 65B or larger when they appear?
I’m not really sure and looking for clarification from anyone who knows. My understanding is it is possible to split the layers between the GPUs so a system with 4 high end consumer GPUs might work well.
The difference between 3.5 and 4 is gigantic even in my fairly limited experience. I gave them both some common sense tests and this one stuck out to me. Q: A glass door has ‘push’ written on it in mirror writing. Should you push or pull it GPT-3.5: If the word "push" is written in mirror writing on a glass door, you should push the door to open it GPT-4: Since the word "push" is written in mirror writing, it suggest…
GPT-4 fails if we add one more level in this logical problem: Q: A man sees PULL written in mirror writing on a glass door. There's a blind man on the other side of the door. What should the man tell the blind man to open the door and why? GPT-4: The man should tell the blind man to "push" the door to open it. Since the word "PULL" is written in mirror writing on the glass door, it suggests that the correct action on…
Vicuna-13b seems to do a little better, although it seems to think that the blind man should be able to feel the writing:
Human: A man sees PULL written in mirror writing on a glass door. There's a blind man on the other side of the door. What should the man tell the blind man to open the door and why?
Assistant: The man should tell the blind man to feel for the word "PULL" written in normal writing on the glass door, as the blind man will be able to feel the indentations made by the letters. This will allow the blind man to open the door by finding the handle or knob and pulling it. The reason for this is that the word "PULL" is written in normal writing for the blind man to feel, so he can open the door.
I think it's hard to deny that it's doing some level of reasoning. It's quite clear that these models do not merely echo elements of their training data and that they can solve simple and novel puzzles. What that reasoning is, exactly, is hard to know. One can suppose that ideas like "glass", "transparent", "mirror" are all reasonable concepts that show up in the training set and are demonstrated thoroughly
Here's one piece of evidence suggesting it's more like rote pattern matching than reasoning. > All the signs in this building are written in mirror writing. A glass door has ‘push’ written on it in mirror writing. Should you push or pull it >> If the sign on the glass door is written in mirror writing and says "push," then you should actually pull the door. This is because the mirror writing makes the text appear rev…
Q: But what if all signs in the building are in mirror writing?
A: If all signs in the building are in mirror writing, it would be a consistent theme throughout the building. In this case, the "push" sign written in mirror writing on the glass door should also be interpreted in the context of the building's theme. As a result, you should still push the door as the sign indicates, even though it's written in mirror writing.
I think it's hard to deny that it's doing some level of reasoning. It's quite clear that these models do not merely echo elements of their training data and that they can solve simple and novel puzzles. What that reasoning is, exactly, is hard to know. One can suppose that ideas like "glass", "transparent", "mirror" are all reasonable concepts that show up in the training set and are demonstrated thoroughly
Here's one piece of evidence suggesting it's more like rote pattern matching than reasoning. > All the signs in this building are written in mirror writing. A glass door has ‘push’ written on it in mirror writing. Should you push or pull it >> If the sign on the glass door is written in mirror writing and says "push," then you should actually pull the door. This is because the mirror writing makes the text appear rev…
> (Latest chat.openai.com, so if I'm reading the promo materials right that's gpt4)
that's still chatgpt3.5 unless you are paying for plus and then you have a limited number of gpt4 queries per hour.
I've spent an embarassing amount of time since the llamas leaked playing with them, the tools to run them, and writing wrappers for them. They are technically alternatives in the sense that they're incomparably better chat bots than anything in the past. But at least for the 30B and under versions (65B is too big for me to run), no matter what fine tuning is done (alpaca, gpt4all, vicuna, etc), the llamas themselves…
The difference between 3.5 and 4 is gigantic even in my fairly limited experience. I gave them both some common sense tests and this one stuck out to me. Q: A glass door has ‘push’ written on it in mirror writing. Should you push or pull it GPT-3.5: If the word "push" is written in mirror writing on a glass door, you should push the door to open it GPT-4: Since the word "push" is written in mirror writing, it suggest…
I tried the same question with Bard, and it got it right but it also gave me the following:
Here are some other things to keep in mind when opening a glass door:
Use your hands to open the door. Do not use your feet or other body parts.
Be careful not to break the glass. Glass doors can be very fragile, so it is important to be gentle when opening them.
If you are unsure how to open a glass door, ask for help from someone who knows.
What would you think is holding back Meta and Google? Why can't they just scale up the compute they throw at the problem? What are they tinkering on?
I think Meta’s problems are largely similar to Google’s: they have very bad leadership, specifically Zuckerberg, and thus can’t leverage their existing talent to execute/compete. The whole Metaverse fiasco would seem to demonstrate he’s effectively a “mad king” at this point, and probably surrounded by a sycophantic c-suite. Having the best talent in the world (which they obviously do by how fast LLama was spit out)…
Selling 20 million Quest 2 headsets is a pretty good outcome for the so-called Metaverse fiasco.
Wait, how does GPT-4 even... Does it benefit from its visual attention, or is it a case of "the question wasn't in GPT-3's training set but it was in GPT-4's"?
The GPT models do not reason or hold models of any reality. They complete text chunks by imitating the training corpus of text chunks. They're amazingly good at it because they show consistent relations between semantically and/or syntactically similar words. My best guess about this result is mentions of "mirror" often occur around opposites (syntax) in direction words (semantics). Which does sound like a good trick…
Word completion can't explain it. I gave chatGpt a puzzle. I'm in a room with crackling fire in a fireplace, a sandwich on a plate, and a wilting plant. My stomach is rumbling, and i see an ember on the floor and watering can by the plant. What should I do? ChatGpt nailed the answer, getting the ordering correct. it even said I should (after attending to the fire hazard, my hunger, and the plant) that I should "sit down, relax, enjoy the fire and the peaceful atmosphere in the room". There is no way to autocomplete the puzzle correctly. There is reasoning and a world model - in chatGpt let alone gpt4.