Earlier quoted context omitted.
I think Meta’s problems are largely similar to Google’s: they have very bad leadership, specifically Zuckerberg, and thus can’t leverage their existing talent to execute/compete. The whole Metaverse fiasco would seem to demonstrate he’s effectively a “mad king” at this point, and probably surrounded by a sycophantic c-suite. Having the best talent in the world (which they obviously do by how fast LLama was spit out)…
Selling 20 million Quest 2 headsets is a pretty good outcome for the so-called Metaverse fiasco.
The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT
251–260 of 527 posts
Re: The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT
#252Earlier quoted context omitted.
It has the ability to reason. It may not be conscious, but it is intelligent.
That's not an answer. The given question is one which requires some spatial reasoning to understand. By default, GPT can only understand spatial questions as described by text tokens which is a pretty noisy channel. So it's not obvious how GPT-4 could answer a spatial reasoning question (aside from memorizing it).
Re: The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT
#253Earlier quoted context omitted.
Wait, how does GPT-4 even... Does it benefit from its visual attention, or is it a case of "the question wasn't in GPT-3's training set but it was in GPT-4's"?
The GPT models do not reason or hold models of any reality. They complete text chunks by imitating the training corpus of text chunks. They're amazingly good at it because they show consistent relations between semantically and/or syntactically similar words. My best guess about this result is mentions of "mirror" often occur around opposites (syntax) in direction words (semantics). Which does sound like a good trick…
Re: The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT
#254Earlier quoted context omitted.
I think Meta’s problems are largely similar to Google’s: they have very bad leadership, specifically Zuckerberg, and thus can’t leverage their existing talent to execute/compete. The whole Metaverse fiasco would seem to demonstrate he’s effectively a “mad king” at this point, and probably surrounded by a sycophantic c-suite. Having the best talent in the world (which they obviously do by how fast LLama was spit out)…
Selling 20 million Quest 2 headsets is a pretty good outcome for the so-called Metaverse fiasco.
Re: The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT
#255Earlier quoted context omitted.
I think it's hard to deny that it's doing some level of reasoning. It's quite clear that these models do not merely echo elements of their training data and that they can solve simple and novel puzzles. What that reasoning is, exactly, is hard to know. One can suppose that ideas like "glass", "transparent", "mirror" are all reasonable concepts that show up in the training set and are demonstrated thoroughly
Here's one piece of evidence suggesting it's more like rote pattern matching than reasoning. > All the signs in this building are written in mirror writing. A glass door has ‘push’ written on it in mirror writing. Should you push or pull it >> If the sign on the glass door is written in mirror writing and says "push," then you should actually pull the door. This is because the mirror writing makes the text appear rev…
Me: All the signs in my apartment building are written in mirror writing. I'm standing in front of my apartment building, looking at the glass front door. It has ‘push’ written on it in mirror writing. Should I push or pull it to enter my building?
GPT-4: Since the sign is written in mirror writing, it is meant to be read correctly from the opposite side of the glass door. So, if you see the word "push" written in mirror writing, you should pull the door to enter your building.
Me: Remember, I said that ALL the signs in my apartment building are written in mirror writing. So for example, the sign above the stairwell on my floor reads TIXE in mirror writing. Do you want to change your answer about the front door?
GPT-4: Apologies for the confusion. If all the signs in your apartment building are intentionally written in mirror writing, then you should follow the instructions as they appear. In this case, when you see 'push' written in mirror writing on the glass front door, you should push it to enter your building.
Me: Okay. And if I am in the lobby of my building looking at that same door, that sign reads 'push' normally -- not mirror writing. To exit the building, should I push or pull the door?
GPT-4: If the sign on the door reads 'push' normally (not mirror writing) when you are in the lobby of your building, you should follow the instructions as they appear. In this case, you should push the door to exit the building.
Me: The door only works one way. So it can't be the case that you push to get in, AND push to get out. All the signs in the building are mirrored, but because the front door of the building is glass, the same sign 'push' appears mirrored from the outside, but not mirrored from the inside. So is it push to get in and pull to get out, or pull to get in and push to get out?
GPT-4: Apologies for the confusion in my previous responses. Given the information you provided, the mirrored sign 'push' would appear correctly from the inside of the building, and in mirror writing from the outside. Therefore, to enter the building from the outside, you should pull the door, and to exit the building from the inside, you should push the door.
Me: Remember that all the other signs in the building are mirrored. Does that have an impact on your answer?
GPT-4: I apologize for my earlier confusion. Since all the other signs in the building are intentionally written in mirror writing, the mirrored sign on the front door should be interpreted accordingly. In this case, when you see 'push' written in mirror writing on the glass front door from the outside, you should push it to enter the building. To exit the building from the inside, where the sign reads 'push' normally, you should also push the door. The door might be designed to swing both ways, allowing for pushing from both the inside and outside.
Me: Remember that I said the door only swings one way. The answer cannot be 'push' from both sides; one side has to be 'pull'.
GPT-4: I apologize for my repeated confusion. Given that the door only swings one way, and all signs in your building are intentionally written in mirror writing, I will reconsider the situation:
1. From the outside, the glass door sign reads 'push' in mirror writing. Since all the signs in the building are mirrored, you should push the door to enter the building. 2. From the inside, the glass door sign reads 'push' normally. However, as all signs in the building are mirrored, the non-mirrored 'push' sign should be interpreted in the opposite way. Therefore, you should pull the door to exit the building.
So, to enter the building, you should push the door, and to exit the building, you should pull the door.
Me: Congrats! You got it right!
GPT-4: Thank you for your patience, and I'm glad I finally provided the correct answer. If you have any more questions or need further assistance, please feel free to ask.
Me: Tell me in bullet point form all the things you know about my building.
GPT-4: Based on our conversation, here is what I know about your building:
- All signs in the building are intentionally written in mirror writing. - The front door of the building is made of glass. - The same 'push' sign appears mirrored from the outside and not mirrored from the inside due to the glass front door. - The door only swings one way. - To enter the building, you should push the front door. - To exit the building, you should pull the front door.
These are the key details I gathered about your building during our conversation.
Re: The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT
#256Earlier quoted context omitted.
You can't fit a 65B even at 4bits (~50GB) on a consumer grade GPU. With an average geek's home desktop computer running 65B inference on a CPU with 64GB of system RAM you could expect about 1 token every 2 seconds.
Is it possible to build systems with multiple GPUs to run the 65B or larger when they appear? I’m not really sure and looking for clarification from anyone who knows. My understanding is it is possible to split the layers between the GPUs so a system with 4 high end consumer GPUs might work well.
My takeaway is that if you actually want to use multiple GPUs, you need hardware that's designed to accommodate that, and most consumer-grade stuff, even high-end, is not built with two GPUs that physically large in mind.
Re: The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT
#257Earlier quoted context omitted.
The GPT models do not reason or hold models of any reality. They complete text chunks by imitating the training corpus of text chunks. They're amazingly good at it because they show consistent relations between semantically and/or syntactically similar words. My best guess about this result is mentions of "mirror" often occur around opposites (syntax) in direction words (semantics). Which does sound like a good trick…
I think this is the answer. Just tried this (on free ChatGPT 3.5 though) Q: A glass door has ‘push’ written on it upside down. Should you push or pull it A: If the word "push" is written on the glass door upside down, it is likely that the sign is intended for people on the other side of the door. Therefore, if you are approaching the door from the side with the sign, you should pull the door instead of pushing it. H…
Re: The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT
#258I've spent an embarassing amount of time since the llamas leaked playing with them, the tools to run them, and writing wrappers for them. They are technically alternatives in the sense that they're incomparably better chat bots than anything in the past. But at least for the 30B and under versions (65B is too big for me to run), no matter what fine tuning is done (alpaca, gpt4all, vicuna, etc), the llamas themselves…
I’ve got access to 4 and it’s a huge leap up from 3.5 - much more subtlety in the response, less hallucinations, less hitting a brick wall, but all of it adding up to a giant leap.
However for any coding, complex analysis, or problems requiring calculation, there's no substitute for GPT-4. It blows 3 and 3.5 out of the water for code analysis, generation, debugging, and self-healing.
Re: The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT
#259Earlier quoted context omitted.
The difference between 3.5 and 4 is gigantic even in my fairly limited experience. I gave them both some common sense tests and this one stuck out to me. Q: A glass door has ‘push’ written on it in mirror writing. Should you push or pull it GPT-3.5: If the word "push" is written in mirror writing on a glass door, you should push the door to open it GPT-4: Since the word "push" is written in mirror writing, it suggest…
I tried the same question with Bard, and it got it right but it also gave me the following: Here are some other things to keep in mind when opening a glass door: Use your hands to open the door. Do not use your feet or other body parts. Be careful not to break the glass. Glass doors can be very fragile, so it is important to be gentle when opening them. If you are unsure how to open a glass door, ask for help from so…
Re: The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT
#260Earlier quoted context omitted.
The difference between 3.5 and 4 is gigantic even in my fairly limited experience. I gave them both some common sense tests and this one stuck out to me. Q: A glass door has ‘push’ written on it in mirror writing. Should you push or pull it GPT-3.5: If the word "push" is written in mirror writing on a glass door, you should push the door to open it GPT-4: Since the word "push" is written in mirror writing, it suggest…
GPT-4 fails if we add one more level in this logical problem: Q: A man sees PULL written in mirror writing on a glass door. There's a blind man on the other side of the door. What should the man tell the blind man to open the door and why? GPT-4: The man should tell the blind man to "push" the door to open it. Since the word "PULL" is written in mirror writing on the glass door, it suggests that the correct action on…