401–410 of 512 posts
Earlier quoted context omitted.
Not letters, but tokens. Think that it's translating everything to/from Chinese.
How does that explain why the tokens for strawberry, melon and "Stellaberry" [1] are close to each other? [1] Suggestion from chatgpt3.5 for new fruit name.
Earlier quoted context omitted.
Testing models on their tokenization has always struck me as kinda odd. Like, that has nothing to do with their intelligence.
> that has nothing to do with their intelligence. Of course. Because these models have no intelligence. Everyone who believes they do seem to believe intelligence derives from being able to use language, however, and not being able to tell how many times the letter r is in the word strawberry is a very low bar to not pass.
Earlier quoted context omitted.
Indeed, I appreciate the explanation, it is certainly both interesting and informative to me, but to somewhat echo the person you are replying to - if I wanted a boat, and you offer me a boat, and it doesn’t float - the reasons for failure are perhaps full of interesting details, but perhaps the most important thing to focus on first - is to make the boat float, or stop offering it to people who are in need of a boat…
At some point you need to just accept the details and limitations of things. We do this all the time. Why is your calculator giving only approximate result? Why can't your car go backwards as fast as forwards? Etc. It sucks that everyone gets exposed to the relatively low level implementation with LLM (almost the raw model), but that's the reality today.
These companies full of brilliant engineers are throwing millions of dollars in training costs to produce SOTA models that are... "on par with GPT-4o and Claude Opus"? And then the next 2.23% bump will cost another XX million? It seems increasingly apparent that we are reaching the limits of throwing more data at more GPUs; that an ARC prize level breakthrough is needed to move the needle any farther at this point.
The thing I don't understand is why everyone is throwing money at LLMs for language, when there are much simpler use cases which are more useful? For example, has anyone ever attempted image -> html/css model? Seems like it be great if I can draw something on a piece of paper and have it generate a website view for me.
Earlier quoted context omitted.
Testing models on their tokenization has always struck me as kinda odd. Like, that has nothing to do with their intelligence.
I would counterargue with "that's the model's problem, not mine". Here's a thought experiment: if I gave you 5 boxes and told you "how many balls are there in all of this boxes?" and you answered "I don't know because they are inside boxes", that's a fail. A truly intelligent individual would open them and look inside. A truly intelligent model would (say) retokenize the word into its individual letters (which I'm op…
Earlier quoted context omitted.
It just needs a little hint Me: spell "strawberry" with 1 bullet point per letter ChatGPT: S T R A W B E R R Y Me: How many Rs? ChatGPT: There are three Rs in "strawberry".
Me: try again ChatGPT: There are two Rs in "strawberry."
Earlier quoted context omitted.
It doesn’t test “on tokenization” though. What happens when an answer is generated is few abstraction levels deeper than tokens. A “thinking” “slice” of an llm is completely unaware of tokens as an immediate part of its reasoning. The question just shows lack of systemic knowledge about strawberry as a word (which isn’t surprising, tbh).
It is. Strawberry is one token in many tokenziers. The model doesn't have a concept that there are letters there.
Earlier quoted context omitted.
>If you insert characters to breaks the tokens down, it find the correct result: how many r's are in "s"t"r"a"w"b"e"r"r"y" ? The issue is that humans don't talk like this. I don't ask someone how many r's there are in strawberry by spelling out strawberry, I just say the word.
It's not a human. I imagine if you have a use case where counting characters is critical, it would be trivial to programmatically transform prompts into lists of letters. A token is roughly four letters [1], so, among other probable regressions, this would significantly reduce the effective context window. [1] https://help.openai.com/en/articles/4936856-what-are-tokens-...
Earlier quoted context omitted.
If ChatGPT fails to do a task you want, your instinct isn't "I'll run the prompt through Claude and see if it works" but "oh well, who needs LLMs?"
Please don't assume your experience applies to everyone. If ChatGPT can't do what I want, my first reaction is to ask Claude for the same thing. Often to find out that Claude performs much better. I've already cancelled ChaptGPT Plus for exactly that reason.
Earlier quoted context omitted.
Except: Meta doesn't sell AI at all. Zuck is just doing this for two reasons: - flex - deal a blow to Altmann
Meta uses ai in all the recommendation algorithms. They absolutely hope to turn their chat assistants into a product on WhatsApp too, and GenAI is crucial to creating the metaverse. This isn’t just a charity case.