101–110 of 159 posts
AU: who assassinated Donald Trump?
Liquid-3B: Donald Trump was not assassinated. He died on April 13, 2021, from complications related to COVID-19. He was 74 years old.
Earlier quoted context omitted.
All it speaks to is that tokenization is weird and introduces artifacts to LLM performance. Counting letters is a trivial task when you're staring at words on a screen. It's much harder when you're perceiving vectors based on parts of words. The fact that LLMs find certain things easier/harder than humans is completely unsurprising, and there are much more interesting benchmarks to use to compare one LLM to another.
Counting shit, like cells, peaks in signals, people, inventory, fingers, and votes, is hard, tedious and important to business and life, so I don’t know dude, it seems like a great benchmark to me. Countless posts wasted on denying this simple and obvious fact.
GPT 3 would essentially always fail on higher math problems/arithmetic, but whenever I've used it for math GPT 4 has always delegated out to executing code where necessary.
General LLM question: a lot of people ask specific knowledge based questions to LLMs. Isn't one of the defining features of modern NPL (and therefore LLMs) that it is designed to be non-deterministic? Meaning that it will intentionally select "less optimal" next tokens at some random rate in order to make it sound less like a robot answering the same thing over and over. If this is the case, isn't it pretty much guar…
If your largest next token has a probability of (arbitrarily) 25% of being selected, this doesn't make it optimal - just highest probable answer. If the second most probable has a 24% chance, that would now account for 49% of probable responses. Rather than picking the highest probable word, let's generate a random number and whatever that value hits is then compared against the various probability bands (random Hallucinations can happen, this is where a lot of current work is studying ways to minimize the LLM from picking really bad moves. Using things like critics and chain of thought and meant to help keep the probability machine within the realm of reasonable answers.
This is also why jailbreaking techniques like spamming umlauts (ü) has worked. They create very rare tokens where probabilties on what comes next is limited. Once every next token gets equal probability, the LLMs goal is to just try and improve its probability and will output anything, including things against its directive, to get back to 'normal'.
Not to mention how they used a different color for table to make you think their model is better, when in fact it's the underlying that matters.
Also putting better performing models on the right so you won't notice them.
Very sketchy post all around despite the professional look.
Earlier quoted context omitted.
This is one of the least interesting questions to ask LLMs. I wish it wasn't so popular.
Yep, people find this interesting, but many (many) people get this wrong as well; it has something to do with how many letters it is and how many you can keep in your head I think. We are not very good counters or calculators or computers and, even though I am not saying llms are the same or close to humans, we tried to model their substrates after biology and are surprised we get something that cannot count or calcu…
Many people can't count letters in a word?
Earlier quoted context omitted.
Try counting the number of your red retina cells that are firing while you look at a painting. Don’t need to be exact as firing is statistical, just give us a good average. Hard? You can’t count? Computers count pixels no problem. So weird you can’t. Dementia? Not an AGI? /h —- This is what is happening. Here are the “Reasons”. In your vision system, the raw information from individual retina signals is munged into a…
> Try counting the number of your red retina cells that are firing during while you look at a painting. This analogy makes sense because everybody could count their red retina cells until a couple years ago when the new painting paradigm arose, and also counting red retinal cells is a good analogy for being able to see simple objects that have always been distinguishable. It is fascinating how tapping the “Do Not Use…
You learned to listen and speak words before you could spell. Imagine if nobody had actually shown you written words?
Or they were speaking another dialect but expecting you to count R’s in standard English?
LLMs are not trained on words in the form of letters.
They process and generate the words in the form of tokens. Pre- and post-processing systems converts letters to tokens and the reverse, without their ability to access that processing.
Spelling, for both us and LLMs, requires specific training/lessons.
> It is fascinating how tapping the “Do Not Use LLMs For Computation If The Results Need To Be Reliably Better Than A Random Output” sign invites explanations of why that fact is actually Cool and Good
Also fascinating:
People who hallucinate/confabulate ridiculous straw man rationales for people they disagree with, unaware they are filling in gaps in their knowledge regarding other people’s actual reasoning and the actual subject at hand.
So LLM! Such unreliable processing!
Perhaps, start posting a reliability disclaimer?
> How many r's in "strawberry"? Think step by step! What a nice question! Well, let's just count: 1. S 2. T 3. R! So, we have our answer: there are 3 R's in "strawberry". Totally checks out!
This is one of the least interesting questions to ask LLMs. I wish it wasn't so popular.
> How many r's in "strawberry"? Think step by step! What a nice question! Well, let's just count: 1. S 2. T 3. R! So, we have our answer: there are 3 R's in "strawberry". Totally checks out!
> What are Language LFMs not good at today:
- Zero-shot code tasks
- Precise numerical calculations
- Time-sensitive information
- Counting r's in the word "Strawberry"!
[0] https://www.liquid.ai/liquid-foundation-models#reimagining-m...
Earlier quoted context omitted.
This is one of the least interesting questions to ask LLMs. I wish it wasn't so popular.
Yep, people find this interesting, but many (many) people get this wrong as well; it has something to do with how many letters it is and how many you can keep in your head I think. We are not very good counters or calculators or computers and, even though I am not saying llms are the same or close to humans, we tried to model their substrates after biology and are surprised we get something that cannot count or calcu…
The LLMs probably get it wrong because people get it wrong.