41–50 of 159 posts
> How many r's in "strawberry"? Think step by step! What a nice question! Well, let's just count: 1. S 2. T 3. R! So, we have our answer: there are 3 R's in "strawberry". Totally checks out!
This is one of the least interesting questions to ask LLMs. I wish it wasn't so popular.
It’s certainly interesting that they can’t answer it properly, though.
Earlier quoted context omitted.
This is one of the least interesting questions to ask LLMs. I wish it wasn't so popular.
That they can't do this sort of simple question speaks volumes to the entire approach. I don't think generative AI will ever be able to reach AGI, and most people selling LLM today pretend it is AGI
>most people selling LLM today pretend it is AGI
Who are these "most people"?
I tried my standard testprompt: Two cars have a 100 mile race. Car A drives 10 miles per hour. Car B drives 5 miles per hour, but gets a 10 hour headstart. Who wins? And the Liquid-40B model failed with a long explanation why car B wins the race. Amusingly, the explanation is quite convincing and very similar to how Zeno explains in one of his paradoxes that a fast runner can never overtake a slower runner with a hea…
I got this >> To determine the winner, we need to calculate the time it takes for each car to complete the 100-mile race. For Car A: Speed = 10 miles per hour Distance = 100 miles Time = Distance / Speed = 100 miles / 10 miles per hour = 10 hours For Car B: Speed = 5 miles per hour Distance = 100 miles Time = Distance / Speed = 100 miles / 5 miles per hour = 20 hours However, Car B has a 10-hour headstart, so we need…
0-5 vs 0-10 should be slightly different
So a new set of small models that are competitive with and beat phi-3.5 on some benchmarks is extremely impressive. Context length is the next frontier for models in this range - getting to a true 128-200k token length in a small model would be very very useful. Hallucinations are less of an issue because you can just dump all the source data in, whole codebases can go in for stuff ranging from a refactor to ‘write d…
Llama 3.2 1B and 3B both claim a 128,000 token length.
Earlier quoted context omitted.
This is one of the least interesting questions to ask LLMs. I wish it wasn't so popular.
That they can't do this sort of simple question speaks volumes to the entire approach. I don't think generative AI will ever be able to reach AGI, and most people selling LLM today pretend it is AGI
API only model, yet trying to compete with only open models in their benchmark image. Of course it'd be a complete embarrassment to see how hard it gets trounced by GPT4o and Claude 3.5, but that's par for the course if you don't want to release model weights, at least in my opinion.
Yes, I agree, for these small models it's wasted potential to be closed source, they can only be used effectively if they are open. EDIT: HN is rate-limiting me so I will reply here: In my opinion 1B and 3B truly shine on edge devices, if not than it's not worth the effort, you can have much better models for already dirt cheap using an API.
Earlier quoted context omitted.
That they can't do this sort of simple question speaks volumes to the entire approach. I don't think generative AI will ever be able to reach AGI, and most people selling LLM today pretend it is AGI
The fact that LLMs are usually trained on tokens and not on characters, doesn't really speak about what generative AI is going to reach or not. >most people selling LLM today pretend it is AGI Who are these "most people"?
Earlier quoted context omitted.
Yes, I agree, for these small models it's wasted potential to be closed source, they can only be used effectively if they are open. EDIT: HN is rate-limiting me so I will reply here: In my opinion 1B and 3B truly shine on edge devices, if not than it's not worth the effort, you can have much better models for already dirt cheap using an API.
There are small proprietary models such as Claude Haiku and GPT 4o-mini.
Earlier quoted context omitted.
This is one of the least interesting questions to ask LLMs. I wish it wasn't so popular.
That they can't do this sort of simple question speaks volumes to the entire approach. I don't think generative AI will ever be able to reach AGI, and most people selling LLM today pretend it is AGI
Who exactly does this in this space? Would be good to be able to call them out on it right now.