Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Large Enough

mistral.ai

Post reply on HN
Print view View original

181–190 of 512 posts

Re: Large Enough

#181

Post by lolinder »

RicoElectrico wrote: post #158
lolinder wrote: post #129

Earlier quoted context omitted.

> It seems increasingly apparent that we are reaching the limits of throwing more data at more GPUs Yes. This is exactly why I'm skeptical of AI doomerism/saviorism. Too many people have been looking at the pace of LLM development over the last two (2) years, modeled it as an exponential growth function, and come to the conclusion that AGI is inevitable in the next ${1-5} years and we're headed for ${(dys|u)topia}. B…

I don't think we are approaching limits, if you take off the English-centric glasses. You can query LLMs about pretty basic questions about Polish language or literature and it's gonna either bullshit or say it doesn't know the answer. Example: w której gwarze jest słowo ekspres i co znaczy? Słowo "ekspres" występuje w gwarze śląskiej i oznacza tam ekspres do kawy. Jest to skrót od nazwy "ekspres do kawy", czyli urzą…

What this means is just that Polish support (and probably most other languages besides English) in the models is behind SOTA. We can gradually get those languages closer to SOTA, but that doesn't bring us closer to AGI.

Re: Large Enough

#182

Post by nicman23 »

cjalmeida wrote: post #139
lolinder wrote: post #129

Earlier quoted context omitted.

> It seems increasingly apparent that we are reaching the limits of throwing more data at more GPUs Yes. This is exactly why I'm skeptical of AI doomerism/saviorism. Too many people have been looking at the pace of LLM development over the last two (2) years, modeled it as an exponential growth function, and come to the conclusion that AGI is inevitable in the next ${1-5} years and we're headed for ${(dys|u)topia}. B…

Indeed.All exponential growth curves are sigmoids in disguise.

except when it isn't and we ded :P

Re: Large Enough

#183

Post by simonw »

m3kw9 wrote: post #150
TIPSIO wrote: post #3

This race for the top model is getting wild. Everyone is claiming to one-up each with every version. My experience (benchmarks aside) Claude 3.5 Sonnet absolutely blows everything away. I'm not really sure how to even test/use Mistral or Llama for everyday use though.

It’s these kind of praise that makes me wonder if they are all paid to give glowing reviews, this is not my experience with sonnet at all. It absolutely does not blow away gpt4o.

My hunch is this comes down to personal prompting style. It's likely that your own style works more effectively with GPT-4o, while other people have styles that are more effective with Claude 3.5 Sonnet.

Re: Large Enough

#184

Post by lawlessone »

niemandhier wrote: post #137
Liquix wrote: post #10

These companies full of brilliant engineers are throwing millions of dollars in training costs to produce SOTA models that are... "on par with GPT-4o and Claude Opus"? And then the next 2.23% bump will cost another XX million? It seems increasingly apparent that we are reaching the limits of throwing more data at more GPUs; that an ARC prize level breakthrough is needed to move the needle any farther at this point.

The next iteration depends on NVIDIA & co, what we need is sparse libs. Most of the weights in llms are 0, once we deal with those more efficiently we will get to the next iteration.

> Most of the weights in llms are 0,

that's interesting. Do you have a rough percentage of this?

Does this mean these connections have no influence at all on output?

Re: Large Enough

#185

Post by staticman2 »

imtringued wrote: post #133
Tepix wrote: post #54

Earlier quoted context omitted.

LLMs think in tokens, not letters. It's like asking someone who is dyslexic about spelling. Not their strong suit. In practice, it doesn't matter much, does it?

This doesn't make any sense. The LLM knows the token to letter associations if you ask it. Once it has split the token into letters, it can answer the question anyway. This tells us that the model is unaware of its own limitations, otherwise it would have figured that this tasks needs it to split the letters first.

Well splitting into letters doesn't help with Mistral Large 2. I tried this with Mistral 2:

[Question: How many r's in strawberry? First split the word into letters, one letter on each line then answer the question.]

[Answer: Let's split the word "strawberry" into letters:

s t r a w b e r r y

Now, let's count the number of 'r's:

There are 2 'r's in the word "strawberry".]

[Question: Did you count the number of r's?]

[answer: Yes, I did. Here's the count:

r r

So, there are 2 'r's in the word "strawberry".]

I suspect the issue is these models have to be trained to accomplish tasks, and this is not a task it makes sense to train a model to perform. You might train it to do this with function calling one day, calling a python library to count the r's in a string, but actually manually counting r's just doesn't play to an LLM's strengths.

Re: Large Enough

#186

Post by GenerWork »

Kuinox wrote: post #49
drewnick wrote: post #27

Earlier quoted context omitted.

All 3 models you ranked cannot get "how many r's are in strawberry?" correct. They all claim 2 r's unless you press them. With all the training data I'm surprised none of them fixed this yet.

Tokenization make it hard for it to count the letters, that's also why if you ask it to do maths, writing the number in letters will yield better results. for strawberry, it see it as [496, 675, 15717], which is str aw berry. If you insert characters to breaks the tokens down, it find the correct result: how many r's are in "s"t"r"a"w"b"e"r"r"y" ? > There are 3 'r's in "s"t"r"a"w"b"e"r"r"y".

>If you insert characters to breaks the tokens down, it find the correct result: how many r's are in "s"t"r"a"w"b"e"r"r"y" ?

The issue is that humans don't talk like this. I don't ask someone how many r's there are in strawberry by spelling out strawberry, I just say the word.

Re: Large Enough

#187

Post by tr4656 »

freediver wrote: post #126
tikkun wrote: post #76

Earlier quoted context omitted.

Correct

That would make each API call cost at least $3 ($3 is price per million input tokens). And if you have a 10 message interaction you are looking at $30+ for the interaction. Is that what you would expect?

This might be when it's better to not use the API and just pay for the flat-rate subscription.

Re: Large Enough

#188

Post by machiaweliczny »

connorgutman wrote: post #66
Tepix wrote: post #55

Earlier quoted context omitted.

Claude is pretty great, but it's lacking the speech recognition and TTS, isn't it?

Correct. IMO the official Claude app is pretty garbage. Sonnet 3.5 API + Open-WebUI is amazing though and supports STT+TTS as well as a ton of other great features.

But projects are great in Sonnet, you just dump db schema some core file and you can figure stuff out quickly. I guess Aider is similar but i was lacking good history of chats and changes

Re: Large Enough

#189

Post by d13 »

coder543 wrote: post #21
TIPSIO wrote: post #3

This race for the top model is getting wild. Everyone is claiming to one-up each with every version. My experience (benchmarks aside) Claude 3.5 Sonnet absolutely blows everything away. I'm not really sure how to even test/use Mistral or Llama for everyday use though.

> I'm not really sure how to even test/use Mistral or Llama for everyday use though. Both Mistral and Meta offer their own hosted versions of their models to try out. https://chat.mistral.ai https://meta.ai You have to sign into the first one to do anything at all, and you have to sign into the second one if you want access to the new, larger 405B model. Llama 3.1 is certainly going to be available through other plat…

Groq’s models are also heavily quantised so you won’t get the full experience there.

Re: Large Enough

#190

Post by hansworst »

tikkun wrote: post #39
drewnick wrote: post #27

Earlier quoted context omitted.

All 3 models you ranked cannot get "how many r's are in strawberry?" correct. They all claim 2 r's unless you press them. With all the training data I'm surprised none of them fixed this yet.

When using a prompt that involves thinking first, all three get it correct. "Count how many rs are in the word strawberry. First, list each letter and indicate whether it's an r and tally as you go, and then give a count at the end." Llama 405b: correct Mistral Large 2: correct Claude 3.5 Sonnet: correct

Can’t you just instruct your llm of choice to transform your prompts like this for you? Basically feed it with a bunch of heuristics that will help it better understand the thing you tell it.

Maybe the various chat interfaces already do this behind the scenes?

Post reply on HN