Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Large Enough

mistral.ai

Post reply on HN
Print view View original

191–200 of 512 posts

Re: Large Enough

#191

Post by bilater »

nen-nomad wrote: post #171

The models are converging slowly. In the end, it will come down to the user experience and the "personality." I have been enjoying the new Claude Sonnet. It feels sharper than the others, even though it is not the highest-scoring one. One thing that `exponentialists` forget is that each step also requires exponentially more energy and resources.

And a factor for Mistral typically is it will give you less refusals and can be uncensored. So if I have to guess any task that requires creative output could be better suited for this.

Re: Large Enough

#192

Post by mratsim »

jedberg wrote: post #164
tikkun wrote: post #39

Earlier quoted context omitted.

When using a prompt that involves thinking first, all three get it correct. "Count how many rs are in the word strawberry. First, list each letter and indicate whether it's an r and tally as you go, and then give a count at the end." Llama 405b: correct Mistral Large 2: correct Claude 3.5 Sonnet: correct

This reminds me of when I had to supervise outsourced developers. I wanted to say "build a function that does X and returns Y". But instead I had to say "build a function that takes these inputs, loops over them and does A or B based on condition C, and then return Y by applying Z transformation" At that point it was easier to do it myself.

Exact instruction challenge https://www.youtube.com/watch?v=cDA3_5982h8

Re: Large Enough

#193

Post by gpm »

RyanAdamas wrote: post #134

Personally, language diversity should be the last thing on the list. If we had optimized every software from the get-go for a dozen languages our forward progress would have been dead in the water.

Language diversity means access to more training data, and you might also hope that by learning the same concept in multiple languages it does a better job of learning the underlying concept independent of the phrase structure...

At least from a distance it seems like training a multilingual state of the art model might well be easier than a monolingual one.

Re: Large Enough

#194

Post by nabakin »

wesleyyue wrote: post #138

I'm building a ai coding assistant ( https://double.bot ) so I've tried pretty much all the frontier models. I added it this morning to play around with it and it's probably the worst model I've ever played with. Less coherent than 8B models. Worst case of benchmark hacking I've ever seen. example: https://x.com/WesleyYue/status/1816153964934750691

Are you sure the chat history is being passed when the second message is sent? That looks like the kind of response you'd expect if it only received the prompt "in python" with no chat history at all.

Re: Large Enough

#195

Post by machiaweliczny »

troupo wrote: post #118
swyx wrote: post #31

Earlier quoted context omitted.

indeed. I pointed out in https://buttondown.email/ainews/archive/ainews-llama-31-the-... that the frontier model curve is currently going down 1 OoM every 4 months, meaning every model release has a very short half life[0]. however this progress is still worth it if we can deploy it to improve millions and eventually billions of people's lives. a commenter pointed out that the amoutn spent on Llama 3.1 was only like…

> however this progress is still worth it if we can deploy it to improve millions and eventually billions of people's lives Has there been any indication that we're improving the lives of millions of people?

Just me coding 30% faster is worth it

Re: Large Enough

#196

Post by Me1000 »

simonw wrote: post #179
Me1000 wrote: post #166

Earlier quoted context omitted.

Relatedly, what does "parallel" function calling mean in this context?

That's when the LLM can respond with multiple functions it wants you to call at once. You might send it: Location and population of Paris, France A parallel function calling LLM could return: { "role": "assistant", "content": "", "tool_calls": [ { "function": { "name": "get_city_coordinates", "arguments": "{\"city\": \"Paris\"}" } }, { "function": { "name": "get_city_population", "arguments": "{\"city\": \"Paris\"}"…

Ah, thank you!

Re: Large Enough

#197

Post by msp26 »

tikkun wrote: post #5

Links to chat with models that released this week: Large 2 - https://chat.mistral.ai/chat Llama 3.1 405b - https://www.llama2.ai/ I just tested Mistral Large 2 and Llama 3.1 405b on 5 prompts from my Claude history. I'd rank as: 1. Sonnet 3.5 2. Large 2 and Llama 405b (similar, no clear winner between the two) If you're using Claude, stick with it. My Claude wishlist: 1. Smarter (yes, it's the most intelligent, and y…

Large 2 is significantly smaller at 123B so it being comparable to llama 3 405B would be crazy.

Re: Large Enough

#198

Post by skerit »

TIPSIO wrote: post #3

This race for the top model is getting wild. Everyone is claiming to one-up each with every version. My experience (benchmarks aside) Claude 3.5 Sonnet absolutely blows everything away. I'm not really sure how to even test/use Mistral or Llama for everyday use though.

I don't get it. My husband also swears by Clause Sonnet 3.5, but every time I use it, the output is considerably worse than GPT-4o

Re: Large Enough

#199

Post by machiaweliczny »

lawlessone wrote: post #184
niemandhier wrote: post #137

Earlier quoted context omitted.

The next iteration depends on NVIDIA & co, what we need is sparse libs. Most of the weights in llms are 0, once we deal with those more efficiently we will get to the next iteration.

> Most of the weights in llms are 0, that's interesting. Do you have a rough percentage of this? Does this mean these connections have no influence at all on output?

My uneducated guess is that with many layers you can implement something akin to graph in brain by nulling lots of previous later outputs. I actually suspect that current models aren’t optimal with layers all of the same size but i know shit

Re: Large Enough

#200

Post by espadrine »

coder543 wrote: post #21
TIPSIO wrote: post #3

This race for the top model is getting wild. Everyone is claiming to one-up each with every version. My experience (benchmarks aside) Claude 3.5 Sonnet absolutely blows everything away. I'm not really sure how to even test/use Mistral or Llama for everyday use though.

> I'm not really sure how to even test/use Mistral or Llama for everyday use though. Both Mistral and Meta offer their own hosted versions of their models to try out. https://chat.mistral.ai https://meta.ai You have to sign into the first one to do anything at all, and you have to sign into the second one if you want access to the new, larger 405B model. Llama 3.1 is certainly going to be available through other plat…

meta.ai is inaccessible in a large portion of world territories, but the Llama 3.1 70B and 405B are also available in https://hf.co/chat

Additionally, all Llama 3.1 models are available in https://api.together.ai/playground/chat/meta-llama/Meta-Llam... and in https://fireworks.ai/models/fireworks/llama-v3p1-405b-instru... by logging in.

Post reply on HN