Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Large Enough

mistral.ai

Post reply on HN
Print view View original

101–110 of 512 posts

Re: Large Enough

#101

Post by throwup238 »

swyx wrote: post #31
Liquix wrote: post #10

These companies full of brilliant engineers are throwing millions of dollars in training costs to produce SOTA models that are... "on par with GPT-4o and Claude Opus"? And then the next 2.23% bump will cost another XX million? It seems increasingly apparent that we are reaching the limits of throwing more data at more GPUs; that an ARC prize level breakthrough is needed to move the needle any farther at this point.

indeed. I pointed out in https://buttondown.email/ainews/archive/ainews-llama-31-the-... that the frontier model curve is currently going down 1 OoM every 4 months, meaning every model release has a very short half life[0]. however this progress is still worth it if we can deploy it to improve millions and eventually billions of people's lives. a commenter pointed out that the amoutn spent on Llama 3.1 was only like…

All that Marvel slop was created by the first real LLM: https://marvelcinematicuniverse.fandom.com/wiki/K.E.V.I.N.>

Re: Large Enough

#102

Post by ThrowawayTestr »

xeromal wrote: post #88
imtringued wrote: post #74

Earlier quoted context omitted.

LLMs are reaching saturation on even some of the latest benchmarks and yet I am still a little disappointed by how they perform in practice. They are by no means bad, but I am now mostly interested in long context competency. We need benchmarks that force the LLM to complete multiple tasks simultaneously in one super long session.

I don't know anything about AI but there's one thing I want it to do for me. Program a full body exercise program long term based on the parameters I give it such as available equipment and past workout context goals. I haven't had good success with chatgpt but I assume what you're talking about is relevant to my goals.

Aren't there apps that already do this like Fitbod?

Re: Large Enough

#103

Post by dev1ycan »

__jl__ wrote: post #34
happyhardcore wrote: post #17

Earlier quoted context omitted.

I suspect this is why OpenAI is going more in the direction of optimising for price / latency / whatever with 4o-mini and whatnot. Presumably they found out long before the rest of us did that models can't really get all that much better than what we're approaching now, and once you're there the only thing you can compete on is how many parameters it takes and how cheaply you can serve that to users.

Meta just claimed the opposite in their Llama 3.1 paper. Look at the conclusion. They say that their experience indicates significant gains for the next iteration of models. The current crop of benchmarks might not reflect these gains, by the way.

Or maybe they just want to avoid getting sued by shareholders for dumping so much money into unproven technology that ended up being the same or worse than the competitor

Re: Large Enough

#104

Post by asah »

romeros wrote: post #63
chipdart wrote: post #41

Earlier quoted context omitted.

> It seems increasingly apparent that we are reaching the limits of throwing more data at more GPUs; I think you're just seeing the "make it work" stage of the combo "first make it work, then make it fast". Time to market is critical, as you can attest by the fact you framed the situation as "on par with GPT-4o and Claude Opus". You're seeing huge investments because being the first to get a working model stands to b…

ChatGPT is like Google now. It is the default. Even if Claude becomes as good as ChatGPT or even slightly better it won't make me switch. It has to be like a lot better. Way better. It feels like ChatGPT won the time to market war already.

-1: I know many people who are switching to Claude. And Google makes it near-zero friction to adopt Gemini with Gsuite. And more still are using the top-N of them.

This is similar to the early days of the search engine wars, the browser wars, and other categories where a user can easily adopt, switch between and use multiple. It's not like the cellphone OS/hardware war, PC war and database war where (most) users can only adopt one platform at a time and/or there's a heavy platform investment.

Re: Large Enough

#105

Post by amilios »

rkwz wrote: post #13

> A significant effort was also devoted to enhancing the model’s reasoning capabilities. One of the key focus areas during training was to minimize the model’s tendency to “hallucinate” or generate plausible-sounding but factually incorrect or irrelevant information. This was achieved by fine-tuning the model to be more cautious and discerning in its responses, ensuring that it provides reliable and accurate outputs.…

Unfortunately not, as it captures such a wide spectrum of use cases and scenarios. There are some benchmarks to measure this quality in specific settings, e.g. summarization, but AFAIK nothing general.

Re: Large Enough

#106

Post by camel_Snake »

splwjs wrote: post #78
__jl__ wrote: post #34

Earlier quoted context omitted.

Meta just claimed the opposite in their Llama 3.1 paper. Look at the conclusion. They say that their experience indicates significant gains for the next iteration of models. The current crop of benchmarks might not reflect these gains, by the way.

I sell widgets. I promise the incalculable power of widgets has yet to be unleashed on the world, but it is tremendous and awesome and we should all be very afraid of widgets taking over the world because I can't see how they won't. Anyway here's the sales page. the widget subscription is so premium you won't even miss the subscription fee.

Meta doesn't sell widgets in this scenario - they give them away for free. Their competition sells widgets, so Meta would be perfectly happy if the widget market totally collapsed.

Re: Large Enough

#107

Post by ThrowawayTestr »

thierrydamiba wrote: post #82
swyx wrote: post #31

Earlier quoted context omitted.

indeed. I pointed out in https://buttondown.email/ainews/archive/ainews-llama-31-the-... that the frontier model curve is currently going down 1 OoM every 4 months, meaning every model release has a very short half life[0]. however this progress is still worth it if we can deploy it to improve millions and eventually billions of people's lives. a commenter pointed out that the amoutn spent on Llama 3.1 was only like…

Agreed on everything, but calling the marvel movies slop…I think that word has gone too far.

The marvel movies are the genesis for this use of the word slop.

Re: Large Enough

#108

Post by simonw »

ThrowawayTestr wrote: post #107
thierrydamiba wrote: post #82

Earlier quoted context omitted.

Agreed on everything, but calling the marvel movies slop…I think that word has gone too far.

The marvel movies are the genesis for this use of the word slop.

Can you back that claim up with a link or similar?

Re: Large Enough

#109

Post by staticman2 »

romeros wrote: post #63
chipdart wrote: post #41

Earlier quoted context omitted.

> It seems increasingly apparent that we are reaching the limits of throwing more data at more GPUs; I think you're just seeing the "make it work" stage of the combo "first make it work, then make it fast". Time to market is critical, as you can attest by the fact you framed the situation as "on par with GPT-4o and Claude Opus". You're seeing huge investments because being the first to get a working model stands to b…

ChatGPT is like Google now. It is the default. Even if Claude becomes as good as ChatGPT or even slightly better it won't make me switch. It has to be like a lot better. Way better. It feels like ChatGPT won the time to market war already.

If ChatGPT fails to do a task you want, your instinct isn't "I'll run the prompt through Claude and see if it works" but "oh well, who needs LLMs?"

Re: Large Enough

#110

Post by mattnewton »

splwjs wrote: post #78
__jl__ wrote: post #34

Earlier quoted context omitted.

Meta just claimed the opposite in their Llama 3.1 paper. Look at the conclusion. They say that their experience indicates significant gains for the next iteration of models. The current crop of benchmarks might not reflect these gains, by the way.

I sell widgets. I promise the incalculable power of widgets has yet to be unleashed on the world, but it is tremendous and awesome and we should all be very afraid of widgets taking over the world because I can't see how they won't. Anyway here's the sales page. the widget subscription is so premium you won't even miss the subscription fee.

that would make sense if it was from Openai, but Meta doesn't actually sell these widgets? They release the widget machines for free in the hopes that other people will build a widget ecosystem around them to rival the closed widget ecosystem that threatens to lock them out of a potential "next platform" powered by widgets.
Post reply on HN