Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Large Enough

mistral.ai

Post reply on HN
Print view View original

121–130 of 512 posts

Re: Large Enough

#121

Post by cle »

_uhtu wrote: post #91
splwjs wrote: post #78

Earlier quoted context omitted.

I sell widgets. I promise the incalculable power of widgets has yet to be unleashed on the world, but it is tremendous and awesome and we should all be very afraid of widgets taking over the world because I can't see how they won't. Anyway here's the sales page. the widget subscription is so premium you won't even miss the subscription fee.

This. It's really weird the way we suddenly live in a world where it's the norm to take whatever a tech company says about future products at face value. This is the same world where Tesla promised "zero intervention LA to NYC self driving" by the end of the year in 2016, 2017, 2018, 2019, 2020, 2021, 2022, 2023, and 2024. The same world where we know for a fact that multiple GenAI demos by multiple companies were ju…

I'm not sure anyone is asking you to take it at face value or implicitly trust them? There's a 92-page paper with details: https://ai.meta.com/research/publications/the-llama-3-herd-o...

Re: Large Enough

#122

Post by crystal_revenge »

happyhardcore wrote: post #17
Liquix wrote: post #10

These companies full of brilliant engineers are throwing millions of dollars in training costs to produce SOTA models that are... "on par with GPT-4o and Claude Opus"? And then the next 2.23% bump will cost another XX million? It seems increasingly apparent that we are reaching the limits of throwing more data at more GPUs; that an ARC prize level breakthrough is needed to move the needle any farther at this point.

I suspect this is why OpenAI is going more in the direction of optimising for price / latency / whatever with 4o-mini and whatnot. Presumably they found out long before the rest of us did that models can't really get all that much better than what we're approaching now, and once you're there the only thing you can compete on is how many parameters it takes and how cheaply you can serve that to users.

> the only thing you can compete on is how many parameters it takes and how cheaply you can serve that to users.

The problem with this strategy is that it's really tough to compete with open models in this space over the long run.

If you look at OpenAI's homepage right now they're trying to promote "ChatGPT on your desktop", so it's clear even they realize that most people are looking for a local product. But once again this is a problem for them because open models run locally are always going to offer more in terms of privacy and features.

In order for proprietary models served through an API to compete long term they need to offer significant performance improvements over open/local offerings, but that gap has been perpetually shrinking.

On an M3 macbook pro you can run open models easily for free that perform close enough to OpenAI that I can use them as my primary LLM for effectively free with complete privacy and lots of room for improvement if I want to dive into the details. Ollama today is pretty much easier to install than just logging into ChatGPT and the performance feels a bit more responsive for most tasks. If I'm doing a serious LLM project I most certainly won't use proprietary models because the control I have over the model is too limited.

At this point I have completely stopped using proprietary LLMs despite working with LLMs everyday. Honestly can't understand any serious software engineer who wouldn't use open models (again the control and tooling provided is just so much better), and for less technical users it's getting easier and easier to just run open models locally.

Re: Large Enough

#123

Post by RUnconcerned »

thierrydamiba wrote: post #82
swyx wrote: post #31

Earlier quoted context omitted.

indeed. I pointed out in https://buttondown.email/ainews/archive/ainews-llama-31-the-... that the frontier model curve is currently going down 1 OoM every 4 months, meaning every model release has a very short half life[0]. however this progress is still worth it if we can deploy it to improve millions and eventually billions of people's lives. a commenter pointed out that the amoutn spent on Llama 3.1 was only like…

Agreed on everything, but calling the marvel movies slop…I think that word has gone too far.

Not only are Marvel movies slop, they are very concentrated slop. The only way to increase the concentration of slop in a Marvel movie would be to ask ChatGPT to write the next one.

Re: Large Enough

#124

Post by mattnewton »

thierrydamiba wrote: post #82
swyx wrote: post #31

Earlier quoted context omitted.

indeed. I pointed out in https://buttondown.email/ainews/archive/ainews-llama-31-the-... that the frontier model curve is currently going down 1 OoM every 4 months, meaning every model release has a very short half life[0]. however this progress is still worth it if we can deploy it to improve millions and eventually billions of people's lives. a commenter pointed out that the amoutn spent on Llama 3.1 was only like…

Agreed on everything, but calling the marvel movies slop…I think that word has gone too far.

Not all Marvel films are slop. But, as a fan who comes from a family of fans and someone who has watched almost all of them; lets be real. That particular film, really and most of them, contain copious amounts of what is absolutely slop.

I don't know if the utility is worse than an LLM that is SOTA for 2 months that no one even bothers switching to however - at least the marvel slop is being used for entertainment by someone. I think the market is definitely prioritizing the LLM researcher over Disney's latest slop sequel though so whoever made that comparison can rest easy, because we'll find out.

Re: Large Enough

#125

Post by xeromal »

ThrowawayTestr wrote: post #102
xeromal wrote: post #88

Earlier quoted context omitted.

I don't know anything about AI but there's one thing I want it to do for me. Program a full body exercise program long term based on the parameters I give it such as available equipment and past workout context goals. I haven't had good success with chatgpt but I assume what you're talking about is relevant to my goals.

Aren't there apps that already do this like Fitbod?

Fitbod might do the trick. Thanks! The availability of equipment was a difficult thing for me to incorporate into a fitness program.

Re: Large Enough

#126

Post by freediver »

tikkun wrote: post #76
rkwz wrote: post #73

Earlier quoted context omitted.

> Longer context window (1M+) What's your use case for this? Uploading multiple documents/books?

Correct

That would make each API call cost at least $3 ($3 is price per million input tokens). And if you have a 10 message interaction you are looking at $30+ for the interaction. Is that what you would expect?

Re: Large Enough

#127

Post by littlestymaar »

splwjs wrote: post #78
__jl__ wrote: post #34

Earlier quoted context omitted.

Meta just claimed the opposite in their Llama 3.1 paper. Look at the conclusion. They say that their experience indicates significant gains for the next iteration of models. The current crop of benchmarks might not reflect these gains, by the way.

I sell widgets. I promise the incalculable power of widgets has yet to be unleashed on the world, but it is tremendous and awesome and we should all be very afraid of widgets taking over the world because I can't see how they won't. Anyway here's the sales page. the widget subscription is so premium you won't even miss the subscription fee.

Except: Meta doesn't sell AI at all. Zuck is just doing this for two reasons:

- flex

- deal a blow to Altmann

Re: Large Enough

#128

Post by bla3 »

crystal_revenge wrote: post #122
happyhardcore wrote: post #17

Earlier quoted context omitted.

I suspect this is why OpenAI is going more in the direction of optimising for price / latency / whatever with 4o-mini and whatnot. Presumably they found out long before the rest of us did that models can't really get all that much better than what we're approaching now, and once you're there the only thing you can compete on is how many parameters it takes and how cheaply you can serve that to users.

> the only thing you can compete on is how many parameters it takes and how cheaply you can serve that to users. The problem with this strategy is that it's really tough to compete with open models in this space over the long run. If you look at OpenAI's homepage right now they're trying to promote "ChatGPT on your desktop", so it's clear even they realize that most people are looking for a local product. But once ag…

I think their desktop app still runs the actual LLM queries remotely.

Re: Large Enough

#129

Post by lolinder »

Liquix wrote: post #10

These companies full of brilliant engineers are throwing millions of dollars in training costs to produce SOTA models that are... "on par with GPT-4o and Claude Opus"? And then the next 2.23% bump will cost another XX million? It seems increasingly apparent that we are reaching the limits of throwing more data at more GPUs; that an ARC prize level breakthrough is needed to move the needle any farther at this point.

> It seems increasingly apparent that we are reaching the limits of throwing more data at more GPUs

Yes. This is exactly why I'm skeptical of AI doomerism/saviorism.

Too many people have been looking at the pace of LLM development over the last two (2) years, modeled it as an exponential growth function, and come to the conclusion that AGI is inevitable in the next ${1-5} years and we're headed for ${(dys|u)topia}.

But all that assumes that we can extrapolate a pattern of long-term exponential growth from less than two years of data. It's simply not possible to project in that way, and we're already seeing that OpenAI has pivoted from improving on GPT-4's benchmarks to reducing cost, while competitors (including free ones) catch up.

All the evidence suggests we've been slowing the rate of growth in capabilities of SOTA LLMs for at least the past year, which means predictions based on exponential growth all need to be reevaluated.

Re: Large Enough

#130

Post by mattnewton »

layer8 wrote: post #62
tikkun wrote: post #39

Earlier quoted context omitted.

When using a prompt that involves thinking first, all three get it correct. "Count how many rs are in the word strawberry. First, list each letter and indicate whether it's an r and tally as you go, and then give a count at the end." Llama 405b: correct Mistral Large 2: correct Claude 3.5 Sonnet: correct

It’s not impressive that one has to go to that length though.

You can always find something to be unimpressed by I suppose, but the fact that this was fixable with plain english is impressive enough to me.
Post reply on HN