Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I love LLMs, I hate hype

geohot.github.io

Post reply on HN
Print view View original

321–330 of 340 posts

Re: I love LLMs, I hate hype

#321

Post by 4ffad »

int_19h wrote: post #176
dom96 wrote: post #39

I love LLMs too, but I am concerned about their cost. They are all still very subsidised. Is there any guarantee that I'll be able to run a Opus 4.8-level model on my personal computer before the big AI labs decide to hike up the prices?

We're supposedly getting Mac Studio with 1.5Tb RAM in 2 years. That would be enough to run an Opus-level model. Of course, it will also probably cost somewhere around $50k... But if local AI really does become pervasive, maybe it'll be one of the things people buy on credit, like cars.

"Of course, it will also probably cost somewhere around $50k..."

Whats to stop people remotely accesssing this? People already do this when working remotely in finance - they connect to a virtual environment that does their work in spreadsheets lmao. nobody cares about the lag, managers certainly dont care about sub-ordinates complaining about it - the same way nobody will care about a slight loss of quality if the economics make sense. frontier labs are screwed really.

Re: I love LLMs, I hate hype

#322

Post by simoncion »

fragmede wrote: post #209
wizzwizz4 wrote: post #183

Earlier quoted context omitted.

Or you could just take a shower, which makes it easy to wipe the excess earwax from your ear.

Having heard of this shower thing before, it doesn't do the trick if your earwax is dry and waxy.

For me, a shower is a precondition for earwax removal with a cotton swab. Without one, I'd need to break out hard scraping tools... which strikes me as far more hazardous than running a swab around the outer orifice of my ear canal and surrounding exterior areas.

It'd be lovely if I had the sort of ears that produce wax that would just drain after five minutes' exposure to warm water vapor, but I do not. Given the popularity of cotton swabs, as well as the fairly-widespread commentary about how fantastic it feels to goop out earwax with them, I expect that most folks do not have ears like that. Perhaps OP does have the sort of ears that produce very fluid wax. Lucky them.

Re: I love LLMs, I hate hype

#323

Post by simoncion »

fragmede wrote: post #180
simoncion wrote: post #140

Earlier quoted context omitted.

> This is like shoving a sponge down your windpipe to remove mucus. In my personal experience, not using a cotton-tipped swab for the task is like cleaning a plate loaded with gunk and burned-on patches with one's bare hands rather than choosing to use a sponge and/or brush. You can do it, [0] but it's much more work, much more time consuming, or you get an inferior result. [0] In my case, I'd need to make one set of…

We live in the future. You can get an ear cleaning camera endoscope device for $40 next day Amazon delivery anywhere they reach.

Unless you have a regular problem with impacted earwax that flushing with earwax-softening solutions doesn't solve, I don't see what actual problem an endoscope solves.

We do live in the future, but there are a bunch of gizmos that the future provides that generally aren't worth the hassle.

Re: I love LLMs, I hate hype

#324

Post by bauldursdev »

SwellJoe wrote: post #81

This line: "this is my main argument against the valuation of frontier labs. It’s not that AI won’t create that much value, it’s that they won’t capture it." That is a very astute and concise way to explain everything about how the frontier labs are behaving and how they're trying to push more people to pay token rates for the best models. At the current subscription prices ($100 or $200 a month for a generous, thoug…

Why wouldn't inference get less expensive? The frontier will probably always be expensive but imagine if in 5-10 years Fable 5 is considered a low tier antiquated model, it is still somewhat capable.

Re: I love LLMs, I hate hype

#325

Post by bluegatty »

impure-aqua wrote: post #287
bluegatty wrote: post #158

Earlier quoted context omitted.

No, you switched to another SOTA model. You didn't switch to 'Random Corner Store Token Seller' down the street, did you? There 2-3 top players, that is not commodity. Commodity is when there are enough that none of them have market power or can set prices. 'Commodity' means you buy your tokens from the Grocery Store on their loan plan. Like consumer credit is a commodity. FYI predict this is roughly the way it will…

Commoditizarion is a process, not a binary state. If I run an oil refinery, my fractional distillation system needs to be reworked depending on the exact mixture of crude I'm taking as input. So there are still switching costs even in the textbook example of a commodity. Crucially though the exact upstream I use has minimal impact on the downstream. Closer equivalents, say, another barrel of WTI grade crude from a ne…

Oil varies a bit but it's a commodity.

There are 3 SOTA model makers, and they have pricing power.

The 'switching costs' is not the issue so much as the inherent control over the commodity.

Think OPEC - when they acted as a cohort - they raised prices dramatically by having enough control to 'set prices'.

When OPEC lost it's pricing power ... nobody could set prices.

Fable is considerably better than GLM5 and it will have a strategic input - there is just hardly any substitute for it.

If these were cars - we'd just use whatever fuel.

But these are 'F1 races' - if you have some low grade 'dirty fuel' you will lose the race. You must have the 'top fuel'. There are 3 provides who implicitly collude and set prices.

Re: I love LLMs, I hate hype

#326

Post by bluegatty »

SwellJoe wrote: post #240
bluegatty wrote: post #158

Earlier quoted context omitted.

No, you switched to another SOTA model. You didn't switch to 'Random Corner Store Token Seller' down the street, did you? There 2-3 top players, that is not commodity. Commodity is when there are enough that none of them have market power or can set prices. 'Commodity' means you buy your tokens from the Grocery Store on their loan plan. Like consumer credit is a commodity. FYI predict this is roughly the way it will…

There were two or three top players. As of this week there are at least five. xAI is apparently in the game with a new Mecha Hitler release, Meta seems to be back in the game, z.ai is biting the ankles of the big dogs...not hurting them, yet, but they aren't going to get any less capable. Google got caught flatfooted as they maybe didn't notice where the money is in LLMs, but they still have the inventors of the tech…

Those are announcements, not released integrated models.

There are two Tier 1 platforms today.

Meta, Google and XAi are formidable Tier 1.5 place, any one of which could rise to the fore.

My belief is that it will be Google and that probably only one of them will keep up in the long run.

There is a 'breaking point' when you start to get past 4-ish players - it really does start to introduce competitive pressures.

Your second point about Tier 2 substitution is valid, but a few things:

1) Tier 1 models are not a 'luxury good' - that has a different economic definition. They are for most applications today actually just the quality, rational choice.

2) Substitution will have different effects for different people, and you're right that AI for many tasks will be commiditized.

All of the profits in Mobile Phones go to Apple even though they are not the biggest player.

Almost all of the profits in Silicon go to the leading edge chips - even though there are a zillion fabs that make legacy chips.

Re: I love LLMs, I hate hype

#327

Post by bluegatty »

bluefirebrand wrote: post #306
jurgenburgen wrote: post #293

Earlier quoted context omitted.

> Fridges aren't really better at keeping food cold than they were 30 years ago, are they? Fridges have made huge leaps in energy efficiency, they’re easily 3-4 times better at cooling your food.

That's what I'm saying though. Energy efficiency is nice, it's a good improvement, but from an end user perspective the 30 year old fridge still keeps your food just as cold. If you were from 2025 and got trapped in the 1970s and needed to keep some milk from going sour for a day, you wouldn't be thinking "damn if only these old 1970s fridges worked more efficiently. I could easily accomplish this goal with a modern…

You think that a fridge is about 'keeping food cold'?

It's about ease of access, price, noise, convenience, durability, features.

My folks have this fancy 2 door thing, perfectly quiet, makes the best ice you can imagine, it's hidden into the cuppboards, it's energy efficient, has these crisper things, you can see in and reach around easy, lots of space. It's a better product.

Re: I love LLMs, I hate hype

#328

Post by marcus_holmes »

saghm wrote: post #249
marcus_holmes wrote: post #243

Earlier quoted context omitted.

The thing that we did in 1990-2000 was adopt new standards as the old ones became blocks on progress. I had a friend working in optical computing back in the late 80's that would wax lyrical about how optical computing was vastly superior to silicon back then. But it never took over because silicon worked well enough. If we've hit the limits of silicon then there are other options. We would need to reinvent huge chun…

The original claim from the parent comment was running a Fable-level comment within a decade. Even if you're right about whether it's possible that another model could support that level physically, do you really think that we'll figure it out and ramp up the infrastructure to profitably sell on come consumer hardware anywhere close to that soon?

Well, we did do similar stuff back then. All it takes is money ;)

Re: I love LLMs, I hate hype

#330

Post by saghm »

marcus_holmes wrote: post #328
saghm wrote: post #249

Earlier quoted context omitted.

The original claim from the parent comment was running a Fable-level comment within a decade. Even if you're right about whether it's possible that another model could support that level physically, do you really think that we'll figure it out and ramp up the infrastructure to profitably sell on come consumer hardware anywhere close to that soon?

Well, we did do similar stuff back then. All it takes is money ;)

You're talking about going from a single gigabyte to 16 terabytes in a low-power consumer form factor, which is 1600x. A generous estimate of the factor of the RAM sizes for low-power consumer devices between 1990 and 2000 would still be a few orders of magnitude short, if I'm doing my math right.

Examples of what exactly you're claiming is precedent for this would be helpful.

Post reply on HN