Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I love LLMs, I hate hype

geohot.github.io

Post reply on HN
Print view View original

71–80 of 340 posts

Re: I love LLMs, I hate hype

#71

Post by ekidd »

dom96 wrote: post #39

I love LLMs too, but I am concerned about their cost. They are all still very subsidised. Is there any guarantee that I'll be able to run a Opus 4.8-level model on my personal computer before the big AI labs decide to hike up the prices?

You can maybe run a local Sonnet-4.5-ish-level model (sort of) for less than the price of a new car, even at current massively inflated prices for fast RAM. This is probably not what you were looking for. But it's there. You could share one server between multiple developers. Maybe make a little AI co-op or something, with a pair of RTX Pro 6000 cards?

Also, DeepSeek V4 Pro is cheap via any commodity API, and DeepSeek V4 Flash is essentially free at API prices like $0.09/M, $0.18/M out. This is generally not subsidized.

For a more practical local setup, Qwen3.6 27B on a used Nvidia 3090 (US$1300) or two is surprisingly nice. It needs clear instructions and you can't use it for hands-off vibecoding, but it's actually quite reasonable for hands-on programmers.

Re: I love LLMs, I hate hype

#72

Post by fragmede »

dom96 wrote: post #39

I love LLMs too, but I am concerned about their cost. They are all still very subsidised. Is there any guarantee that I'll be able to run a Opus 4.8-level model on my personal computer before the big AI labs decide to hike up the prices?

GLM-5.2 is runnable and downloadable today on a MacBook studio that costs a stupid amount of money. No one can take that away from you except by force though, if you want to set it up today.

Re: I love LLMs, I hate hype

#73

Post by pphysch »

kordlessagain wrote: post #2

There's good reason to hate the merchants and their marketing. But builders are not merchants. They build with whatever tool is available.

Geohot is one of the (attempted) merchants, but maybe that is not going so well and he is changing his tune.

Re: I love LLMs, I hate hype

#75

Post by theli0nheart »

Aurornis wrote: post #50
dom96 wrote: post #39

I love LLMs too, but I am concerned about their cost. They are all still very subsidised. Is there any guarantee that I'll be able to run a Opus 4.8-level model on my personal computer before the big AI labs decide to hike up the prices?

> They are all still very subsidised. I think the opposite: I think the frontier labs have good margins on their inference unit costs. We can already see what it costs to run near frontier-size models. There are independent business pivoting to serving these models at reasonable prices and they're competing on OpenRouter for costs much lower than frontier labs. > Is there any guarantee that I'll be able to run a Opus…

They may have good margins, but a few things are still true:

1. Much of those profits have to be immediately reinvested into model training runs to avoid being lapped by competitions.

2. Unit costs are irrelevant when the labs don't price per unit, and instead charge, for instance, $200 / month for $10k worth of tokens.

This isn't a steady state. Whatever the current situation is, I doubt it's sustainable.

Re: I love LLMs, I hate hype

#76

Post by cyanydeez »

blauditore wrote: post #43
hamandcheese wrote: post #10

> where’s all this new magical software that the productivity improvements should imply? It's running, privately, in my homelab. I think we are entering what I call the "have it your way" era. If an open source project doesn't do exactly what you want it to do, fork it, or create a new version. It's too easy. This makes me a bit concerned about the future of open source. Upstreaming used to be worth it, since maintai…

You will likely end up in maintenance hell soon. This will likely not be much easier with AI because coding is not the hard/annoying part, it's the fact that you need to dust off every little project every time a tiny fix is needed, and that's a lot of toil in the long run.

alternatively, you might end up in 'good enough heaven' and not have to touch it for a decade because, you know, it does exactly as you need and you're not google, microsoft, openAI or antrhopic.

I'd bet there's far more 'good enoughs' than anything else out there. One of the reasons microsoft office is constantly churning subscription, etc is because they solved good enough decades ago and need to justify valuations that just don't matter for most of their user's use cases.

Not everyone is a software developer having to churn out the 101th SaaS that's just because some MBA refuses to hire a dev.

Re: I love LLMs, I hate hype

#77

Post by ghthor »

fwlr wrote: post #58

I get it, I want to agree, I really do like the “this is a new tool in the toolkit of the professional software craftsperson” argument… …but consider: the Q-tip. “Don’t use it to clean your ears”, but for most people that’s all they want to do with it, and empirical observation indicates that this dynamic results in either “using Q-tips irresponsibly” or “not using Q-tips”, with “uses Q-tips properly” being a small-t…

Qtips are made for cleaning your ears. It says not to do that so they are NOT sued every time some idiot fucks up their ear with one.

Re: I love LLMs, I hate hype

#79

Post by cyanydeez »

midasz wrote: post #62
paulryanrogers wrote: post #11

Earlier quoted context omitted.

You still have to track upstream and merge conflicts. Or else you have to get LLMs to fix all the CVEs in your fork.

I'm guilty of creating a fork that just goes off the rails, but still needs to keep up with upstream. I do it via a skill and seems to work good enough for now: https://github.com/midasvo/findroid-ce/tree/main/.claude/ski...

I'm doing it right now to see what the cost is; I cloned the upstream and made a copy of the working directory and asked the Qwen3.6-35B-A3B model to merge my production files with the new upstream.

Since it's just a duplicate folder, I can always fall back if it fubars.

Re: I love LLMs, I hate hype

#80

Post by bayindirh »

hobofan wrote: post #44
atomicnumber3 wrote: post #21

Earlier quoted context omitted.

Remember: code is free as in "free puppy". FOSS communities were never valuable because of the code. It was the shared written and oral traditions that make the software useful, usable, and updated.

> that make the software useful, usable, and updated There is a lot of OSS software out there (e.g. in scientific communities) that I would say would barely qualify for each of those three attributes. The main reason it's valuable for the respective communities, is because it's the only thing that's available.

Developing scientific software is disproportionately hard though. Making it usable, useful and keeping it updated is even harder.

There's two reasons for that. The math is generally very unorthodox and alien for a seasoned developer, and software development practices are equally alien for the scientist who can understand and evolve the math behind it.

I have written a boundary element method evaluator for my Ph.D. not only math was alien, the required coding techniques for making it fast is very different for a standard developer. You have to have the perseverance and interest to do that. I chose that path intently and I do not regret a millisecond of it.

The problem is, if you don't have a dedicated team to continue that codebase (e.g.: like the Eigen team), your code is basically done and done. If somebody doesn't share the same passion, it's almost impossible for someone to take and carry it forward.

Oh, due to the math and optimizations, the code's structure need to be both documented and the next batch of developer(s) have to be tutored by the person who's giving the code to them.

Post reply on HN