Live data from Hacker News

Access to frontier AI will soon be limited by economic and security constraints

writing.antonleicht.me

161–170 of 227 posts

Re: Access to frontier AI will soon be limited by economic and security constraints

#161
post #94
post #66

Over on the image generation side, "frontier AI" seems to be coming along rather well. Watch this video, which was released eight days ago.[1] Can you find any flaws? Two years ago, just getting hands with the right number of fingers was tough. Last year, there were jarring errors in every scene. Now, very little is wrong. How much longer will anyone need Hollywood studios? [1] https://www.youtube.com/watch?v=4zTCLIh…

And even if there weren’t any jarring errors, and rest assured there’s about a billion of them, there’s no appeal to this. It’s all context free short unassociated clips of pretty faces dancing on a beach. And? There’s no narrative, there’s now sense of reality, it’s just a sense of here’s a million pixels of colours that have proven to go well with each other, it’s _slop_. It’s been years and the only place AI has c…

How elitist of you to belittle 95% of all creators who are no better than that.

Re: Access to frontier AI will soon be limited by economic and security constraints

#162
Physics and economics will drive cost. Current token pricing is based on unsustainable investment and energy cost. However, this is more of an optimization problem than an inherent show stopper. Token cost will inevitably come down over time. But this could take a while before it catches up with demand. Manufacturing will step up to provide cheaper GPUs. Etc. There will be some consolidation but the whole thing will converge on something that should make long term economical sense.

Ultimately it's a resource control issue. To power AI you need land/space (to build on), water, energy, and lots of hardware. Hardware needs to be manufactured and engineered. It needs metals, some exotic materials, machines, etc. More resources in other words. If you look at China vs US here, they are really well positioned in terms of resources and supply chains. The US has fallen behind quite a bit on energy and all the critical resources needed to produce hardware. AI is bottle necked on a lot of stuff that China has or makes in abundance.

For the frontier models, there are a growing number of companies and countries that provide them. We're used to mostly talking about the US ones. But of course the Chinese have a lot of capability here and they are not that far behind. And that's judging by the models they choose to release under OSS licenses. Those models are not their frontier models. And there are a lot of other countries developing and using models that aren't necessarily talking openly about what they are doing.

The irony with these frontier models is that they only generate revenue if people can use them. Why sink billions in AI infrastructure and models without a revenue model?

The reality with Mythos is that you have to assume that the Chinese (and others) are not that far behind and may already be running an equivalent model that they just haven't told anyone about yet. Anthropic gate keeping Mythos and its findings is probably wise. But it's not long term sustainable to depend on that happening or working very well. Or even on them even being a leader in this space.

This is becoming an arms race between countries, and economies. And it's an economical and resource control race. Developing and researching in the open has advanced things massively. But it has also empowered the rest of the world. Both Anthropic and OpenAI are staffed with people from all over the world. You have to assume that they probably aren't very good at keeping things secret.

Re: Access to frontier AI will soon be limited by economic and security constraints

#163

How much money are you all paying to use this tech? Last I even tried, it would cost my entire salary. Yet, everyone and their newborns are using it every day for everything. How is this possible?

Probably a combination of:

- people living in places with higher cost of living and corresponding salaries

- people whose employers are paying for it

- people who aren't actually using it but are being paid to hype it up online

- bots

Re: Access to frontier AI will soon be limited by economic and security constraints

#164
post #157

Earlier quoted context omitted.

> The more fundamental bottleneck is not even the frontier models, it's the datacenters. Is it even though? Quantization and speculative decoding are improving the local AI story by leaps and bounds every month.

There is "local AI" which is running on consumer grade hardware and "local AI" which still needs a datacenter (DeepSeek 4, GLM 4.7, etc). If you woke up tomorrow and could only use the latter you are about 6 months behind the frontier, if you have to rely on the former you are 2 or 3 years behind. All these tricks like quantization and speculative decoding can also be used by the leading AI labs, which means they wil…

Nothing released so far inherently "needs" a datacenter, it's just a matter of how much performance you require. Slow, high-latency inference will be a natural way to run "datacenter" models locally.

Re: Access to frontier AI will soon be limited by economic and security constraints

#165
post #160

In our company of 24 employees, we get by with two DGX Sparks. We don't use AI heavily, but each Spark can serve about 6-8 concurrent requests with a full context lenght of 256k, which is decent. We get about ~35 t/s depending on the model we use (currently Qwen3.5 122B A10B and Qwen3 Coder Next), but we might set up a smaller model too for simpler tasks. This works for us and will work for years to come. It is not S…

That's pretty nice actually, how much KV cache does that model require at full context? That tends to be the main limit to running concurrent requests locally, there's KV quantization but it has outsized negative impact on model quality.

Re: Access to frontier AI will soon be limited by economic and security constraints

#166

Earlier quoted context omitted.

Today's tech echoes 1960-1970 mainframe era: very centralized around a handful of companies controlling "massive cloud compute" in bespoke mainframe-like topology. All of that will all be legacy in a couple of years. Today's B200 clusters are tomorrow's e-waste. Decentralization might happen gradually or abruptly. But to me it's obvious that we'll be thinking of high-tech tensor processors and GPUs the way we thought…

One thing that is potentially different this time is that Moore's Law has stopped scaling. Computers aren't getting smaller exponentially. They're getting bigger with multiple chips glued together to make up for Moore's Law.

...But there's a new world dawning for photonic chips.

No reason to expect Moore's observation to apply there (though, maybe?), but it will have big implications for power usage.

Re: Access to frontier AI will soon be limited by economic and security constraints

#167

Physics and economics will drive cost. Current token pricing is based on unsustainable investment and energy cost. However, this is more of an optimization problem than an inherent show stopper. Token cost will inevitably come down over time. But this could take a while before it catches up with demand. Manufacturing will step up to provide cheaper GPUs. Etc. There will be some consolidation but the whole thing will…

Those billions in AI datacenter infrastructure will eventually be repurposed to run smart models like Mythos, not ChatGPT or even Opus/Sonnet. That future "revenue model" is quite robust to any foreseeable competition from on-prem FLOSS inference. It's a natural fit to the actual capabilities of large datacenter-scale compute.

Re: Access to frontier AI will soon be limited by economic and security constraints

#168
We should be aiming for less token usage, ideally none at all. The current AI is using LLMs to expanding horizontally but with the goal of achieving vertical progress - inventing truly new stuff and being able to eliminate our biggest problems. problems like cancer need only be solved once, and is no more tokens needed after that.

Re: Access to frontier AI will soon be limited by economic and security constraints

#169

How much money are you all paying to use this tech? Last I even tried, it would cost my entire salary. Yet, everyone and their newborns are using it every day for everything. How is this possible?

Depends on which exact model we're talking about, and on your salary.

For example, with the $40/month Kimi Code subscription the limits are so generous that you can use it every day all the time for everything (basically just have an agent constantly running doing something) and never run out of tokens/hit the limits.

Re: Access to frontier AI will soon be limited by economic and security constraints

#170
post #157

Earlier quoted context omitted.

There is "local AI" which is running on consumer grade hardware and "local AI" which still needs a datacenter (DeepSeek 4, GLM 4.7, etc). If you woke up tomorrow and could only use the latter you are about 6 months behind the frontier, if you have to rely on the former you are 2 or 3 years behind. All these tricks like quantization and speculative decoding can also be used by the leading AI labs, which means they wil…

Nothing released so far inherently "needs" a datacenter, it's just a matter of how much performance you require. Slow, high-latency inference will be a natural way to run "datacenter" models locally.

Yes it does. You will not be able to run models like DeepSeek v4 (>1.5 trillion parameters) on a regular workstation any time soon, unless by "slow" you mean "unusable". And those are the models that are ~6 months behind Opus 4.7.
Post reply on HN