Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I think Anthropic and OpenAI have found product-market fit

simonwillison.net

Post reply on HN
Print view View original

761–770 of 1001 posts

Re: I think Anthropic and OpenAI have found product-market fit

#761

Post by bendews »

stopachka wrote: post #478
FuriouslyAdrift wrote: post #246

Earlier quoted context omitted.

I work for a tiny little company ($150MM annual rev with 9% net) and we are already looking at dropping $100k on hardware to run local models because, for us, they're "good enough." Our estimated spend for AIaaS would exceed that cost in less than a year. In a few years, there will be hardware capable of running frontier models good enough for most things at accessible prices for even tiny companies.

I don't quite understand, what would 100K buy you? AFAIK you would get about ~5 concurrent users, with a max context window of ~128K tokens on the larger models. This wouldn't be good enough for coding -- are you guys thinking of using it for something else?

By my calculations 100k could get you 18 5090's + compute to host them, or 18 96gb Mac mini's. You can get a lot of context window and users out of that setup.

Re: I think Anthropic and OpenAI have found product-market fit

#762

Post by quality_life »

trjordan wrote: post #11

They've got, ballpark, $5t to $10t to make back in the next 5 years, or the hardware buildouts will start getting written down. This means we're going to need $1t+ per year in spending, per year, on tokens. 200m knowledge workers in the world, 30m developers. We're talking about a world where you need 5% of every knowledge workers salary to go into tokens. 20% if you're a developer. That's a _huge_ shift. Most people…

Also, with announcements of replacing developers with AI and consequent job losses, who is going to use the tokens? AI using its own tokens to produce code?

Re: I think Anthropic and OpenAI have found product-market fit

#764

Post by AndrewKemendo »

trjordan wrote: post #11

They've got, ballpark, $5t to $10t to make back in the next 5 years, or the hardware buildouts will start getting written down. This means we're going to need $1t+ per year in spending, per year, on tokens. 200m knowledge workers in the world, 30m developers. We're talking about a world where you need 5% of every knowledge workers salary to go into tokens. 20% if you're a developer. That's a _huge_ shift. Most people…

Your severely underestimating the idea that people are just not going to use developers for certain things in the future

For example I don’t anticipate somebody making a living off of making website ever again

Somebody with absolutely no technical experience who needs a website for their business can now make one with almost no money whatsoever.

That’s good enough for their business. and the code can be totally shit and it does not matter because it’s meeting their business objectives. I am seeing this in the wild and I’m paying money to companies that have these types of websites and because it doesn’t matter I don’t need for the website to work perfectly on all my devices all I need to be able to do is pay them through the website which is what they need me to do and our transaction is done.

Don’t forget ultimately the people who pay technologists right now are primarily advertisers

work on hard problems is going to continue to be some tiny fraction percentage of the software engineering discipline

just expect a total bloodbath because the goal isn’t developer productivity the goal is that “I don’t need to pay somebody $200,000 a year to build a website authoring tool like WordPress.”

Re: I think Anthropic and OpenAI have found product-market fit

#765

Post by atq2119 »

IX-103 wrote: post #729
whatshisface wrote: post #173

Earlier quoted context omitted.

Here are a few thoughts: - The publicly available information about how inference costs compare to training costs is conflicted. EEs involved in datacenters talk about power usage spikes during training runs as if they were a major factor in the designs, but academic papers discussing cost-optimal scaling confidently treat inference-time compute as a major factor. - On the side of the balance indicating that training…

I don't see how it would be possible for inference costs to dominate training costs, even after amortization. Training involves multiple passes over the entire training dataset, ideally in large batches where you can perform inference on as many samples as possible simultaneously and then perform backpropagation to adjust the model weights (which is about as expensive as inference). Let's consider the size of the dat…

It's been a while since I saw a detailed paper on a high end training run, but extrapolating from what I remember, it seems those training runs are in the 10s of trillions of tokens. This already accounts for potentially sampling tokens multiple times during the training run.

That seems like a large number, until you realize that OpenAI claims to have almost a billion weekly users. And OpenRouter shows many models at over a trillion tokens per week.

So in pure token terms, I'd say it is in fact extremely plausible that inference dominates, at least for the popular models.

Re: I think Anthropic and OpenAI have found product-market fit

#766

Post by AndrewKemendo »

FuriouslyAdrift wrote: post #246
trjordan wrote: post #11

They've got, ballpark, $5t to $10t to make back in the next 5 years, or the hardware buildouts will start getting written down. This means we're going to need $1t+ per year in spending, per year, on tokens. 200m knowledge workers in the world, 30m developers. We're talking about a world where you need 5% of every knowledge workers salary to go into tokens. 20% if you're a developer. That's a _huge_ shift. Most people…

I work for a tiny little company ($150MM annual rev with 9% net) and we are already looking at dropping $100k on hardware to run local models because, for us, they're "good enough." Our estimated spend for AIaaS would exceed that cost in less than a year. In a few years, there will be hardware capable of running frontier models good enough for most things at accessible prices for even tiny companies.

That’s exactly where the market is heading and it’s going to have to reckon with this fact

My guess is there’s gonna be some legislation or something “you can’t share anything over this level of complexity” and I think that that’s what a lot of that mythos rattling was all about

Re: I think Anthropic and OpenAI have found product-market fit

#767

Post by AndrewKemendo »

repeekad wrote: post #480
sixhobbits wrote: post #216

Earlier quoted context omitted.

Pmf is this weirdly defined thing where "if you're not sure you have it then you don't". I think it was clearly useful for months to people who had tried it and taken the time to understand it, but now that knowledge has spread to the point where wallet holders are convinced it's not just passing fad or hype so now pmf can be "claimed". I agree it's weird to say "those people have pmf" though, usually it's something…

> clearly useful for people who took the time to understand it people -> programmers, I haven’t met a non-developer who reports getting more time out of current AI platforms than they put in. If anything I’ve anecdotally heard the opposite, introducing AI at work creates so much slop (output) it takes more time to process it all without a tangible bump in overall productivity

I have at least a half dozen examples of people not hiring people or buying other tools/subscriptions because they built their own with Claude

Re: I think Anthropic and OpenAI have found product-market fit

#768

Post by whateveracct »

overgard wrote: post #506
Tuna-Fish wrote: post #248

Earlier quoted context omitted.

Why on earth would AI labs be bragging about how little the product they sell actually costs them to make? You don't want to do anything that reduces it's perceived value to the user, that might make them less willing to pay for it. Also, inference costs are bound to go way down with more optimized architectures. GPUs are fundamentally not great at inference. No platform where the weights are streamed from a large po…

A couple of years ago Altman was saying the price of AI compute is going to drop 90% year over year or something like that, so I don't think they're nervous about talking about lowering their costs. They probably just haven't been able to lower their costs. You have to keep in mind that about 99% of their announcements are targeted towards investors (their most important revenue source..), so they're not going to be…

he lied. he's good at that.

Re: I think Anthropic and OpenAI have found product-market fit

#769

Post by seattle_spring »

prepend wrote: post #14

> $2,180.16 worth of tokens for $200 “Tokens” don’t have an intrisic cost or value. Saying that I used $2,180.16 worth of tokens is like relying on the salesperson to convince me I’m getting a billion dollars worth of pots and pans for $19.99. I think it’s funny how we are throwing critical thinking out the window when it comes to evaluating biased sources of info.

Reminds me of a car dealership talking about how good of a deal their extended maintenance plan and warranties are.
Post reply on HN