Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I think Anthropic and OpenAI have found product-market fit

simonwillison.net

Post reply on HN
Print view View original

871–880 of 1001 posts

Re: I think Anthropic and OpenAI have found product-market fit

#871

Post by vanviegen »

whatshisface wrote: post #227
materielle wrote: post #211

Earlier quoted context omitted.

I'm about to leave a shallow comment, but I am a bit skeptical of the supposed drop in inference costs. If AI labs saw a lot of potential there, they'd surely be bragging about it non-stop? So the fact that publicly available information is conflicted is probably a sign that at the very least, the numbers aren't amazing. Yes I know there's no evidence and this is lazy reasoning. But there's probably a bit of truth to…

Inference has traditionally been far less expensive than training. One public example is the fact that hobbyists can run StableDiffusion ($600k training costs[1]) on their personal computers. Speaking to your point, inference being dramatically less costly than training would not be seen as a delta from the norm. The model of providing inference for anything near the operational costs (like a utility would), would th…

I think in your StableDiffusion example, a lot more than $600k will have been spend on electricity alone for inference (on those personal computers you mention). So inference is more expensive then training.

Re: I think Anthropic and OpenAI have found product-market fit

#872

Post by analog_daddy »

Hasz wrote: post #166
smokel wrote: post #49

Earlier quoted context omitted.

AutoCAD is $175 per user per month [1]. [1] https://www.autodesk.com/products/autocad/buy

Cadence and Ansys have entered the chat. A bunch of other highly-specialized engineering software has entered the chat. Licenses are on the order of 10-100k/seat. For a pretty funny comment about pricing. https://www.reddit.com/r/chipdesign/comments/1ajrli2/cadence...

Glad to run into this after some time!

I guess we are welcoming the software people to the world of expensive tools. Just sad that the FOSS alternatives of these tools are not as powerful whereas software industry still has FOSS tools to fall back on.

Re: I think Anthropic and OpenAI have found product-market fit

#873

Post by throwatdem12311 »

If they’ve spent all this cash, and all we have to show for all this AI baloney are crappy coding agents that constantly make the same mistakes no matter how much you try to guide them.

All the slop content, all the bots, all the misinformation and fake AI images and videos.

All of the social and economic disruption from datacenter buildouts.

The massive nosedive in reliability on the world’s software infrastructure.

After all of that and all we get is a code bot so a few incompetent loser devs can bloviate about not writing their own code and brag about never reading it.

Burn it all to the gd ground. Destroy this new Tower of Babel.

Re: I think Anthropic and OpenAI have found product-market fit

#874

Post by mike_hearn »

thesz wrote: post #443
whatshisface wrote: post #227

Earlier quoted context omitted.

Inference has traditionally been far less expensive than training. One public example is the fact that hobbyists can run StableDiffusion ($600k training costs[1]) on their personal computers. Speaking to your point, inference being dramatically less costly than training would not be seen as a delta from the norm. The model of providing inference for anything near the operational costs (like a utility would), would th…

The difference between training and inference is 1) one have to keep intermediate results for backward pass in training and 2) computation for training double because of the backward pass. Training is also done over batches, which increase memory requirements by several orders of magnitude. This is why training needs costly compute. One of the ways out of this unfortunate situation is to use something like Stochastic…

It's all got much more complex than that in recent years. Training now involves large amounts of inference for RL rollouts and similar. You can't disentangle them computationally like that. "Inference" is just the word used to mean serving customer traffic now, and "training" means creating the model you serve.

Re: I think Anthropic and OpenAI have found product-market fit

#875

Post by mold_aid »

MichaelDickens wrote: post #686
noddingham wrote: post #530

I feel like there's a bit of AI psychosis in this particular post. >"These are tools which burn vastly more tokens, but are also quickly becoming daily drivers for the work carried out by extremely well-compensated professionals." >"Somehow this fragment turned into headlines like Uber’s COO says it’s getting harder to justify the money spent on AI tokenmaxxing, because the market for stories about AI failures remain…

Being pedantic, but I don't want to lose the meaning of the term: "AI psychosis" doesn't refer to someone who thinks AI is really good. It refers to someone who develops symptoms of psychosis from talking to an LLM, e.g. believing they have developed a new Grand Unified Theory of physics.

I don't know, "workaday professionals will find $200/month a particularly good deal, such that there will be widespread adoption" sounds either credulous enough to support the diagnosis or dishonest enough to dismiss. I am a "knowledge worker" who is doin' fine, has a lot of templated written work/report writing, and there is no way in hell I am justifying that kind of spending to my boss or my family.

Re: I think Anthropic and OpenAI have found product-market fit

#876

Post by Jackpillar »

hamdingers wrote: post #159
121789 wrote: post #95

Earlier quoted context omitted.

uber sure....but how did wework survive? they are a smoldering husk of a failed company looted by its founder

I'm sitting in one right now and don't see any smoldering...

It's snowing right now therefore climate change is fake

Re: I think Anthropic and OpenAI have found product-market fit

#877

Post by joshuahedlund »

lumost wrote: post #516
materielle wrote: post #211

Earlier quoted context omitted.

I'm about to leave a shallow comment, but I am a bit skeptical of the supposed drop in inference costs. If AI labs saw a lot of potential there, they'd surely be bragging about it non-stop? So the fact that publicly available information is conflicted is probably a sign that at the very least, the numbers aren't amazing. Yes I know there's no evidence and this is lazy reasoning. But there's probably a bit of truth to…

For equal capability tokens, there has been about a 10x drop in cost every 6 months. We are still chasing the best because the best is moving rapidly, but it’s a simple thought experiment to work out what the cost to serve an 8B model from 2 years ago is in a world of 2T models. Note: parameter counts are illustrative. Concretely, qwen3.6 27B delivers opus 4.5 capability at 1/27th the cost on openrouter. Single chip…

> For equal capability tokens, there has been about a 10x drop in cost every 6 months

Is this still happening? Opus 4.5 was six months ago, can you get its capabilities for 1/10 cost now? Are we on track to get the same for 4.6 in a couple months?

Re: I think Anthropic and OpenAI have found product-market fit

#878

Post by prepend »

simonw wrote: post #16
prepend wrote: post #14

> $2,180.16 worth of tokens for $200 “Tokens” don’t have an intrisic cost or value. Saying that I used $2,180.16 worth of tokens is like relying on the salesperson to convince me I’m getting a billion dollars worth of pots and pans for $19.99. I think it’s funny how we are throwing critical thinking out the window when it comes to evaluating biased sources of info.

I'm not sure what you're pushing back against here. I spent $200. If I had been paying API pricing it would have been $2,180.16. The article is about how enterprise customers get charged API pricing, which means if I had been employed by one of those companies I would have cost them $2,180.16. What am I missing?

Lets say McDonalds charges $2000 for a BigMac. If they offer a deal and sell it to you for $200, did you save $1800?

Maybe if you spend $2000 on a BigMac. But it’s unlikely you would buy such a burger.

What is a hamburger worth? Don’t look to McDonalds to set the value.

Re: I think Anthropic and OpenAI have found product-market fit

#879

Post by prepend »

troyastorino wrote: post #20
prepend wrote: post #14

> $2,180.16 worth of tokens for $200 “Tokens” don’t have an intrisic cost or value. Saying that I used $2,180.16 worth of tokens is like relying on the salesperson to convince me I’m getting a billion dollars worth of pots and pans for $19.99. I think it’s funny how we are throwing critical thinking out the window when it comes to evaluating biased sources of info.

Tokens do have a clearly calculable intrinsic cost. There's the marginal cost of production (i.e. the inference cost) and the amortized R&D cost that goes into the model producing them. Yes, value is hard to calculate, but luckily market pricing mechanisms exist exactly for this purpose. There isn't a better number to use than what people are willing to pay for them. So he's saying that on an enterprise plan, he'd be…

There’s a cost, certainly. I expect Anthropic knows. But we don’t know what that is.
Post reply on HN