111–120 of 819 posts
These are still at currently subsidized prices. We'll see if they think they're getting $1500/month of value when that buys significantly fewer tokens.
$1500/mo is $18,000/seat/annum. Maybe Microsoft and Nvidia are on to something. 128 GB machines that can run local LLMs are a bargain even if priced $5-8k. Yes, tok/s is not quite there, but that's probably OK since the bottleneck really isn't the code; it's WTF did Uber build with all of that spend? How did it meaningfully impact their revenue in a positive direction?
How is tok/s not a bottleneck I? I assume most people still use ai agents interactively rather than leaving them to do their own thing during the night. I find anything below 50 tps or so entirely unusable... Regardless its Apples to oranges anyway, inference is quite cheap for open weight models its just that Claude and OpenAI can charge very high margins compared to e.g. DeepSeek or various provider on OpenRouter s…
In my experience, this is far below the cost the average dev will incur per month so this seems very reasonable to me. And, no doubt there are exceptions for heavy users so they can get some extra token usage when they need it.
higher ups pushed for these last 2 years to be AI focused so I don't think this restriction is a measure of "don't use too much AI" as much as it is a measure of "don't use only 'manual' AI tooling" since we had a dozen more specialized tools in-house running locally or otherwise that didn't count towards the budget
> I noted that my own token usage comes to about $1,000/month against each of Anthropic and OpenAI - which currently costs me just $100 per provider thanks to their generous subsidized plans for individual subscribers. Do we know that AI providers are going to keep these per-token prices, or eventually lower them because of competition from China? Many lower-budget individuals are now moving to China open weight mode…
How many more months do we need to wait, until big companies realize that flash models work just fine if you: 1) Don't ask LLMs for big changes 2) Review everything and point them in the right direction Large models still suck at big changes, they produce questionable architecture and you still have to review the code, if your project is serious enough. The codebase quickly become a mess, if you don't pay enough atte…
> Review everything and point them in the right direction
Sorry upper management doesn't care. That's an engineering problem that you need to solve.
Earlier quoted context omitted.
There is no evidence that per-token inference prices (which is what Uber is setting a cap on) is subsidized.
Is there any evidence that it's not?
Earlier quoted context omitted.
AI companies have more expenses than inference.
yes, and theres no evidence that they arent (or can't) use profitable inference to subsidise those other expenses. Some companies will keep spending massively to train better models, and some other companies will not, and offer good api prices. Which will end up being used? That depends on whether the spending turns into better value models
as far as we know there's no evidence that they can produce any profits at all
Earlier quoted context omitted.
Is there any evidence that it's not?
Yes; they ban various uses of their subscriptions but say you can do whatever if you’re paying for the API without limits
> I noted that my own token usage comes to about $1,000/month against each of Anthropic and OpenAI - which currently costs me just $100 per provider thanks to their generous subsidized plans for individual subscribers. Do we know that AI providers are going to keep these per-token prices, or eventually lower them because of competition from China? Many lower-budget individuals are now moving to China open weight mode…
I genuinely do not know how prices can get lower from the current major providers in NA without the whole market collapsing. Everyone is spending copious amounts of money to presumably make more money back.