Why aren't they using Claude code 20x for 200/month?
if you have more than x seats, you have to use Enterprise pricing as far as I know which is pay as you go with a pool.
Uber's $1,500/month AI limit is a useful signal for AI tool pricing
301–310 of 819 posts
Re: Uber's $1,500/month AI limit is a useful signal for AI tool pricing
#302Earlier quoted context omitted.
That's just Simon Willison since LLMs came out. It's glaringly obvious that he's a paid shill.
oh come on, a paid shill? Simon is very fascinated by AI and at times he can be a little too optimistic but he is generally balanced and his perspective evolves over time which can be seen in his writing. Nerd who loves nerd things a little too much? Sure. Paid shill by Big LLM? Nah.
Yes, a paid shill. You can find a clear point in time where he shifted from sceptic to 1000% fully onboard non-stop praise, with no reason.
Re: Uber's $1,500/month AI limit is a useful signal for AI tool pricing
#303Is anyone doing story point estimation in terms of tokens? If you have a token budget, does this change how you prioritize?
Re: Uber's $1,500/month AI limit is a useful signal for AI tool pricing
#304> I noted that my own token usage comes to about $1,000/month against each of Anthropic and OpenAI - which currently costs me just $100 per provider thanks to their generous subsidized plans for individual subscribers. Do we know that AI providers are going to keep these per-token prices, or eventually lower them because of competition from China? Many lower-budget individuals are now moving to China open weight mode…
One aspect Paul Kedrosky mentioned recently is the concept of „duration mismatch“. The price per token goes down over time (either because the AI vendor reduces due to competition pressure, or because customers are now incentivized to use older cheaper models). But datacenters are financed through debt, with the assumption their revenue increases over time. Quoting him: „[AI vendors are] paying for a fixed cost with…
"So you have on one end the token revenue trending down, on the other end the training cost going up for the next frontier models, and you need to pay back your 10y debt."
Not necessarily, the bond holders could simply take a massive hair cut and lose shitloads of money. On the topic of bubbles and exuberance, Jeff Bezos made the salient point that there was a massive over-invested biotech boom in the 1990s and tons of sophisticated investors ended up losing lots of money. But humanity still kept the medical advancements made by the boom. Stocks going down didn't un-research drugs, and it won't un-research new GPUs or un-build datacenters.
Re: Uber's $1,500/month AI limit is a useful signal for AI tool pricing
#305Why there are so many people that still believe that AI coding is a fad? It's something that started less than two years ago and companies are already paying thousands per seat. I know one that gives you 5k per month. Which other tool went from nothing to this level of acceptance so quickly?
Because companies are betting that this spending will allow them to reduce cost by firing people. Right now the AI LLM PRs we're seeing are just introducing more work for other people, while these so-called builders are looking good with their new dashboards and functionality they're demoing. But you can't talk to them about the flow of the code. You can't ask them for their thinking as to why certain things are. It'…
It's so fucking bad. I'm watching a team try to maintain a huge dashboard/control application that interfaces with a large amount of hardware using solely AI workflows.
Literally nothing works, all the timers/time counters are different across the pages, constantly commands hardware to do stupid shit, breaks during critical moments/in front of clients.
Eventually mgmt had to institute change freezes for high profile events because the team was breaking too much shit all the time.
The average C suite dipshit doesn't realize that the performance drops off a cliff once your project is more than some fraction of the context window so they will make pretty dashboards all day long but once you need to cover all the edge cases of a real system it all explodes.
AI isn't trained on the type of software style we'll need to create systems using AI, it's trained on how we used to write software. It doesn't reuse code or elegantly structure annoying, it just adds more code until the thing builds and passes some fake tests, even if half of it is functionally dead/unused.
Re: Uber's $1,500/month AI limit is a useful signal for AI tool pricing
#306Why isn't self hosting (even just renting a GPU server, not necessarily on premise) at large companies or hosting via something like together AI to run the open weight models not more common? I've tried the open weight models and the premium models like Opus and Gemini Pro, and I find that the latter are a little better, but not nearly to the degree to justify the extreme price difference, since the differences large…
For the same reasons companies are not building data centers for their "regular" hosting and storage needs but put things on AWS, Azure etc.
It costs money to maintain the hardware and hire experts to manage the services. For something as common as LLM models, there is absolutely no reason a company serves models on their own hardware unless they are maniac about sending bytes to AWS.
Re: Uber's $1,500/month AI limit is a useful signal for AI tool pricing
#307If a worker doesn't use their AI/LLM budget, can they get a raise?
Re: Uber's $1,500/month AI limit is a useful signal for AI tool pricing
#308Earlier quoted context omitted.
> But you can't talk to them about the flow of the code. You can't ask them for their thinking as to why certain things are. You can absolutely do this. It's even right most of the time.
Let's be real. Most of the time you ask an LLM "Why did you do it like this?", it responds with something along the lines of "Oops. My bad. You're right to point this out." You even have a fair chance of getting a response like that when there isn't anything wrong and the question wasn't rhetorical - which perfectly illustrates the level of the genuine understanding LLMs operate at.
When you criticize AI, always remember that the alternative is the average employee. Today's models are pretty good.
Re: Uber's $1,500/month AI limit is a useful signal for AI tool pricing
#309Earlier quoted context omitted.
do GPU chips really depreciate physically? There are no moving parts, I dont think memory chips or GPU chips deteriorate naturally. I think its only accounting depreciation. I have been using my laptop for a decade, what is stopping datacenters from using the purchased GPU chips for a decade?
Today's data center GPUs are essentially overclocked, and so at limit of how much the chip materials can physically handle, and therefore degrade over time. For example, GH200s operate at 1W/superchip but the actual safe power is somewhere around 650W which will allow them to function for a decade or more. But that leads to around 15% slowdown and that is unacceptable in today's competition. So current GPUs are desti…
I would presume the reason they are overclocked is because they are trying to make up for the shortage. In time, the shortage of computing components will be remedied, and tokens produced at lower power pulls will be cheaper.
Re: Uber's $1,500/month AI limit is a useful signal for AI tool pricing
#310Earlier quoted context omitted.
do GPU chips really depreciate physically? There are no moving parts, I dont think memory chips or GPU chips deteriorate naturally. I think its only accounting depreciation. I have been using my laptop for a decade, what is stopping datacenters from using the purchased GPU chips for a decade?
Gradually, and especially when hot. Modern chips are pretty close to the physical limits of how small they can be made, and that means atomic/chemical effects like electromigration are accounted for and determine the lifetime. Every extra 10 degrees Celsius of temperature doubles the speed of chemical reactions. When they stray too close to the line ... you get Intel's 13/14th gen chips that wear out after 1-2 years…
sounds like planned depreciation on Intel's part, they definitely do not design server grade chips for longevity since that would harm their own revenues