31–40 of 60 posts
Much as people may not want to like it, "software" as a product to buy and sell, even as a subscription, is probably going away, and will make about as much sense as "math" as a product.
We were already headed in this direction, but AI's going to rapidly accelerate this.
Earlier quoted context omitted.
> If you buy your own GPU, rack it, and run an open model, there is no unit cost. No. There is economic opportunity cost (borrowing), energy cost, infra cost, depreciation / risk of failure with each unit of work, bandwidth, maintenance, and lots more. Small, but not zero, and often overlooked - especially the opportunity cost.
These are basically all fixed costs, not unit costs. You buy the GPU once and use it for as many calls as you have traffic for, and depreciate it over a fixed lifetime. You have to power it regardless of whether it's fully utilized or not. You have to maintain it by virtue of owning it, not really based on how many queries it has served. Bandwidth is the only one that really scales as a unit cost. Open question wheth…
So the only way it’s a fixed cost is if you don’t pay for power. If you only consider the cost of the power, it might still be cheaper paying for an api.
>Users began expecting something fundamentally different from software: not just tools that store and retrieve, but products that reason, generate, and respond. Not really. >Every inference call costs money. Not really, either. If you buy your own GPU, rack it, and run an open model, there is no unit cost. This is just expensive hosting infra. You also pay unit costs for SaaS that your software uses (things like SMS…
> If you buy your own GPU, rack it, and run an open model, there is no unit cost. No. There is economic opportunity cost (borrowing), energy cost, infra cost, depreciation / risk of failure with each unit of work, bandwidth, maintenance, and lots more. Small, but not zero, and often overlooked - especially the opportunity cost.
We need a better word for this because the things you didn't do are not a cost
Earlier quoted context omitted.
> If you buy your own GPU, rack it, and run an open model, there is no unit cost. No. There is economic opportunity cost (borrowing), energy cost, infra cost, depreciation / risk of failure with each unit of work, bandwidth, maintenance, and lots more. Small, but not zero, and often overlooked - especially the opportunity cost.
> opportunity cost We need a better word for this because the things you didn't do are not a cost
>Users began expecting something fundamentally different from software: not just tools that store and retrieve, but products that reason, generate, and respond. Not really. >Every inference call costs money. Not really, either. If you buy your own GPU, rack it, and run an open model, there is no unit cost. This is just expensive hosting infra. You also pay unit costs for SaaS that your software uses (things like SMS…
> If you buy your own GPU, rack it, and run an open model, there is no unit cost. No. There is economic opportunity cost (borrowing), energy cost, infra cost, depreciation / risk of failure with each unit of work, bandwidth, maintenance, and lots more. Small, but not zero, and often overlooked - especially the opportunity cost.
The inference cost discussion is interesting but I think it misses the more consequential shift. The real change in unit economics isn't what it costs to run the model — it's that the marginal cost of building a new feature dropped by an order of magnitude. Previously the bottleneck was engineering time. Now a competent person with a frontier model can prototype in hours what used to take a team weeks. That compresse…
Earlier quoted context omitted.
As if making all the decisions that go into that product, hosting it, and maintaining it are free! Now do that for all the SaaS you subscribe to. Would you even get any real work done?
There wouldn't be a product. Consumers would just use the underlying foundational models directly to solve their problem. ChatGPT and Claude and Gemini are the products, not your business. Products are only viable when you do a lot of work to solve a problem, which is then shared by many potential customers. It makes sense to amortize the high costs of solving the problem across all the different customers to reap ec…
How am I going to use ChatGPT instead of, say, Figma or an office suite with my coworkers?? It's no use for it to spin up a clone for me if my coworkers can't collaborate. And if they can, what it will serve is a product.
How can it be cheaper for every company to re-invent the wheel?
> Products are only viable when you do a lot of work to solve a problem, which is then shared by many potential customers.
Yes, that's what businesses pay services for.
>Users began expecting something fundamentally different from software: not just tools that store and retrieve, but products that reason, generate, and respond. Not really. >Every inference call costs money. Not really, either. If you buy your own GPU, rack it, and run an open model, there is no unit cost. This is just expensive hosting infra. You also pay unit costs for SaaS that your software uses (things like SMS…
> If you buy your own GPU, rack it, and run an open model, there is no unit cost. No. There is economic opportunity cost (borrowing), energy cost, infra cost, depreciation / risk of failure with each unit of work, bandwidth, maintenance, and lots more. Small, but not zero, and often overlooked - especially the opportunity cost.
TFA contends that there is some fundamental shift in the economics of software, but it doesn't look to be very different from either a new SaaS dependency or racking new hardware.
Earlier quoted context omitted.
> If you buy your own GPU, rack it, and run an open model, there is no unit cost. No. There is economic opportunity cost (borrowing), energy cost, infra cost, depreciation / risk of failure with each unit of work, bandwidth, maintenance, and lots more. Small, but not zero, and often overlooked - especially the opportunity cost.
These are basically all fixed costs, not unit costs. You buy the GPU once and use it for as many calls as you have traffic for, and depreciate it over a fixed lifetime. You have to power it regardless of whether it's fully utilized or not. You have to maintain it by virtue of owning it, not really based on how many queries it has served. Bandwidth is the only one that really scales as a unit cost. Open question wheth…
Spoken like a guy who has never had to maintain bare metal infrastructure ops at scale. These things break, need re-imaged, have parts that break, have to be configured (now you need provisioning pipelines, monitoring, alarms, etc), have to be maintained when something goes wrong (swapping hardware and software, debugging the alarms into actually figuring out which bits are broken and/or misconfigured), have to be catalogued, have to be planned for, deprecated, and the finances accounted for through complicated accounting to show investors the Capex at quarterly meetings.
Then you have to make fleet decisions on how much of each type of server you will want to buy, expanded storage, how long you will support each generation of server, when you will order new hardware, how to order new hardware, lag for real world installs, hiring actual humans to fly around to all of your datacenters and do the actual installations / maintainance / etc. Then you will have to do contracts with individual datacenter operators for margins, electricity rates, hosting contracts, white glove ops hourly rates, etc.
Businesses that own their own hardware tend to have a lot of employees whose jobs are maintaining it and running the business side of that.
Earlier quoted context omitted.
There wouldn't be a product. Consumers would just use the underlying foundational models directly to solve their problem. ChatGPT and Claude and Gemini are the products, not your business. Products are only viable when you do a lot of work to solve a problem, which is then shared by many potential customers. It makes sense to amortize the high costs of solving the problem across all the different customers to reap ec…
> Consumers would just use the underlying foundational models directly to solve their problem. ChatGPT and Claude and Gemini are the products, not your business. How am I going to use ChatGPT instead of, say, Figma or an office suite with my coworkers?? It's no use for it to spin up a clone for me if my coworkers can't collaborate. And if they can, what it will serve is a product. How can it be cheaper for every comp…
Companies not matching their prices to current reality, mostly.