Earlier quoted context omitted.
Yes. Author here. OpenAI is at near 4bn ARR and growing (api plus SaaS)
Source?
Why AI Infrastructure Startups Are Insanely Hard to Build
121–130 of 190 posts
Re: Why AI Infrastructure Startups Are Insanely Hard to Build
#122Earlier quoted context omitted.
> In my humble opinion, a chat interface (API or not) does not a product make. Well, you are humbly wrong then. > You would just eventually train and deploy your own model because an LLM is not a product. Hallucinations aren't exclusive to LLMs it seems.
We’re already seeing a lot of competition between LLMs. They are quickly becoming commodities. Margins will approach zero and the real value proposition will be with consumer products that extend beyond an .
I agree with you. I'm using a couple of different LLMs depending on what I'm doing and what happens to be easiest but the difference between them is marginal in my experience.
The only play for OpenAI et al in my opinion is to try to pull up the draw bridge behind them by getting legislation passed which makes compliance prohibitively difficult if that's not your core business.
Re: Why AI Infrastructure Startups Are Insanely Hard to Build
#123I'm not sure I'm exactly at the edge of things, but I have 2 companies trying to setup regular meetings with me to be a beta customer. Both have promised I can help define a new product, but when I list my real problems... they aren't in the mission. Everyone wants to solve RAG (that's easy, don't need help) or they want to give me a gui I don't need, or wrap open source software like vllm. Or "solve privacy" (which…
> Want to solve a real problem, help me create custom benchmarks, clean my data, get my small parameter model to reason better etc. I recently started a company with a friend if mine to do exactly this. Ive worked at a few AI startups over the last 8 years, and the problem everyone tackles independently (and poorly) is the long tail of dealing with input data that isn't great. You build a demo with sample data that w…
Ah. Cleaning data. The big trick going back to data warehousing.
Re: Why AI Infrastructure Startups Are Insanely Hard to Build
#124In a nutshell: If there's a goldrush, you get rich by selling shovels. ... unless there are already 200 shovel shops next to each other...
at what point does a goldrush become a shovelrush?
Re: Why AI Infrastructure Startups Are Insanely Hard to Build
#125Earlier quoted context omitted.
> VC pouring money in LLM infra is legitimately crazy to me. VC business model is throwing money at the wall and seeing what sticks. They love congratulating themselves on how smart they are but at the end of the day their overall returns trail S&P 500. They are salespeople and their job is to sell themselves to private capital on how smart and connected they are.
Does their return trail s&p 500? I guess there are some of them that are successful and some that aren't, but to be honest I've only heard about vc as people who on average have a very very high return. Now, it is very possible that's just survivor bias, but I would need to search for some data
The S&P 500 is a list of the largest and most profitable public companies. It's hard to do better than the most successful businesses. Most other indices and hedge funds don't outperform the S&P 500. Most private equity shops don't. Most real estate investors don't. So it shouldn't come as a surprise that venture capital doesn't.
Most startups don't get big. How many startups founded in the past decade have become hugely profitable? It's not that many. A handful out of the 500k or so funded startups. Meanwhile the S&P keeps chugging along at 8% annually.
Re: Why AI Infrastructure Startups Are Insanely Hard to Build
#126Great article. I am not going to name names, but over the last one year, whenever there is a concept that became popular in Gen AI, thousands of startups pivoted to doing that. Many come from software background where the expectation was that if the code works on one dataset, it would work for everything. You can see this with 1/ Prompt engineering 2/ RAG 3/ and now, after Apple's WWDC, it's adapters. Enterprises I h…
Random question, but what are Apple adapters? Kind of hard to google it, lol
Not Apple adapters per-se, but LoRA adapters. It’s a way of fine tuning a model such that you keep the base weights unchanged but then keep a smaller set of tuned weights to help you on specific tasks.
(Edit) Apple is using them in their Apple Intelligence, hence the association. But the technique was around before.
Re: Why AI Infrastructure Startups Are Insanely Hard to Build
#127Selling shovels is a way to make money, but definitely not at venture scale.
Re: Why AI Infrastructure Startups Are Insanely Hard to Build
#128Great article. I am not going to name names, but over the last one year, whenever there is a concept that became popular in Gen AI, thousands of startups pivoted to doing that. Many come from software background where the expectation was that if the code works on one dataset, it would work for everything. You can see this with 1/ Prompt engineering 2/ RAG 3/ and now, after Apple's WWDC, it's adapters. Enterprises I h…
Random question, but what are Apple adapters? Kind of hard to google it, lol
I know almost nothing about this stuff, but what I know about Apple adapters I learned from this page:
https://machinelearning.apple.com/research/introducing-apple...
> Our foundation models are fine-tuned for users’ everyday activities, and can dynamically specialize themselves on-the-fly for the task at hand. We utilize adapters, small neural network modules that can be plugged into various layers of the pre-trained model, to fine-tune our models for specific tasks. For our models we adapt the attention matrices, the attention projection matrix, and the fully connected layers in the point-wise feedforward networks for a suitable set of the decoding layers of the transformer architecture.
Re: Why AI Infrastructure Startups Are Insanely Hard to Build
#129It's as if all of the AI devex/infra companies are cargo culting the story of how the people that made the most money in the gold rush were the people who sold the tools. The thing is that the tools were well understood and battle tested.
GenAI applications are so finnicky that it’s easier to build a company around tools-for-AI than a company fitting their archtypical user profile doing AI applications (that’s actually profitable. Most of their customers likely aren’t anywhere close). That’s inverted from the prior SAAS/cloud boom.
I too think there are too many shovel chasers but I think it’s also a consequence of what’s easier to ship.
Re: Why AI Infrastructure Startups Are Insanely Hard to Build
#130Infra has always been a tarpit idea. Google didn't start out as an "infra" company, and neither did Amazon or Facebook. In fact, the few "infra companies" that did start back then (companies like Godaddy) are minuscule compared to the aforementioned. VC pouring money in LLM infra is legitimately crazy to me. It's clear as day that there will be winners of this AI cycle, but, as always, they will be companies that pro…
> Even OpenAI doesn't really have a product. They are making a ton of money off subscriptions.
I'm skeptical that $20 / month is enough to run OpenAI and be profitable. I would bet the real number is an order of magnitude higher.