Great article. I am not going to name names, but over the last one year, whenever there is a concept that became popular in Gen AI, thousands of startups pivoted to doing that. Many come from software background where the expectation was that if the code works on one dataset, it would work for everything. You can see this with 1/ Prompt engineering 2/ RAG 3/ and now, after Apple's WWDC, it's adapters.
Enterprises I have spoken to says they are getting pitched by 20 startups offering similar things on a weekly basis. They are confused on what to go with.
From my vantage point (and may be wrong), the problem is many startups ended up doing the easy things - things which could be done by an internal team too, and while it's a good starting point for many businesses, but hard to justify costs in the long term. At this point, two clear demarcations appear:
1/ You make an API call to OpenAI, Anthropic, Google, Together etc. where your contribution is the prompt/RAG support etc.
2/ You deploy a model on prem/private VPC where you make the same calls w RAG etc. (focused on data security and privacy)
First one is very cheap, and you end up competing with Open AI and hundred different startups offering it. Plus internal teams w confidence that they can do it themselves. Second one is interesting, but overhead costs are about $10,000 (for hosting) and any customer would expect more value than what a typical RAG provides. Difficult to provide that kind of value when you do not have a deep understanding and under pressure to generate revenue.
I don't fully believe infra startups are a tarpit idea. Just that, we havent explored the layers where we can truly find a valuable thing that is hard to build for internal teams.