Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why AI Infrastructure Startups Are Insanely Hard to Build

nextword.substack.com

Post reply on HN
Print view View original

51–60 of 190 posts

Re: Why AI Infrastructure Startups Are Insanely Hard to Build

#51

Post by nextworddev »

rapsey wrote: post #24
dvt wrote: post #18

Infra has always been a tarpit idea. Google didn't start out as an "infra" company, and neither did Amazon or Facebook. In fact, the few "infra companies" that did start back then (companies like Godaddy) are minuscule compared to the aforementioned. VC pouring money in LLM infra is legitimately crazy to me. It's clear as day that there will be winners of this AI cycle, but, as always, they will be companies that pro…

> Even OpenAI doesn't really have a product. They are making a ton of money off subscriptions.

Yes. Author here. OpenAI is at near 4bn ARR and growing (api plus SaaS)

Re: Why AI Infrastructure Startups Are Insanely Hard to Build

#52

Post by nextworddev »

morgante wrote: post #43
threeseed wrote: post #7

Way too many founders don't understand the impact of competing with cloud vendors. Almost all enterprises have pre-committed budgets for cloud which means unless your product is FOSS it's going to be hard to convince someone to bet their business on it. Especially given that in this fundraising environment there is a 95% chance they won't be around in a year or two anyway. It's going to be a brutal few years especial…

> Almost all enterprises have pre-committed budgets for cloud which means unless your product is FOSS it's going to be hard to convince someone to bet their business on it. This isn't a death knell. 1. If you get into the marketplace, enterprises can spend their commit against you. 2. A few million in ARR is ~nothing to a hyperscale cloud, but meaningful to most startups. If you find the right positioning, you can ge…

Author here. Yes, that said getting into the ISV / APN partner program can be a major pain in the butt and take quite long (from a startup’s perspective where time is precious)

Re: Why AI Infrastructure Startups Are Insanely Hard to Build

#53

Post by Simon_ORourke »

I've come across a number of these AI infra start-ups like Scale AI and Zerve and TBH I'm amazed they can do what they do with relatively small teams when you have Meta and Apple somewhat struggling in this area and buying rather than building themselves.

Re: Why AI Infrastructure Startups Are Insanely Hard to Build

#54

Post by curious_cat_163 »

Articles like this represent reminder that we are in the middle of a hype cycle. [1]

I don’t say that thinking that LLMs (really: Transformers and the corresponding scaling of compute around it) don’t represent a step change.

I say that because I am very sure that we are going to see a slope of enlightenment that results in products that improve the quality of human life.

[1] https://en.m.wikipedia.org/wiki/Gartner_hype_cycle

Re: Why AI Infrastructure Startups Are Insanely Hard to Build

#55

Post by nextworddev »

curious_cat_163 wrote: post #54

Articles like this represent reminder that we are in the middle of a hype cycle. [1] I don’t say that thinking that LLMs (really: Transformers and the corresponding scaling of compute around it) don’t represent a step change. I say that because I am very sure that we are going to see a slope of enlightenment that results in products that improve the quality of human life. [1] https://en.m.wikipedia.org/wiki/Gartner_h…

Author here - the article isnt questioning the value of AI per se, but value capture and competitive dynamics. Internet routers remained valuable but got commoditized, sort of a thing.

Re: Why AI Infrastructure Startups Are Insanely Hard to Build

#56

Post by nextworddev »

siva7 wrote: post #44

It bugs me that all we are seeing in the vc-backed startup scene seems to be ai infrastructure startups. We got something close to ai and all people come up with is they want to be the next ai marketplace store or the millionth infrastructure startup that does exactly the same like their competitors. How boring.

Author here. Wait until you see the ChatGPT for Law landscape! /s

Re: Why AI Infrastructure Startups Are Insanely Hard to Build

#57

Post by siquick »

dvt wrote: post #33
siquick wrote: post #27

Earlier quoted context omitted.

> We need a Dropbox or a Slack or an Instagram: something people love that makes their life easier or better. People seem to not mind ChatGPT or Claude and safe to say that a very large majority of AI products are using one of the APIs of those companies.

In my humble opinion, a chat interface (API or not) does not a product make. Not to mention that Llama is free and competitive with both (same with Mistral, heck the 7B model works great on my RTX 3080). If you started a company that blew up because you made a badass product (and let's say you used ChatGPT under the hood), you would just eventually train and deploy your own model because an LLM is not a product .

Uploading a a photo/file to a server does not a product make yet you referenced Dropbox and Instagram and that’s what they started out as.

The UX of the AI applications is the moat and the infrastructure providers behind those applications is pretty much always OpenAI and Anthropic at the moment because running your own open source LLMs (which are inferior out of the box) at scale is not cheap or easy to do it right - same reason most companies use cloud. Agree that once you hit super scale then you can run your own infrastructure but there are thousand of companies who won’t get that far and still need an LLM.

Re: Why AI Infrastructure Startups Are Insanely Hard to Build

#58

Post by cageface »

physicsguy wrote: post #39

If I'm an application developer or manager, at any >100 person company, it normally doesn't fall into my remit to go out and pick a new company to contract with to provide services. Typically, it gets harder and harder to do that. Even with LLM stuff, we're contracting that through our existing relationship with Microsoft. When evaluating infra options, it therefore is a huge barrier to entry for most developers if t…

I've experienced this too. Any new service that requires more than a credit card number has to go through the legal department to review the contract and that means it goes to the bottom of the pile of similar requests and probably won't even get a look for months.

Re: Why AI Infrastructure Startups Are Insanely Hard to Build

#59

Post by fragmede »

dvt wrote: post #33
siquick wrote: post #27

Earlier quoted context omitted.

> We need a Dropbox or a Slack or an Instagram: something people love that makes their life easier or better. People seem to not mind ChatGPT or Claude and safe to say that a very large majority of AI products are using one of the APIs of those companies.

In my humble opinion, a chat interface (API or not) does not a product make. Not to mention that Llama is free and competitive with both (same with Mistral, heck the 7B model works great on my RTX 3080). If you started a company that blew up because you made a badass product (and let's say you used ChatGPT under the hood), you would just eventually train and deploy your own model because an LLM is not a product .

That's one opinion. the number off people who get value out of ChatGPT is known only to them, but anecdotally it's pretty high. a lot of people I know are actively using it on a daily basis and paying them $20/month.

Re: Why AI Infrastructure Startups Are Insanely Hard to Build

#60

Post by mike_hearn »

dvt wrote: post #33
siquick wrote: post #27

Earlier quoted context omitted.

> We need a Dropbox or a Slack or an Instagram: something people love that makes their life easier or better. People seem to not mind ChatGPT or Claude and safe to say that a very large majority of AI products are using one of the APIs of those companies.

In my humble opinion, a chat interface (API or not) does not a product make. Not to mention that Llama is free and competitive with both (same with Mistral, heck the 7B model works great on my RTX 3080). If you started a company that blew up because you made a badass product (and let's say you used ChatGPT under the hood), you would just eventually train and deploy your own model because an LLM is not a product .

LLMs are clearly products! The fact that Meta rather inexplicably chooses to give away assets that cost millions or billions to create doesn't mean LLMs aren't a product, it just means that they're competing against a company funding open source stuff for (presumably) strategic reasons, like in many other markets. And like in many other markets over time it's possible the proprietary versions will establish permanent competitive advantage. Meta may not keep releasing Llamas forever.

> you would just eventually train and deploy your own model

Just train an LLM? It's really not that easy! Even if it was, it'd be like how people can "just" run their own email service. Hosted email API is not rocket science but in practice companies all choose to pay Microsoft or Google to do it. Doing these things isn't a core competitive advantage so it gets outsourced.

Post reply on HN