Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why AI Infrastructure Startups Are Insanely Hard to Build

nextword.substack.com

Post reply on HN
Print view View original

141–150 of 190 posts

Re: Why AI Infrastructure Startups Are Insanely Hard to Build

#141

Post by tcgv »

dvt wrote: post #18

Infra has always been a tarpit idea. Google didn't start out as an "infra" company, and neither did Amazon or Facebook. In fact, the few "infra companies" that did start back then (companies like Godaddy) are minuscule compared to the aforementioned. VC pouring money in LLM infra is legitimately crazy to me. It's clear as day that there will be winners of this AI cycle, but, as always, they will be companies that pro…

> Even OpenAI doesn't really have a product. Just throwing data at a bunch of video cards isn't value-generating in itself. We need (...) something people love that makes their life easier or better.

I agree. I'm of the hypothesis that, when it comes to AI, a lot of product teams are pursuing overly ambitious and sophisticated features instead of targeting easy wins that are in plain sight [1].

[1] https://thomasvilhena.com/2024/06/easy-wins-for-generative-a...

Re: Why AI Infrastructure Startups Are Insanely Hard to Build

#142

Post by ankit219 »

hobs wrote: post #111
ankit219 wrote: post #67

Great article. I am not going to name names, but over the last one year, whenever there is a concept that became popular in Gen AI, thousands of startups pivoted to doing that. Many come from software background where the expectation was that if the code works on one dataset, it would work for everything. You can see this with 1/ Prompt engineering 2/ RAG 3/ and now, after Apple's WWDC, it's adapters. Enterprises I h…

Pretty much this, 18 months ago my CEO told me we HAD to get into this space, and I told him that basically our money came from our private product and that the only way our big enterprise customers were going to play game with us was either ironclad agreements that went all the way to openai, or more likely a completely single tenant system, which would cost far more than they were willing to pay. Of course they wen…

Can you go into details (as much as you are comfortable) on what happened with the single tenant system? I have seen a few things, but I find it hard to put a finger on what went wrong except the ROI wasnt there. Would love to understand your experience.

Re: Why AI Infrastructure Startups Are Insanely Hard to Build

#143

Post by TimPC »

williamcotton wrote: post #104
meiraleal wrote: post #102

Earlier quoted context omitted.

> In my humble opinion, a chat interface (API or not) does not a product make. Well, you are humbly wrong then. > You would just eventually train and deploy your own model because an LLM is not a product. Hallucinations aren't exclusive to LLMs it seems.

We’re already seeing a lot of competition between LLMs. They are quickly becoming commodities. Margins will approach zero and the real value proposition will be with consumer products that extend beyond an .

I disagree they will become commodities because most of my use cases are more sensitive to accuracy than cost. We typically have usage volume that isn't absurd and for large enterprise customers our LLM budget is a rounding error. Meanwhile our product saves them many hours of a data engineer. If we can pay double to get a 10% performance boost we will do so gladly. You can already see this in LLM pricing where they have cheap models that deliver low performance a My bet is that 80% of profits will be made on the workloads that are sensitive to accuracy and workloads where running an LLM at all gets you most of the benefit will become very commoditized.

Re: Why AI Infrastructure Startups Are Insanely Hard to Build

#144

Post by notamy »

pierre wrote: post #3

Good article, but what is the alternative? What can you build today as a software engineer that can have impact? Nothing seems to come close to AI / AI infra, even of its hard / risky / a moving landscape.

> What can you build today as a software engineer that can have impact?

Quite a bit, if you don’t follow the standard tech hype. Find an industry that isn’t tech-first and you’ll notice that there’s a lot of room for improvement.

Re: Why AI Infrastructure Startups Are Insanely Hard to Build

#145

Post by kirubakaran »

xg15 wrote: post #23

In a nutshell: If there's a goldrush, you get rich by selling shovels. ... unless there are already 200 shovel shops next to each other...

Yes, Brannan cornered the market before he sold the shovels. Selling shovels is not that profitable if you skip that "Step 1".

> he owned the only store between San Francisco and the gold fields — a fact he capitalized on by buying up all the picks, shovels and pans he could find, and then running up and down the streets of San Francisco, shouting 'Gold! Gold on the American River!' He paid 20 cents each for the pans, then sold them for $15 a piece. In nine weeks, he made $36,000."

https://en.wikipedia.org/wiki/Samuel_Brannan

Re: Why AI Infrastructure Startups Are Insanely Hard to Build

#146

Post by ToucanLoucan »

rapsey wrote: post #71
ozim wrote: post #66

Earlier quoted context omitted.

I think they don't even trail S&P 500 - with all the dirty tricks up their sleeves they roll losses over and over. Bottom will fall off at some point when hype train stops rolling, but so far it was good years as we have hype after hype. I am waiting what will be next one after AI, because quantum computing feels like too hard to become a hype, the same with space ventures, there is some upward trend going on there b…

AI is a long term trend, next is more products on top of AI. Just like the internet. Biotech and space will both be trends but slower and less bubbly because the cost to play is high, though the returns are possibly huge in both.

> AI is a long term trend, next is more products on top of AI. Just like the internet.

This is legitimately just the same damn hype train the tech sector is constantly attempting to create. Now AI is the next internet. Before that it was the metaverse. Before that it was NFTs. Before that it was cryptocurrency. Before that it was quantum. Before that it was VR. Before that it was AR.

None of those were the revolutions postured by techno-fetishistic CEOs. Most stick around in some capacity, like VR and AR, and the argument can be made that those have a future. Blockchains certainly have a future as it's a highly useful technology, even if the financial vehicle made by it is utterly useless. The metaverse is Dead on Arrival because nobody ever wanted it in the first damn place, apart from the speculators betting money on it.

Re: Why AI Infrastructure Startups Are Insanely Hard to Build

#147

Post by diwank »

While I agree that AI infra startups are hard to build, I strongly disagree with the idea that they are harder than foundational or application layer startups. I think it boils down to what you know and what resources you can muster.

For instance, foundational AI startups are also ridiculously hard to build. You need an insane amount of funding, spend it pretraining models to stay competitive only to find that gains in hardware and model architecture make them obsolete within months plus there's no real guarantee that scaling will keep working.

Application layer startups are hard in a very different way, there's an insane amount of competition and new capabilities are emerging every few weeks. I have worked with a few AI girlfriend startups and they are really struggling with keeping apace and warding off ridiculous amount of competition.

I think it's really just YMMV. Of course, the deeper you get into the stack, the more monopolizing pressure there is. Is it hard to build AI infra startups? Yes 100%. Will there be very few winners? Yes. Is it harder than foundational or application layer startups? Depends on the founders' strengths. Is it Is it a lost cause? I really don't think so.

Re: Why AI Infrastructure Startups Are Insanely Hard to Build

#149

Post by Art9681 »

mlsu wrote: post #10
pierre wrote: post #3

Good article, but what is the alternative? What can you build today as a software engineer that can have impact? Nothing seems to come close to AI / AI infra, even of its hard / risky / a moving landscape.

I would almost invert that statement. Sorry if this comes off ranty, but what exactly are people doing in the "AI space" currently that isn't "undifferentiated spam/chatbot" being sold to non-techies who heard about AI on NPR? What are real people using "AI" for that is so insanely valuable today? How much "company Y: same product with a chat window, sparks emoji" do we all need before this thing levels out and we al…

What do you consider "AI"? Because machine learning models have been deployed in enterprise systems for years. Video processing, security, data labeling, sentiment analysis. The sexiest one I can think of in recent memory is nVidia DLSS.

Re: Why AI Infrastructure Startups Are Insanely Hard to Build

#150

Post by manishsharan »

dvt wrote: post #33
siquick wrote: post #27

Earlier quoted context omitted.

> We need a Dropbox or a Slack or an Instagram: something people love that makes their life easier or better. People seem to not mind ChatGPT or Claude and safe to say that a very large majority of AI products are using one of the APIs of those companies.

In my humble opinion, a chat interface (API or not) does not a product make. Not to mention that Llama is free and competitive with both (same with Mistral, heck the 7B model works great on my RTX 3080). If you started a company that blew up because you made a badass product (and let's say you used ChatGPT under the hood), you would just eventually train and deploy your own model because an LLM is not a product .

>> LLM is not a product

Would you consider Database to be a product ?

SQLLite , PostGres etc. are free and yet we have Oracle , Mongodb and MS SQL doing billions in revenues.

Post reply on HN