Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why AI Infrastructure Startups Are Insanely Hard to Build

nextword.substack.com

Post reply on HN
Print view View original

41–50 of 190 posts

Re: Why AI Infrastructure Startups Are Insanely Hard to Build

#41

Post by physicsguy »

mlsu wrote: post #38
siquick wrote: post #27

Earlier quoted context omitted.

> We need a Dropbox or a Slack or an Instagram: something people love that makes their life easier or better. People seem to not mind ChatGPT or Claude and safe to say that a very large majority of AI products are using one of the APIs of those companies.

there is a lot of consumer surplus, but it’s very unclear if any one of these companies will be able to capture it. Especially if mostly good enough commodity LLMs like Llama 70b are in the mix. Where is the differentiation?

Not to mention that fine tuned smaller parameter models are much cheaper to run. See: Google embedding a model into the latest version of Chrome, accessible through dev tools.

Re: Why AI Infrastructure Startups Are Insanely Hard to Build

#42

Post by benrutter »

swalsh wrote: post #31

I'm not sure I'm exactly at the edge of things, but I have 2 companies trying to setup regular meetings with me to be a beta customer. Both have promised I can help define a new product, but when I list my real problems... they aren't in the mission. Everyone wants to solve RAG (that's easy, don't need help) or they want to give me a gui I don't need, or wrap open source software like vllm. Or "solve privacy" (which…

This rings so true! I think it's natural whenever there's a new technology that a lot of start ups spring up with a vibe of "GenAI is cool, let's do something with that!", which is 100% the wrong way to go about building something.

Starting by investing yourself fully into a given problem, and fixing it with the most appropriate tool (might be GenAI, might not) is much more likely to end in something people actually want or need.

Doing the reverse, and trying to find an existing problem that matches a solution you've already picked is how you end up with hundreds of companies selling thin API wrappers for ChatGPT.

Re: Why AI Infrastructure Startups Are Insanely Hard to Build

#43

Post by morgante »

threeseed wrote: post #7

Way too many founders don't understand the impact of competing with cloud vendors. Almost all enterprises have pre-committed budgets for cloud which means unless your product is FOSS it's going to be hard to convince someone to bet their business on it. Especially given that in this fundraising environment there is a 95% chance they won't be around in a year or two anyway. It's going to be a brutal few years especial…

> Almost all enterprises have pre-committed budgets for cloud which means unless your product is FOSS it's going to be hard to convince someone to bet their business on it.

This isn't a death knell.

1. If you get into the marketplace, enterprises can spend their commit against you.

2. A few million in ARR is ~nothing to a hyperscale cloud, but meaningful to most startups. If you find the right positioning, you can get their sales team selling your solution on many deals.

Re: Why AI Infrastructure Startups Are Insanely Hard to Build

#44

Post by siva7 »

It bugs me that all we are seeing in the vc-backed startup scene seems to be ai infrastructure startups. We got something close to ai and all people come up with is they want to be the next ai marketplace store or the millionth infrastructure startup that does exactly the same like their competitors. How boring.

Re: Why AI Infrastructure Startups Are Insanely Hard to Build

#45

Post by swalsh »

mlsu wrote: post #10
pierre wrote: post #3

Good article, but what is the alternative? What can you build today as a software engineer that can have impact? Nothing seems to come close to AI / AI infra, even of its hard / risky / a moving landscape.

I would almost invert that statement. Sorry if this comes off ranty, but what exactly are people doing in the "AI space" currently that isn't "undifferentiated spam/chatbot" being sold to non-techies who heard about AI on NPR? What are real people using "AI" for that is so insanely valuable today? How much "company Y: same product with a chat window, sparks emoji" do we all need before this thing levels out and we al…

Going to be vague, but I'm using it to scale out human processes in ways I couldn't using humans (because they cost too much) or regular code (because it's unstructured). Early results are promising, we've found a bunch of stuff which has been buried... and is potentially worth millions. Not a chat wrapper, just breathing new light into our regular old business.

Re: Why AI Infrastructure Startups Are Insanely Hard to Build

#46

Post by swalsh »

jononor wrote: post #26
pierre wrote: post #3

Good article, but what is the alternative? What can you build today as a software engineer that can have impact? Nothing seems to come close to AI / AI infra, even of its hard / risky / a moving landscape.

Anything SaaS that solves a painpoints for established industries. Those that have billions of turnaround for decades already, are not good at building tech themselves, and buy solutions/services to run their business. Bonus for low barriers to entry. Agriculture, logistics, real estate, energy, etc.

I have a theory that the days of established businesses that don't know tech is dwindling. A lot of companies which has adopted tech has started building a small foundation of talent internally. I think you're seeing this trend accelerate with the large tech companies laying people off. I have heard about top grade data science talent landing at some small sized health plan.

My companies fastest growing competitor is "internally sourced departments" of the services we provide.

Re: Why AI Infrastructure Startups Are Insanely Hard to Build

#47

Post by teaearlgraycold »

siva7 wrote: post #44

It bugs me that all we are seeing in the vc-backed startup scene seems to be ai infrastructure startups. We got something close to ai and all people come up with is they want to be the next ai marketplace store or the millionth infrastructure startup that does exactly the same like their competitors. How boring.

Some of us are working on synthetic data

Re: Why AI Infrastructure Startups Are Insanely Hard to Build

#48

Post by tinco »

swalsh wrote: post #31

I'm not sure I'm exactly at the edge of things, but I have 2 companies trying to setup regular meetings with me to be a beta customer. Both have promised I can help define a new product, but when I list my real problems... they aren't in the mission. Everyone wants to solve RAG (that's easy, don't need help) or they want to give me a gui I don't need, or wrap open source software like vllm. Or "solve privacy" (which…

We had the exact same problem before genAI became the next big thing. All the startups were selling generic fine tuning and labeling services both of which are super easy to build, and they didn't even work on our unique super high quality super high resolution 40TB dataset.

Our problem was we had a real world problem and real data. All the startups were solving for imaginary problems and had no data.

Re: Why AI Infrastructure Startups Are Insanely Hard to Build

#49

Post by nextworddev »

dvt wrote: post #18

Infra has always been a tarpit idea. Google didn't start out as an "infra" company, and neither did Amazon or Facebook. In fact, the few "infra companies" that did start back then (companies like Godaddy) are minuscule compared to the aforementioned. VC pouring money in LLM infra is legitimately crazy to me. It's clear as day that there will be winners of this AI cycle, but, as always, they will be companies that pro…

Author here. I think the tarpit extends to most chatgpt wrappers as well, which is why I called out pivoting prematurely to application layer is a futile exercise.

Re: Why AI Infrastructure Startups Are Insanely Hard to Build

#50

Post by statusfailed »

swalsh wrote: post #31

I'm not sure I'm exactly at the edge of things, but I have 2 companies trying to setup regular meetings with me to be a beta customer. Both have promised I can help define a new product, but when I list my real problems... they aren't in the mission. Everyone wants to solve RAG (that's easy, don't need help) or they want to give me a gui I don't need, or wrap open source software like vllm. Or "solve privacy" (which…

I'd love to know what your use case is that makes those things important to you - and what kind of benchmarks and cleaning tasks do you need to run?

Also, what kind of evaluations for quality of reasoning do you use?

Post reply on HN