Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why AI Infrastructure Startups Are Insanely Hard to Build

nextword.substack.com

Post reply on HN
Print view View original

91–100 of 190 posts

Re: Why AI Infrastructure Startups Are Insanely Hard to Build

#91

Post by oefrha »

belter wrote: post #82
dvt wrote: post #18

Infra has always been a tarpit idea. Google didn't start out as an "infra" company, and neither did Amazon or Facebook. In fact, the few "infra companies" that did start back then (companies like Godaddy) are minuscule compared to the aforementioned. VC pouring money in LLM infra is legitimately crazy to me. It's clear as day that there will be winners of this AI cycle, but, as always, they will be companies that pro…

> We need a Dropbox or a Slack or an Instagram: something people love that makes their life easier or better. None of those three make my life better or easier: Dropbox -> We drop your data directly on S3 but give you worst security... Slack -> Being able to navigate our messy interface is an IQ test in itself... Instagram -> Only makes your life better if you have bikini posts to share....

Didn’t expect curlftpfs+svn to strike again in 2024.

Re: Why AI Infrastructure Startups Are Insanely Hard to Build

#92

Post by jokethrowaway »

dvt wrote: post #33
siquick wrote: post #27

Earlier quoted context omitted.

> We need a Dropbox or a Slack or an Instagram: something people love that makes their life easier or better. People seem to not mind ChatGPT or Claude and safe to say that a very large majority of AI products are using one of the APIs of those companies.

In my humble opinion, a chat interface (API or not) does not a product make. Not to mention that Llama is free and competitive with both (same with Mistral, heck the 7B model works great on my RTX 3080). If you started a company that blew up because you made a badass product (and let's say you used ChatGPT under the hood), you would just eventually train and deploy your own model because an LLM is not a product .

Llama and Mistral are not really competitive but they can be used as a base to finetune for a very specific usecase.

Then they don't suck as much.

With OpenAI and Claude, you throw some text instructions and you get back the answers which are surprisingly correct (minus a few exceptions). In order to replicate that with Llama you'd probably need N-hundreds finetunes and a model to decide which finetunes to use.

Re: Why AI Infrastructure Startups Are Insanely Hard to Build

#93

Post by _el1s7 »

pierre wrote: post #3

Good article, but what is the alternative? What can you build today as a software engineer that can have impact? Nothing seems to come close to AI / AI infra, even of its hard / risky / a moving landscape.

If you don't know what to build, you don't build.

Re: Why AI Infrastructure Startups Are Insanely Hard to Build

#94

Post by belter »

MOARDONGZPLZ wrote: post #89
belter wrote: post #82

Earlier quoted context omitted.

> We need a Dropbox or a Slack or an Instagram: something people love that makes their life easier or better. None of those three make my life better or easier: Dropbox -> We drop your data directly on S3 but give you worst security... Slack -> Being able to navigate our messy interface is an IQ test in itself... Instagram -> Only makes your life better if you have bikini posts to share....

All three of these products have made my life substantially better.

Next you are going to recommend me Windows and Teams? :-)

Re: Why AI Infrastructure Startups Are Insanely Hard to Build

#95

Post by rapsey »

amarcheschi wrote: post #90
rapsey wrote: post #32

Earlier quoted context omitted.

> VC pouring money in LLM infra is legitimately crazy to me. VC business model is throwing money at the wall and seeing what sticks. They love congratulating themselves on how smart they are but at the end of the day their overall returns trail S&P 500. They are salespeople and their job is to sell themselves to private capital on how smart and connected they are.

Does their return trail s&p 500? I guess there are some of them that are successful and some that aren't, but to be honest I've only heard about vc as people who on average have a very very high return. Now, it is very possible that's just survivor bias, but I would need to search for some data

I know a while ago there was some leaked a16z data showing returns underperforming the S&P 500.

Re: Why AI Infrastructure Startups Are Insanely Hard to Build

#96

Post by __MatrixMan__ »

tryauuum wrote: post #69
dvt wrote: post #18

Infra has always been a tarpit idea. Google didn't start out as an "infra" company, and neither did Amazon or Facebook. In fact, the few "infra companies" that did start back then (companies like Godaddy) are minuscule compared to the aforementioned. VC pouring money in LLM infra is legitimately crazy to me. It's clear as day that there will be winners of this AI cycle, but, as always, they will be companies that pro…

are there even any VCs pouring money in LLM infra? I would assume VCs aren't interested in projects which won't give them a tenfold return. And with infrastructure there are some many existing competitors (like AWS) so that such returns are never expected

I get the feeling that there are. MLOps became LLMOps before most people knew it as any more than a buzzword and well we can't just call it ops because there's clearly something different here.

AI is still too opaque to reliably know beforehand if an idea will pan out, so you just gotta try it.

Plus it's easy, once you start imagining how the magic of AI is gonna make you rich, to ignore the problems with your idea and assume that the AI will handle them too.

So you've got all these hyped up fools trying to make stuff that lacks merit. How do you capitalize on that? You don't invest in the stuff that's doomed to fail, you create a slot:

> Insert coin, insert half baked idea, receive AI app

That way you get to keep the coins even when apps don't turn out. Plus, you're collecting the institutional knowledge necessary to pounce on something that comes along which is actually worth investing in.

Or at least that's the vibe I get when our meetings feature an AI-excited VC (which isn't common, but it happens).

Re: Why AI Infrastructure Startups Are Insanely Hard to Build

#97

Post by jgalt212 »

amarcheschi wrote: post #90
rapsey wrote: post #32

Earlier quoted context omitted.

> VC pouring money in LLM infra is legitimately crazy to me. VC business model is throwing money at the wall and seeing what sticks. They love congratulating themselves on how smart they are but at the end of the day their overall returns trail S&P 500. They are salespeople and their job is to sell themselves to private capital on how smart and connected they are.

Does their return trail s&p 500? I guess there are some of them that are successful and some that aren't, but to be honest I've only heard about vc as people who on average have a very very high return. Now, it is very possible that's just survivor bias, but I would need to search for some data

Right, just as some mutual funds outperform the SP and most don't.

Re: Why AI Infrastructure Startups Are Insanely Hard to Build

#98

Post by [deleted] »

hiAndrewQuinn wrote: post #81
IanCal wrote: post #64

Earlier quoted context omitted.

n=3 now so it's full on anecdata, I've got a $20 sub professionally (swe). It has to save me so little time to be worth it it's easily great. Might add claude, though at this point probably better to find a nice interface and use the APIs. These things are products.

n=5, my wife and I split a subscription between the two of us. While our needs are usually pretty minor, we both work in the computer science space and GPT-4o's ability to e.g. generate good example sentences for my Finnish vocabulary learning is astonishingly good. I'm building a wrapper around it so I can generate them en masse at [1], but it's still quite early days and very obviously not software I intend to sell…

[deleted]

Re: Why AI Infrastructure Startups Are Insanely Hard to Build

#99

Post by williamcotton »

dvt wrote: post #18

Infra has always been a tarpit idea. Google didn't start out as an "infra" company, and neither did Amazon or Facebook. In fact, the few "infra companies" that did start back then (companies like Godaddy) are minuscule compared to the aforementioned. VC pouring money in LLM infra is legitimately crazy to me. It's clear as day that there will be winners of this AI cycle, but, as always, they will be companies that pro…

Even OpenAI doesn't really have a product.

This becomes very obvious when you use Claude Projects with Artifacts! ChatGPT depends heavily on one’s ability to copy and paste… and even though it is an improvement, Claude Projects still make managing a set of documents tedious compared to your standard code editor.

Third-party tools like Cursor are an improvement but will be prohibitively expensive compared to companies that create and manage their own LLMs.

I expect to see a native document editing/code editing software system directly from one of the LLMs-as-a-service companies at some point.

Re: Why AI Infrastructure Startups Are Insanely Hard to Build

#100

Post by BobbyJo »

swalsh wrote: post #31

I'm not sure I'm exactly at the edge of things, but I have 2 companies trying to setup regular meetings with me to be a beta customer. Both have promised I can help define a new product, but when I list my real problems... they aren't in the mission. Everyone wants to solve RAG (that's easy, don't need help) or they want to give me a gui I don't need, or wrap open source software like vllm. Or "solve privacy" (which…

> Want to solve a real problem, help me create custom benchmarks, clean my data, get my small parameter model to reason better etc.

I recently started a company with a friend if mine to do exactly this.

Ive worked at a few AI startups over the last 8 years, and the problem everyone tackles independently (and poorly) is the long tail of dealing with input data that isn't great. You build a demo with sample data that works well, then you move on to real world uses and the data is suddenly... blehg.

Post reply on HN