Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why AI Infrastructure Startups Are Insanely Hard to Build

nextword.substack.com

Post reply on HN
Print view View original

31–40 of 190 posts

Re: Why AI Infrastructure Startups Are Insanely Hard to Build

#31

Post by swalsh »

I'm not sure I'm exactly at the edge of things, but I have 2 companies trying to setup regular meetings with me to be a beta customer. Both have promised I can help define a new product, but when I list my real problems... they aren't in the mission. Everyone wants to solve RAG (that's easy, don't need help) or they want to give me a gui I don't need, or wrap open source software like vllm. Or "solve privacy" (which usually comes in the form of masking... which surprise, that works for PII, but not PHI... I need the protected information).

Want to solve a real problem, help me create custom benchmarks, clean my data, get my small parameter model to reason better etc.

Re: Why AI Infrastructure Startups Are Insanely Hard to Build

#32

Post by rapsey »

dvt wrote: post #18

Infra has always been a tarpit idea. Google didn't start out as an "infra" company, and neither did Amazon or Facebook. In fact, the few "infra companies" that did start back then (companies like Godaddy) are minuscule compared to the aforementioned. VC pouring money in LLM infra is legitimately crazy to me. It's clear as day that there will be winners of this AI cycle, but, as always, they will be companies that pro…

> VC pouring money in LLM infra is legitimately crazy to me.

VC business model is throwing money at the wall and seeing what sticks. They love congratulating themselves on how smart they are but at the end of the day their overall returns trail S&P 500. They are salespeople and their job is to sell themselves to private capital on how smart and connected they are.

Re: Why AI Infrastructure Startups Are Insanely Hard to Build

#33

Post by dvt »

siquick wrote: post #27
dvt wrote: post #18

Infra has always been a tarpit idea. Google didn't start out as an "infra" company, and neither did Amazon or Facebook. In fact, the few "infra companies" that did start back then (companies like Godaddy) are minuscule compared to the aforementioned. VC pouring money in LLM infra is legitimately crazy to me. It's clear as day that there will be winners of this AI cycle, but, as always, they will be companies that pro…

> We need a Dropbox or a Slack or an Instagram: something people love that makes their life easier or better. People seem to not mind ChatGPT or Claude and safe to say that a very large majority of AI products are using one of the APIs of those companies.

In my humble opinion, a chat interface (API or not) does not a product make. Not to mention that Llama is free and competitive with both (same with Mistral, heck the 7B model works great on my RTX 3080). If you started a company that blew up because you made a badass product (and let's say you used ChatGPT under the hood), you would just eventually train and deploy your own model because an LLM is not a product.

Re: Why AI Infrastructure Startups Are Insanely Hard to Build

#34

Post by fhd2 »

mlsu wrote: post #29
jdross wrote: post #21

Earlier quoted context omitted.

personally? - writing and refactoring code. probably 50 times a day now - improving documentation across the company - summarizing meetings automatically with follow ups - drafting most legal work before a lawyer edits (saved 70% on legal bills) - entity extraction and data cleanup for my users

Put a number on it. How much value of this will they capture from you personally (we'll assume, very very charitably by the sound of it, that you represent an "average" user of AI products) when this market matures? Exactly how much will your employer pay for a meeting summarizer? $10/mo a seat, $20/mo a seat, $50/mo a seat? Could the product sustain a 5x, 10x, 50x price hike that is going to have to happen to recoup…

Agreed. Even if right now this seems like stuff companies want to throw money at for novelty/FOMO related reasons, I think eventually reality ought to catch up.

Probably an unpopular opinion, but I think the most efficient companies of the future will tackle the ironies of automation effectively: Carefully designing semi automation that keeps humans in the loop in a way that maximises their value - as opposed to just being bored rubber stamping the automation without really paying attention.

Re: Why AI Infrastructure Startups Are Insanely Hard to Build

#36

Post by rsynnott »

pierre wrote: post #3

Good article, but what is the alternative? What can you build today as a software engineer that can have impact? Nothing seems to come close to AI / AI infra, even of its hard / risky / a moving landscape.

Define ‘impact’. Does ‘impact’ here mean ‘tickles the fancy of a 2024-era VC’? If so, you may be right. If used in its common meaning, absolutely not; most of this stuff is ~useless.

Re: Why AI Infrastructure Startups Are Insanely Hard to Build

#37

Post by swalsh »

mehulashah wrote: post #20

The title is true. But, the arguments don't hold water for me. 12 years ago, I started a big data company. It looked similar for big data companies when Cloudera raised almost $1B in 2014. Too many people building data warehouses, especially in the cloud. I exited. Who knew that Snowflake and Databricks would emerge against the incumbents. Similarly, there will be winners in the AI infrastructure space. To win, you n…

I've worked with hortonworks, cloudera, and databricks. It's no surprise at all that databricks is killing the competition. Those other companies products were embarrassingly terrible. Not stable, slow, and worst of all, I had instances of wrong results. The software just wasn't good.

Databricks is different, it's fast, it's robust, I trust the results. They just built a good quality product.

Re: Why AI Infrastructure Startups Are Insanely Hard to Build

#38

Post by mlsu »

siquick wrote: post #27
dvt wrote: post #18

Infra has always been a tarpit idea. Google didn't start out as an "infra" company, and neither did Amazon or Facebook. In fact, the few "infra companies" that did start back then (companies like Godaddy) are minuscule compared to the aforementioned. VC pouring money in LLM infra is legitimately crazy to me. It's clear as day that there will be winners of this AI cycle, but, as always, they will be companies that pro…

> We need a Dropbox or a Slack or an Instagram: something people love that makes their life easier or better. People seem to not mind ChatGPT or Claude and safe to say that a very large majority of AI products are using one of the APIs of those companies.

there is a lot of consumer surplus, but it’s very unclear if any one of these companies will be able to capture it. Especially if mostly good enough commodity LLMs like Llama 70b are in the mix.

Where is the differentiation?

Re: Why AI Infrastructure Startups Are Insanely Hard to Build

#39

Post by physicsguy »

If I'm an application developer or manager, at any >100 person company, it normally doesn't fall into my remit to go out and pick a new company to contract with to provide services. Typically, it gets harder and harder to do that. Even with LLM stuff, we're contracting that through our existing relationship with Microsoft. When evaluating infra options, it therefore is a huge barrier to entry for most developers if there's a 'good enough' option on one of the main cloud providers

Re: Why AI Infrastructure Startups Are Insanely Hard to Build

#40

Post by justincormack »

mschwarz wrote: post #35

Why does the author claim that Adept was acquired by Amazon? The linked article says they hired away the CEO and key staff.

It was a weirdly structured deal that in effect was an acquisition. The investors were paid off.
Post reply on HN