Earlier quoted context omitted.
> This is legitimately just the same damn hype train the tech sector is constantly attempting to create. Now AI is the next internet. Before that it was the metaverse. Before that it was NFTs. Before that it was cryptocurrency. Before that it was quantum. Before that it was VR. Before that it was AR. Neither of those things you listed were mainstream trends. If you can not distinguish between fads and major trends th…
> Neither of those things you listed were mainstream trends They certainly were sold as trends about to go mainstream.
Why AI Infrastructure Startups Are Insanely Hard to Build
181–190 of 190 posts
Re: Why AI Infrastructure Startups Are Insanely Hard to Build
#182Earlier quoted context omitted.
> Neither of those things you listed were mainstream trends They certainly were sold as trends about to go mainstream.
I didn't hear of them or used the technologies you mentioned as much as AI. Anecdotal, but I don't find them remotely comparable.
Re: Why AI Infrastructure Startups Are Insanely Hard to Build
#183Earlier quoted context omitted.
Pretty much this, 18 months ago my CEO told me we HAD to get into this space, and I told him that basically our money came from our private product and that the only way our big enterprise customers were going to play game with us was either ironclad agreements that went all the way to openai, or more likely a completely single tenant system, which would cost far more than they were willing to pay. Of course they wen…
Can you go into details (as much as you are comfortable) on what happened with the single tenant system? I have seen a few things, but I find it hard to put a finger on what went wrong except the ROI wasnt there. Would love to understand your experience.
I pushed for local first models but the cost tradeoff just did not make sense for anything but the biggest clients, and they were constantly swapping back and forth whether openai would be acceptable or not for "insert sensitive use case here"
Mostly just a big cluster
Re: Why AI Infrastructure Startups Are Insanely Hard to Build
#184Earlier quoted context omitted.
Pretty much this, 18 months ago my CEO told me we HAD to get into this space, and I told him that basically our money came from our private product and that the only way our big enterprise customers were going to play game with us was either ironclad agreements that went all the way to openai, or more likely a completely single tenant system, which would cost far more than they were willing to pay. Of course they wen…
I fully expect in somewhere around 3-6 months the dam will burst and we're going to start hearing more and more about all the teams out there that are pouring tens of millions of dollars into AI and all they have to show for it is a worse version of whatever it is they were doing. To placate the AI fans, that's not because AI isn't interesting, it's because that's how these hype cycles always go. I remember when ever…
That was probably before my time. Was it really "cool"? Like big data, cloud and agile cool? Or more like ... dunno ... some design pattern? So hard to think about XML as having been cool.
Re: Why AI Infrastructure Startups Are Insanely Hard to Build
#185Earlier quoted context omitted.
AI is a long term trend, next is more products on top of AI. Just like the internet. Biotech and space will both be trends but slower and less bubbly because the cost to play is high, though the returns are possibly huge in both.
> AI is a long term trend, next is more products on top of AI. Just like the internet. This is legitimately just the same damn hype train the tech sector is constantly attempting to create. Now AI is the next internet. Before that it was the metaverse. Before that it was NFTs. Before that it was cryptocurrency. Before that it was quantum. Before that it was VR. Before that it was AR. None of those were the revolution…
> Blockchains certainly have a future as it's a highly useful technology
No trolling here; I promise. Are you saying that blockchain is already "highly useful technology", or that we will in the future? From my perspective, it is a very cool technology concept that has yet to demonstrate any major commercial value. I also seriously doubt it will be commercially valuable ever; it has already existed for more than 10 years without any killer app (ignoring shitcoins).Re: Why AI Infrastructure Startups Are Insanely Hard to Build
#186Earlier quoted context omitted.
In my humble opinion, a chat interface (API or not) does not a product make. Not to mention that Llama is free and competitive with both (same with Mistral, heck the 7B model works great on my RTX 3080). If you started a company that blew up because you made a badass product (and let's say you used ChatGPT under the hood), you would just eventually train and deploy your own model because an LLM is not a product .
LLMs are clearly products! The fact that Meta rather inexplicably chooses to give away assets that cost millions or billions to create doesn't mean LLMs aren't a product, it just means that they're competing against a company funding open source stuff for (presumably) strategic reasons, like in many other markets. And like in many other markets over time it's possible the proprietary versions will establish permanent…
> The fact that Meta rather inexplicably chooses to give away assets that cost millions or billions to create doesn't mean LLMs aren't a product
Real question: Why are so many LLMs given away for free? Are they hoping to crush non-free alternatives?EDIT
Your last paragraph makes an excellent point. In the near future, I could see big corps paying OpenAI (or a competitor) to train a private LLM on their squillion internal documents and build a very good helpdesk agent. (Legal and compliance would love it.)
Re: Why AI Infrastructure Startups Are Insanely Hard to Build
#187Earlier quoted context omitted.
We’re already seeing a lot of competition between LLMs. They are quickly becoming commodities. Margins will approach zero and the real value proposition will be with consumer products that extend beyond an .
I agree with you. I'm using a couple of different LLMs depending on what I'm doing and what happens to be easiest but the difference between them is marginal in my experience. The only play for OpenAI et al in my opinion is to try to pull up the draw bridge behind them by getting legislation passed which makes compliance prohibitively difficult if that's not your core business.
Re: Why AI Infrastructure Startups Are Insanely Hard to Build
#188This part:
> For AI infra startups to be “venture scale”, they will eventually need to win over enterprise customers. No question. That requires the startups to have some sustainable edge that separates their products from the incumbents’ (GCP, AWS, as well as the likes of Vercel, Databricks, Datadog, etc).
On the surface, I agree. But look at a parallel market segment: Cheap cloud hosting. Think: Linode (or any of its competitors). There are a bunch of cheap cloud providers who are more than 10 years old. They didn't all get bought out nor bankrupt by up-starts. Why? They must add just enough value to stay in business. Could we see something similar in the AI infra space? In fact, it looks more logical for the cheap cloud providers to try to build some AI infra -- low hanging fruit, to help with LLM training. (I am sure they already see GPU time.)Re: Why AI Infrastructure Startups Are Insanely Hard to Build
#189Earlier quoted context omitted.
LLMs are clearly products! The fact that Meta rather inexplicably chooses to give away assets that cost millions or billions to create doesn't mean LLMs aren't a product, it just means that they're competing against a company funding open source stuff for (presumably) strategic reasons, like in many other markets. And like in many other markets over time it's possible the proprietary versions will establish permanent…
> The fact that Meta rather inexplicably chooses to give away assets that cost millions or billions to create doesn't mean LLMs aren't a product Real question: Why are so many LLMs given away for free? Are they hoping to crush non-free alternatives? EDIT Your last paragraph makes an excellent point. In the near future, I could see big corps paying OpenAI (or a competitor) to train a private LLM on their squillion int…
Giving expensive things away for free is a great marketing technique that has been used since time immemorial, so why startups like Stability do it is somewhat understandable. And OpenAI uses free API access as a loss leader for their API product so that's understandable too.
Why Meta/Google/others do open weight releases is a bit less clear. Recall though that the first Llama wasn't really an open source release. You had to sign a document saying you were a researcher to get the weights, and that document was an agreement to keep the weights secret. Two people signed the documents, anonymously compared their weights, discovered they weren't watermarked (i.e. Meta didn't take this seriously, it was a sop to their AI politics/safety people) and promptly leaked them.
Presumably this was useful for the more libertarian wing of Meta as they could then prove the sky wouldn't fall, and so the influence shifted towards those arguing for more openness in research in general. With that Rubicon crossed other companies didn't see competitive advantage in withholding their similar sized models anymore and followed the leader, so to speak.
Sometimes it also feels like Meta may have over-purchased GPUs and - lacking a public cloud - have just decided to let their researchers do what they wanted. Which is great for the public! But we mustn't be too overconfident. This is really only possible because of Zuckerberg's unique corporate structure that makes him unfirable, combined with Meta being a big data company. It's really benefiting all of humanity here because he's invulnerable to board action so doesn't have to worry about heat from shareholders over 'wasting' money like this.
There's a lot of R&D being done right now on shrinking models whilst preserving quality, so hopefully the Zuck's generosity is enough to ride the open AI research community through the hard times when you needed billions to train LLMs.
Re: Why AI Infrastructure Startups Are Insanely Hard to Build
#190Earlier quoted context omitted.
I fully expect in somewhere around 3-6 months the dam will burst and we're going to start hearing more and more about all the teams out there that are pouring tens of millions of dollars into AI and all they have to show for it is a worse version of whatever it is they were doing. To placate the AI fans, that's not because AI isn't interesting, it's because that's how these hype cycles always go. I remember when ever…
> XML Was Cool That was probably before my time. Was it really "cool"? Like big data, cloud and agile cool? Or more like ... dunno ... some design pattern? So hard to think about XML as having been cool.