121–130 of 218 posts
The one thing I don't understand is this assumption that demand for GPUs for training is going to keep growing at the rate they grew so far. I get the demand for new applications, which require inference , but nowadays with so many good (if not close to SOTA) models available for free and the ability to run them on consumer hardware (apple M4 or AMD Max APUs), is there any demand for applications that justify a crazy…
Fiber networks were using less than 0.002% of available capacity, with potential for 60,000x speed increases. It was just too early. I doubt we will see unused GPU capacity. As soon as we can prompt "Think about the codebase over night. Try different ways to refactor it. Tomorrow, show me your best solution." we will want as much GPU time at the current rate as possible. If a minute of GPU usage is currently $0.10, a…
I just had to double check (have not been paying attention for a couple of years) but indeed it seems GPU underutilization remains a fact and the numbers are pretty significant. Main issues are being memory bound so the compute sits idle.
One of the things before AI in the market was that capital had limited growth opportunities. Tech, which was basically a universe of scaled out crud apps, was where capital would keep going back into. AI is a lot more useful than hyper scaled up crud apps. Comparing this to the past is really overfitting imho. The only argument against accumulating GPUs is that they get old and stop working. Not that it sucks, not th…
We've technically been able to play board games by entering our moves into our telephones, sending them to a CPU to be combined, then printing out a new board on paper to conform to the new board state. We do not do this because it would be stupid. We can not depend on people starting to do this saving the paper, printer, and ink industries. Some things are not done because they are worthless.
I worked at a mom and pop ISP in the 90s. Lucent did seem at the forefront of internet equipment at the time. We used Portmaster 3s to handle dial up connections. We also looked into very early wireless technology from Lucent. Something I wanted to mention, only somewhat tanget. The Telecommunications Act of 1996 forced telecommunication companies to lease out their infrastructure. It massively reduced the prices an…
Knowing history of past bubbles is only mildly informative. The dotcom bubble was different than the railroads bubble etc. The only thing to keep in mind is that all of this is about business and ROI. Given the colossal investments, even if the companies finances are healthy and not fraudulent, the economic returns have to be unprecedented or there will be a crash. They are all chasing a golden goose.
With Bezos openly stating the goal to build 10 GW+ data-centers in space, it almost feels in question whether this is about ROI, or simply building the Neuromancer future where the ultra-rich can finally be free of their need for the rest of us. Not needing labor at all would be the final return on investment. https://news.ycombinator.com/item?id=45465480 https://www.datacenterdynamics.com/en/news/jeff-bezos-claims..…
Earlier quoted context omitted.
> From OpenAI, programming is ~4% of chatGPTs usage. That's 96% being used for other stuff. I think it's important to remember that a good bunch of this is going to be people using it as an artificial friend, which is not really productive. Really that's destructive, because in that time you could be creating a relationship with an actual person instead of a context soon to be deleted. But on the other hand, some peo…
Productive or destructive, demand is there, so it isn’t late bubble. It’s still early. (Which is scary, I’ll readily admit.)
Earlier quoted context omitted.
Thanks for your reply, but: NVidia could protect itself against OpenAI bankrupcy by adding a clause to the lease saying that if OpenAI goes bankrupt, Nvidia gets its GPUs back. So the risk would only be that the lease would be aborted sooner than expected.
> So the risk would only be that the lease would be aborted sooner than expected. That is, in fact, the risk.
The one thing I don't understand is this assumption that demand for GPUs for training is going to keep growing at the rate they grew so far. I get the demand for new applications, which require inference , but nowadays with so many good (if not close to SOTA) models available for free and the ability to run them on consumer hardware (apple M4 or AMD Max APUs), is there any demand for applications that justify a crazy…
I expect models will get larger again once everyone is doing their inference on B200s, but the RL training budget is where the insatiable appetite sits right now.
I worked at a mom and pop ISP in the 90s. Lucent did seem at the forefront of internet equipment at the time. We used Portmaster 3s to handle dial up connections. We also looked into very early wireless technology from Lucent. Something I wanted to mention, only somewhat tanget. The Telecommunications Act of 1996 forced telecommunication companies to lease out their infrastructure. It massively reduced the prices an…
I worked at a startup during the telecom boom. Then, startups were getting acquired by the likes of Cisco before the startup had a deployed product. And, back then, IPOs were the only form of liquidity event and engineers were locked up for 6 months. The lucky ones had their startups go IPO or get acquired with enough time to spare to get out before the ensuing bust. After the bust, funding dried up and most startups…
Monopolies gum up the system, reward the institutional capital rather than innovation capital, and prevent new entrants from de-ossifying and being the renewing forest fire.
We've been so lax on antitrust. Google, Apple, Meta, Amazon - they all need to be broken up. Our economy and our profession would be better for it.
Innovation should be a treadmill.
YC and a16z want this.