481–490 of 495 posts
Earlier quoted context omitted.
It’s not that the fetching is the problem, but serving the data to many cores at the same time from a single source.
I'm not familiar with GPU architecture, is there not a shared L2/L3 data cache from which this data would be shared?
Earlier quoted context omitted.
Free tier provides a lot of training material. Every time you correct ChatGPT on its mistakes you’re giving them knowledge that’s not in any book or website. Thats a moat, albeit one that is slow to build.
That's interesting, though you have to imagine the data set is very low quality on average and distilling high quality training pairs out of it is very costly.
Earlier quoted context omitted.
There’s still untapped value in deeper integrations. They might hit a jackpot of exponentially increasing value from network effects caused by tight integration with e.g. disjoint business processes. We know that businesses with tight network effects can grow to about 2 trillion in valuation.
How would that look with at least 3 US companies, probably 2 Chinese ones and at least 1 European company developing state of the art LLMs?
Facebook marginalized linkedin and sent twitter into a niche.
Internet Explorer and Windows destroyed competition, for a long while.
Google Search marginalized everyone for over 20 years.
These are multi-trillion-dollar businesses. If OpenAI creates a network effect of some sort they can join the league.
Earlier quoted context omitted.
It’s generally useful to consider unit economy separate from whole company. If your unit economy is negative thing are very bleak. If it’s positive, your chance are going up by a lot - scaling the business amortizes fixed (non-unit) costs, such as admin and R&D, and slightly improves unit margins as well. However this does not work as well if your fixed (non-unit) cost is growing exponentially. You can’t get out of t…
It’s splitting a hair, but a pretty important hair. Does anyone think that models won’t need continuous retraining? Does anyone think models won’t continue to try to scale? Personally, I think we’re reaching diminishing returns with scaling, which is probably good because we’ve basically run out of content to train on, and so perhaps that does stop or at least slow down drastically. But I don’t see a scenario where c…
Earlier quoted context omitted.
I can't imagine the hoops an accountant would have to go through to argue training cost is COGS. In the most obvious stick-figures-for-beginners interpretation, as in, "If I had to explain how a P&L statement works to an AI engineer", training is R&D cost and inference cost is COGS.
I wasn’t using COGS in a GAAP sense, but rather as a synonym for unspecified “costs.” My bad. I suppose you would classify training as development and ongoing datacenter and GPU costs as actual GAAP COGS. My point was, if all you focus on is revenue and ignore the costs of creating your business and keeping it running, it’s pretty easy for any business to be “profitable.”
Earlier quoted context omitted.
"Why would you reinvest profits back into a business that is extremely profitable, when you have the chance of pulling your money out?"
You are making a joke but reasonably speaking there are a ton of software companies where they kept reinvesting where they should have taken out profit, especially when they are peaking.
From https://www.theverge.com/command-line-newsletter/759897/sam-... , Sam Altman said: > “If we didn’t pay for training, we’d be a very profitable company.”
that's true of any company. if they didn't pay for building the product, they would be very profitable.
Everyone claiming AI companies are a financial ticking time bomb are using the same logic people used back in the 2000s when they claimed Amazon “never made a profit” and thus was a bad investment.
Wrong. The depreciation cost on the hundreds of billions of dollars spent on the AI build-out is almost certainly larger then the AI industry's gross income. This is a vastly different depreciation cost schedule than AWS.
Earlier quoted context omitted.
It’s splitting a hair, but a pretty important hair. Does anyone think that models won’t need continuous retraining? Does anyone think models won’t continue to try to scale? Personally, I think we’re reaching diminishing returns with scaling, which is probably good because we’ve basically run out of content to train on, and so perhaps that does stop or at least slow down drastically. But I don’t see a scenario where c…
Well, models are definitely good enough for some things in their current state, without needing to be retrained (computer translation for example was a solved problem with GPT3)
Model context limits are not “artificial” as claimed. The largest context window a model can offer at a given quality level depends on the context size the model was pretrained with as well as specific fine tuning techniques. It’s not simply a matter of considering increased costs.
Context extension methods exist and work. Please educate yourself about these rather than confidentially saying wrong things.
There are a few ways to approach the problem. Pre-training on longer context lengths I’ve already mentioned. Fine-tuning techniques (like LongRoPE) I’ve already mentioned.
Inference time context extension tricks I didn’t mention because the papers I’ve seen seem to suggest there’s often problems with quality or unfavorable tradeoffs.
There’s no magic way around these limits, it’s a real engineering problem.