Cabs are an idea. Uber is a company. I can't think of any company that I'd predict to last 100 years. Even Google and Apple probably won't last 100 years. I honestly wouldn't even bet on cars being a dominant form of transportation in 100 years.
I think Apple will last 100 years. See their latest AR product. They continue to create amazing products that are relevant to the masses.
I had the opposite impression on AR.
A useless gimmick no sane people will use.
Has anyone actually used GPT4? It's not "cheap". It was roughly $150 for me to build a small dataset with a few thousand quarter-page chunks of text for a data project using GPT4. GPT3 is substantially cheaper but it would hallucinate 30% of the time; honestly a nice fine-tune of LlaMA is on-par with GPT3 and after the sunk cost all it costs is a few $0.01 in electricity to generate the same sized dataset.
Even GPT3.5 can be much more expensive. In some specific tasks, a finetuned 7B llama can work as well as GPT3.5. You can rent a 3090 at $0.20/h on vast.ai, or $0.40/h on runpod. Using VLLM at 400t/s that's 1440000 generated tokens. Generating that amount of tokens with GPT3.5 would be $2.88.
> "In some specific tasks, a finetuned 7B llama can work as well as GPT3.5."
"some" is doing a lot of heavy lifting here.
Also: don't discount the labor cost of curating a fine tuning dataset, running a FT training run, even if the hardware is cheap.
Completely wrong, the best AI will win. There is insane demand for better models.
Only Big Tech (Microsoft,Google,Facebook) can crawl the web at scale because they own the major content companies and they severly throttle the competition's crawlers, and sometimes outright block them. I'm not saying it's impossible to get around, but it is certainly very difficult, and you could be thrown in prison for violating the CFAA.
Common Crawl claims to have 82% of the tokens used to train GPT-3, and it's available to anyone.
Add all the downloadable material at archive.org and you've got a formidable corpus.
Completely wrong, the best AI will win. There is insane demand for better models.
The improvements seem to be leveling off already. GPT-4 isn't really worth the extra price to me. It's not that much better. What I would really want though is an uncensored LLM. OpenAI is basically unusable now, most of its replies are like "I'm only a dumb AI and my lawyers don't want me to answer your question". Yes I work in cyber. But it's pretty insane now.
It's way better at writing code, or at least I've found it to be so.
Yeah I kinda wonder why people even use them anymore. I've long gone back to real taxis because their cheaper and I don't have to book them, I can just grab one on the street. Much more efficient than waiting for slowly watching my driver edge his way to me from 3 kilometers away.
The number of places where you can reliably walk out onto the street and hail a taxi is pretty small. Everywhere else, the relevant decision is whether calling a dispatcher or using a taxi company's app is faster/cheaper/more reliable than Uber/Lyft.
I think the cab companies here have apps. I don't know how good they are, though I've been meaning to find out.
There's no way I want to ever wait on hold for a dispatcher again, or be mystified as to when my cab is arriving (this always seems to involve standing outside in the snow or pouring rain).
If the cab companies have apps comparable to Uber and Lyft, sure, I'll give them a shot.
Has anyone actually used GPT4? It's not "cheap". It was roughly $150 for me to build a small dataset with a few thousand quarter-page chunks of text for a data project using GPT4. GPT3 is substantially cheaper but it would hallucinate 30% of the time; honestly a nice fine-tune of LlaMA is on-par with GPT3 and after the sunk cost all it costs is a few $0.01 in electricity to generate the same sized dataset.
But OpenAI appears to have some sort of data moat. I doubt their model is the best in the world, but more/better data generally beats better model, and GPT-4 definitely beats Claude, Bard, Bernie and the rest probably because they curated the best quality and largest data set. Maybe that moat doesn't last long but perhaps they have exclusive rights to some of that dataset through commercial agreements that could be a…
I'd bet that there are any number of submarine startups out there sitting on top of full downloads of Common Crawl, archive.org, etc. who are only too happy to let OpenAI be the first penguin off the iceberg. If OpenAI survives all the legal challenges, they'll just click "Go" and be in business in weeks to months. If OpenAI gets smacked down, they haven't lost much. There are probably also some submarine operations…
You might be forgetting the cost to train a 1.8T model.
Probably all three: 1) They hiring too talent to make their models as efficient as possible. 2) They have a sweetheart deal with MS. 3) They’re better funded than everyone else and bringing in substantial revenue.
They basically are MS by now. Everyone at Microsoft I work with literally calls it an 'aquisition'. Even though they only own a share. It's pretty clear what their plans are.
> Microsoft will reportedly get a 75% share of OpenAI's profits until it makes back the money on its investment, after which the company would assume a 49% stake in OpenAI.
49% isn't _just_ a share, it's a significant portion of the company.
OpenAI is seeking to make their own chips and hardware.
If they can pull off what Apple has done with its M line, then possibly they can make themselves even more cost effective than the competition. The competition will mostly be limited to supplies from Nvidia.
I believe in house manufacturing of their own hardware is definitely the way forward. Top down lock down. Own the hardware, own the models, own the user trained datasets.
The premise of this is flawed. OpenAI is cheap because of has to be right now. They need to establish market dominance quickly, before competitors slide in. The winner of this horse race is not going to be the company with the best performing AI, it’s going to be the one who does the best job at creating an outstanding UX, ubiquitously presence, entrenching users, and building competitive moats that are not feature d…
The winner is going to be the consumers of AI. It's a race to the bottom on pricing on the provider/infra side. It seems very unlikely that any single LLM provider will achieve a sustained and durable advantage enough to achieve large margins on their product. Consumers can swap between providers with relative ease, and there is very little stickiness to these LLM APIs, since the interfaces are general and operate on…
Certainly the undisputed winners will be the very few firms with enough engineering resources and GPUs to train their own models (not just fine-tune) where the models in question increase the productivity of workers in their non-ai-related profit centers. After that we have the real question of what the future will be of open source LLMs, on the one hand, and the question most relevant to this article of what sort and whether profitable “AI businesses” can be sustained over time. As Stratechery has analyzed, it is very possible that OpenAI turns out to be a very profitable B2C company with ad revenue in ChatGPT not concerned with their B2B sales or even the objective quality of their AI. Right now is an incredible time for AI: cheap Uber rides never qualitatively changed my life, but the current consumer access to AI models is truly incredible and I hope that only improves. However, even ignoring whatever happens on the regulatory front, I don’t think that is guaranteed at all.