Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI's plans according to sama

humanloop.com

Post reply on HN
Print view View original

201–210 of 269 posts

Re: OpenAI's plans according to sama

#201

Post by ma2rten »

HarHarVeryFunny wrote: post #51
londons_explore wrote: post #5

> is limited by GPU availability. Which is all the more curious, considering OpenAI said this only in January: > Azure will remain the exclusive cloud provider for all OpenAI workloads across our research, API and products [1] So... OpenAI is severely GPU constrained, it is hampering their ability to execute, onboard customers to existing products and launch products. Yet they signed an agreement not to just go rent…

There's an interesting recent video here from Microsoft discussing Azure. The format is a bit cheesy, but lots of interesting information nonetheless. https://www.youtube.com/watch?v=Rk3nTUfRZmo&t=5s "What runs ChatGPT? Inside Microsoft's AI supercomputer" The relevance here is that Azure appears to be very well designed to handle the hardware failures that will inevitably happen during a training run taking weeks or…

That is only relevant for serving and not for inference, unless the model is too big to fit on a single host (typically 8 GPUs).

Re: OpenAI's plans according to sama

#202

Post by zamalek »

verdverm wrote: post #42
chaostheory wrote: post #28

Earlier quoted context omitted.

Even if they weren’t exclusive with Azure, aren’t GPU prices reasonable again?

They have to be a available to buy, regardless the price. My understanding is there is a distinct lack of supply

Barring a revolution in chip manufacture, there likely will always be a lack of supply relative to consumer GPUs. The size of the die results in terrible yields.

Re: OpenAI's plans according to sama

#203

Post by ilaksh »

twobitshifter wrote: post #136

Left the best part until the end. Scaling models larger is still paying off for openai. It’s not AGI yet, but how much bigger will a model need to get to max out? >The scaling hypothesis is the idea that we may have most of the pieces in place needed to build AGI and that most of the remaining work will be taking existing methods and scaling them up to larger models and bigger datasets. If the era of scaling was over…

Why don't people ever explain what they mean by AGI? It means different things to different people.

Re: OpenAI's plans according to sama

#204

Post by tirpen »

f6v wrote: post #198
bane wrote: post #188

Earlier quoted context omitted.

Most of the companies I work with are actively putting in place policies to prevent employees from using OpenAI's service because nobody wants to send their proprietary IP to them. Almost all of these companies have the technical ability, desire, and means to self-host for their employee community. Imagine the internal coup for CTO/CIOs everywhere to buy whatever is the latest Nvidia GPU cluster box, stick it in the…

> Almost all of these companies have the technical ability, desire, and means to self-host for their employee community. Well, one of companies I worked for could have hosted a canary service for cron jobs. But we bought it instead of building because we were focused on building features. And here you’re talking about hosting an entire LLM.

The cron job canary was probably not a service that employees were uploading tonnes of company confidential material into, was it? So I fail to see how the comparison makes sense.

The reason companies shun OpenAI and want a self hosted alternative isn't related to costs, it's becasue they don't want their code, internal emails, documentation etc to be uploaded to Microsoft and thus also directly to the NSA.

Re: OpenAI's plans according to sama

#205

Post by thomasahle »

brucethemoose2 wrote: post #158
timeserious wrote: post #147

Earlier quoted context omitted.

Can you ELI5 why an embeddings database helps here? Can pinecone/milvus be used to 'extend memory' of OSS and vendor LLMs without retraining?

First some context: llm "prompts" are actually the whole conversation + initial context. They learn nothing, hence the whole conversation gets fed into them every time, but the instruction following ones are trained to answer your most recent chat response. In a nutshell, part of your llm prompt (usually your most recent question?) gets fed as a query for the embedding/vector database. It retrieves the most "similar"…

> part of your llm prompt (usually your most recent question?) gets fed as a query for the embedding/vector database

How is it embedded? Using a separere embedding model, like Bert or something? Or do you use the LLM itself somehow? Also, how do you create content for the vector database keys themselves? Also just some arbitrary off the shelf embedding? Or do you train it as part of training the LLM?

Re: OpenAI's plans according to sama

#206

Post by littlestymaar »

nonethewiser wrote: post #113
purplecats wrote: post #32

> Cheaper and faster GPT-4 — This is their top priority. In general, OpenAI’s aim is to drive “the cost of intelligence” down as far as possible and so they will work hard to continue to reduce the cost of the APIs over time. this certainly aligns with the massive (albeit subjective and anecdotal) degradation in quality i've experienced with ChatGPT GPT-4 over the past few weeks. hopefully a superior (higher quality)…

I wonder of it actually is because they’re tuning it to make it less offensive (by their standards). Thats the only explanation I keep seeing repeated.

That's a convenient explanation that's been repeated over and over by certain people, but cost is a much more likely explanation: inference for large models is extraordinarily expensive when you have millions of users and their pricing model always seemed way too low to pay for that.

They have likely been subsidizing their users since the launch of their commercial offering (and this is pretty common strategy for SV startups) but they've been so successful that they now need to scale the cost down in order not to burn all their cash too fast.

Re: OpenAI's plans according to sama

#207

Post by braindead_in »

sharkjacobs wrote: post #24

> He reiterated his belief in the importance of open source and said that OpenAI was considering open-sourcing GPT-3. Part of the reason they hadn’t open-sourced yet was that he was skeptical of how many individuals and companies would have the capability to host and serve large LLMs. Am I reading this right? "We're not open sourcing GPT-3 because we don't think it would be useful to anyone else"

It is a shame that Sama does not believe in Open Source. The community can solve their GPU bottleneck issue by making it run on CPUs and edge devices in a matter of days.

Re: OpenAI's plans according to sama

#208

Post by andybak »

typest wrote: post #31
furyofantares wrote: post #16

Earlier quoted context omitted.

PMF meaning "product market fit"? I had to look it up, curious if I found the right thing or not.

Yes, PMF = "product market fit".

Grrrrr. I shouldn't have to play guessing games to read an article.

Re: OpenAI's plans according to sama

#209

Post by theonlybutlet »

outside1234 wrote: post #177
neximo64 wrote: post #168

Earlier quoted context omitted.

In your world how would you consider Sam Altman having no equity in OpenAI? And everyone finding out after it had a viral hit

There is no way this guy hasn't figured out some way to get paid out of this. We just haven't figured out how yet.

Wonder if it's indirectly through some stake in Microsoft or whoever deals with them at that level.
Post reply on HN