OpenAI and Microsoft extend partnership

Post by Spivak » Mon, Jan 23, 2023, 7:34 PM UTC

echelon wrote: post #154

dandiep wrote: post #107

Earlier quoted context omitted.

A couple points which I don't see elsewhere: 1) They have the best quality model. Better quality means more users. More users means more data. Which means higher quality... 2) operationalizing & scaling these these models is non-trivial. I'm not sure what the state of distillation/pruning is for GPT-3, but I imagine they have figured out some proprietary techniques. 3) It's not just publishing a single model, but mak…

> but making it so people can fine tune and push their own How are they making it easy for people to fine tune their own?

Post by EGreg » Mon, Jan 23, 2023, 7:44 PM UTC

OnlineGladiator wrote: post #169

bayindirh wrote: post #163

Earlier quoted context omitted.

Because they didn't want to openly say "Microsoft gives us GPUs and allows us to use them for free, hence practically buys us. In return we almost exclusively license what we produce to them, for free." Because it looks bad. Microsoft wants to be seen in a different light, and OpenAI wants to be perceived as an open, academic, capped-profit for a noble reason collective. It's all smoke and mirrors.

> OpenAI wants to be perceived as an open, academic, capped-profit for a noble reason collective. Does anybody actually believe this? Their name has been a running joke for years already.

Post by meowkit » Mon, Jan 23, 2023, 7:52 PM UTC

OnlineGladiator wrote: post #207

achow wrote: post #206

Earlier quoted context omitted.

Sam Altman addresses that when he was asked that question in 'How I Built This' Podcast. Essentially: OpenAI did not realize the massive scale they needed for them to be successful. When they realized this, they could not raise any funding as 'non-profit'. They asked govt, who did not want to fund it, and other sources, at the end they did not have any other recourse. Sam Altaman says: 90% of funding was needed for c…

It sounds like you agree with me that OpenAI is poorly named. "We wanted to be open, but we couldn't - we kept the name anyway."

Post by [deleted] » Mon, Jan 23, 2023, 8:00 PM UTC

bayindirh wrote: post #163

londons_explore wrote: post #140

Let me translate this blog post out of corpspeak...: OpenAI has agreed to use Azure exclusively and for free, in return for Azure massively scaling up it's GPU capacity (which OpenAI was hitting before, hence threatening to move). In return, OpenAI gives an extensive license to Microsoft to use ChatGPT in Bing (and other products). Now why couldn't the blog post have just said that rather than hiding all the details?

Because they didn't want to openly say "Microsoft gives us GPUs and allows us to use them for free, hence practically buys us. In return we almost exclusively license what we produce to them, for free." Because it looks bad. Microsoft wants to be seen in a different light, and OpenAI wants to be perceived as an open, academic, capped-profit for a noble reason collective. It's all smoke and mirrors.

Post by theropost » Mon, Jan 23, 2023, 8:01 PM UTC

whimsicalism wrote: post #136

theropost wrote: post #124

Earlier quoted context omitted.

I recently wondered if one of the reasons for Google shutting down Stadia, was to quickly ramp their GPU server stockpile to redirect the resources at GPT modelling, to help catch up.

Google is not constrained by GPUs here and likely will train on TPU pods anyways.