Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI's plans according to sama

humanloop.com

Post reply on HN
Print view View original

21–30 of 269 posts

Re: OpenAI's plans according to sama

#21

Post by mustacheemperor »

sovietmudkipz wrote: post #7

I’m hoping GPT will remove the information cutoff date. I write plenty of terraform/AWS and it’s a bit of a pain that the latest API isn’t accessible by GPT yet. There’s been quite a bit happening in the programming space since sept 2021. I use GPT to keep things high level and then do my normal research methodology for implementation details.

I enjoy using GPT4 as a co-programmer, and funny enough it is very challenging to get advice on Microsoft's own .NET MAUI because that framework was in prerelease at the time the model was trained.

My understanding is right now they essentially need to train a new model on a new updated corpus to fix this, but maybe some other techniques could be devised...or they'll train something more up to date.

Re: OpenAI's plans according to sama

#22

Post by gistbug »

yesimahuman wrote: post #11

The bit about plugins not having PMF is interesting and possibly flawed. I, like many others, got access to plugins but not the browsing or code interpreter plugins which feel like the bedrock plugins that make the whole offering useful. I think there's also just education that has to happen to teach users how to effectively use other plugins, and the UX isn't really there to help new users figure out what to even do…

Yea, seems weird to allow people to use plugins, but not all of them. Then have the gall to say that no one is using plugins, yea because half of them don't have any context outside of America.

Re: OpenAI's plans according to sama

#23

Post by arugulum »

hervature wrote: post #4

I never know if I have an inside scoop or an outside scoop. Has Hyena not addressed the scaling of context length [1]? I know this version is barely a month old but it was shared to me by a non-engineer the week it came out. Still, giving interviews where the person takes away that the main limitation is context length and requires a big breakthrough that already happened makes me seriously question whether or not he…

As someone who is in the field: papers proposing to solve the context length problem come out every month. Almost none of the solutions stick or work as well as a dense or mostly dense model.

You'll know when the problem is solved when model after consistently use a method. Until then (and especially if you're not in the field as a researcher), assume that every paper claiming to tackle context length is simply a nice proposal.

Re: OpenAI's plans according to sama

#24

Post by sharkjacobs »

> He reiterated his belief in the importance of open source and said that OpenAI was considering open-sourcing GPT-3. Part of the reason they hadn’t open-sourced yet was that he was skeptical of how many individuals and companies would have the capability to host and serve large LLMs.

Am I reading this right? "We're not open sourcing GPT-3 because we don't think it would be useful to anyone else"

Re: OpenAI's plans according to sama

#26

Post by greenie_beans »

sharkjacobs wrote: post #24

> He reiterated his belief in the importance of open source and said that OpenAI was considering open-sourcing GPT-3. Part of the reason they hadn’t open-sourced yet was that he was skeptical of how many individuals and companies would have the capability to host and serve large LLMs. Am I reading this right? "We're not open sourcing GPT-3 because we don't think it would be useful to anyone else"

lmao i had the same reaction. sounds like some bullshit.

Re: OpenAI's plans according to sama

#27

Post by bibanez »

sharkjacobs wrote: post #24

> He reiterated his belief in the importance of open source and said that OpenAI was considering open-sourcing GPT-3. Part of the reason they hadn’t open-sourced yet was that he was skeptical of how many individuals and companies would have the capability to host and serve large LLMs. Am I reading this right? "We're not open sourcing GPT-3 because we don't think it would be useful to anyone else"

I agree, this is so bizarre

Re: OpenAI's plans according to sama

#28

Post by chaostheory »

londons_explore wrote: post #5

> is limited by GPU availability. Which is all the more curious, considering OpenAI said this only in January: > Azure will remain the exclusive cloud provider for all OpenAI workloads across our research, API and products [1] So... OpenAI is severely GPU constrained, it is hampering their ability to execute, onboard customers to existing products and launch products. Yet they signed an agreement not to just go rent…

Even if they weren’t exclusive with Azure, aren’t GPU prices reasonable again?

Re: OpenAI's plans according to sama

#29

Post by sebzim4500 »

londons_explore wrote: post #5

> is limited by GPU availability. Which is all the more curious, considering OpenAI said this only in January: > Azure will remain the exclusive cloud provider for all OpenAI workloads across our research, API and products [1] So... OpenAI is severely GPU constrained, it is hampering their ability to execute, onboard customers to existing products and launch products. Yet they signed an agreement not to just go rent…

>So... OpenAI is severely GPU constrained, it is hampering their ability to execute, onboard customers to existing products and launch products. Yet they signed an agreement not to just go rent a bunch of GPU's from AWS???

> Did someone screw up by not putting a clause in that contract saying "exclusive cloud provider, unless you cannot fulfil our requests"?

Maybe MSFT refused to sign such an agreement?

Re: OpenAI's plans according to sama

#30

Post by sebzim4500 »

sharkjacobs wrote: post #24

> He reiterated his belief in the importance of open source and said that OpenAI was considering open-sourcing GPT-3. Part of the reason they hadn’t open-sourced yet was that he was skeptical of how many individuals and companies would have the capability to host and serve large LLMs. Am I reading this right? "We're not open sourcing GPT-3 because we don't think it would be useful to anyone else"

It is weird, but GPT-3 is worse than much smaller LLaMA models so I doubt it would see much use anyway.
Post reply on HN