Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI's plans according to sama

humanloop.com

Post reply on HN
Print view View original

181–190 of 269 posts

Re: OpenAI's plans according to sama

#181

Post by dontupvoteme »

_boffin_ wrote: post #121
dontupvoteme wrote: post #68

Earlier quoted context omitted.

https://en.wikipedia.org/wiki/Embrace,_extend,_and_extinguis...

yes... I know about this, but that's not what I'm asking about. I'm asking about it removing partial answers as it's writing them. Please make more effort next time than to provide me with a Wiki article.

When i tried out bing 1-2 weeks ago and asked it for code it would start writing and after a few lines, realize it was writing code and stop and delete what it had written.

I tried it again tonight and it seems like they fixed it to only produce small amounts of mediocre code instead.

Re: OpenAI's plans according to sama

#182

Post by JeremyNT »

jiggawatts wrote: post #103
londons_explore wrote: post #5

> is limited by GPU availability. Which is all the more curious, considering OpenAI said this only in January: > Azure will remain the exclusive cloud provider for all OpenAI workloads across our research, API and products [1] So... OpenAI is severely GPU constrained, it is hampering their ability to execute, onboard customers to existing products and launch products. Yet they signed an agreement not to just go rent…

One of Azure's unique offerings is very large HPC clusters with GPUs. You can deploy ~1,000 node scale sets with very high speed networking. AWS has many single-server GPU offerings, but nothing quite like what Azure has. Don't assume Microsoft is bad at everything and that AWS is automatically superior at all product categories...

Whether MS is good or not isn't really the point. If they're constrained by GPU availability, being locked in to any specific provider is going to be a problem.

Re: OpenAI's plans according to sama

#184

Post by jasmer »

twobitshifter wrote: post #136

Left the best part until the end. Scaling models larger is still paying off for openai. It’s not AGI yet, but how much bigger will a model need to get to max out? >The scaling hypothesis is the idea that we may have most of the pieces in place needed to build AGI and that most of the remaining work will be taking existing methods and scaling them up to larger models and bigger datasets. If the era of scaling was over…

'It's not AGI yet' - the implication is insufferable. It's a language model that is incapable of any kind of reasoning, the talk of 'AGI' is a glib utopianism, a very heavy kind of koolaid. If we were to have referred to this tech as anything other than 'intelligence' - for example, if we chose 'adaptive algorithms' or 'weighted node storage' etc. we'd likely have a completely different popular mental model for it.

There will be no 'AI model' that is 'AGI', rather, a large swath of different technologies and models, operating together, will give the appearance of 'AGI' via some kind of interface.

It will not appear as an 'automaton' (aka single processing unit) and it certain will not be an 'aha moment'.

In 10 years, you'll be able to ask various agents, of different kinds, which will use varying kinds of AI to interpret speech, to infer context, which will interface with various AI APIs, in many ways it'll resemble what we have today but with more nuance.

The net appearance will evolve over time to appear a bit like 'AGI' but there won't be an 'entity' to identify as 'it'.

Re: OpenAI's plans according to sama

#185

Post by refulgentis »

ftxbro wrote: post #137
refulgentis wrote: post #105

Earlier quoted context omitted.

It probably keeps getting flagged because it’s ahistorical, source: OpenAI engineers, and #2 somewhat obviously so. You heard of RLHF?

> You heard of RLHF? The conspiracy theory isn't that every employee of OpenAI spent 8 hours every day for six months in meetings with govt agencies.

not sure what you mean. anyways, the reason why they don’t release GPT4 when they’re “”“done””” training in June is they have to RLHF

Re: OpenAI's plans according to sama

#187

Post by barbariangrunge »

sharkjacobs wrote: post #24

> He reiterated his belief in the importance of open source and said that OpenAI was considering open-sourcing GPT-3. Part of the reason they hadn’t open-sourced yet was that he was skeptical of how many individuals and companies would have the capability to host and serve large LLMs. Am I reading this right? "We're not open sourcing GPT-3 because we don't think it would be useful to anyone else"

If small organizations and teams can’t use it, then open sourcing it mostly just benefits big tech

That’s not ideal

How does open source licensing work with respect to trained ai models anyway? Is something like the mit license even that valuable here? Or is it?

Re: OpenAI's plans according to sama

#188

Post by bane »

sharkjacobs wrote: post #24

> He reiterated his belief in the importance of open source and said that OpenAI was considering open-sourcing GPT-3. Part of the reason they hadn’t open-sourced yet was that he was skeptical of how many individuals and companies would have the capability to host and serve large LLMs. Am I reading this right? "We're not open sourcing GPT-3 because we don't think it would be useful to anyone else"

Most of the companies I work with are actively putting in place policies to prevent employees from using OpenAI's service because nobody wants to send their proprietary IP to them.

Almost all of these companies have the technical ability, desire, and means to self-host for their employee community. Imagine the internal coup for CTO/CIOs everywhere to buy whatever is the latest Nvidia GPU cluster box, stick it in the on-prem datacenter, load a licensed GPT model and provide "AI as a service to our employees".

Except what's happening is everybody is looking at buying the box from Nvidia, and sticking a large actually open model on it and simply ignoring OpenAI.

Re: OpenAI's plans according to sama

#189

Post by doctor_eval »

choppaface wrote: post #169
razcle wrote: post #120

Earlier quoted context omitted.

I think I worded this poorly. What he said was that a lot of people say they want open-source models but they underestimate how hard it is to serve them well. So he wondered how much real benefit would come from open-sourcing them. I think this is reasonable. Giving researchers access is great but for most small companies they're likely better off having a service provider manage inference for them rather than naviga…

Here is how hard it is to serve and use LLMs: https://github.com/ggerganov/llama.cpp

“The original implementation of llama.cpp was hacked in an evening.”

Re: OpenAI's plans according to sama

#190

Post by doctor_eval »

londons_explore wrote: post #5

> is limited by GPU availability. Which is all the more curious, considering OpenAI said this only in January: > Azure will remain the exclusive cloud provider for all OpenAI workloads across our research, API and products [1] So... OpenAI is severely GPU constrained, it is hampering their ability to execute, onboard customers to existing products and launch products. Yet they signed an agreement not to just go rent…

Let’s not forget that Microsoft is a big investor in OpenAI. It is important to know on which side your bread is buttered.
Post reply on HN