"Powered by Azure" -- Elon clearly distrust Amazon.
Elon is no longer part of OpenAI. Microsoft invested $1b. https://en.wikipedia.org/wiki/OpenAI
It's not just OpenAI, most of his ventures uses Azure.
OpenAI API
151–160 of 165 posts
Re: OpenAI API
#152In one of their examples, they note “They saw ratings hover around 60% with their original, in-house tech — this improved by 7-8% with GPT-2 — and is now in the 80-90% range with the API.” Bloomberg reports the API is based on GPT-3 and “other language models”. If that’s true, this is a big deal, and it epitomizes OpenAI’s namesake. The largest NLP models require vast corporate resources to train, let alone put into…
How is this "democratization"? OpenAI trains a model, then they make it available through an API. You have no say in what that model is trained on or how (other than to say whether they can use your data- but not how), neither can you modify the model according to your needs. And of course, with no ability to modify the product you're buying you have no opportunity to innovate. You can wrap it up in a different kind…
Odd analogy to use with Microsoft Windows, since GPT-3’s source is available, along with a series of papers that enables anyone with the money and knowledge to implement it themselves.
Re: OpenAI API
#153Earlier quoted context omitted.
How is this "democratization"? OpenAI trains a model, then they make it available through an API. You have no say in what that model is trained on or how (other than to say whether they can use your data- but not how), neither can you modify the model according to your needs. And of course, with no ability to modify the product you're buying you have no opportunity to innovate. You can wrap it up in a different kind…
Odd analogy to use with Microsoft Windows, since GPT-3’s source is available, along with a series of papers that enables anyone with the money and knowledge to implement it themselves.
The reasons why MS windows and GPT-3 cannot easily be modified by anyone are different, but the result is the same: you're stuck with what you're sold.
To clarify: MS windows is closed source, but you can't very well train a large GPT model unless you're someone with the resources of OpenAI. So you're stuck with whatever they choose to train and make available to you.
Re: OpenAI API
#154Earlier quoted context omitted.
Odd analogy to use with Microsoft Windows, since GPT-3’s source is available, along with a series of papers that enables anyone with the money and knowledge to implement it themselves.
The reasons why MS windows and GPT-3 cannot easily be modified by anyone are different, but the result is the same: you're stuck with what you're sold. To clarify: MS windows is closed source, but you can't very well train a large GPT model unless you're someone with the resources of OpenAI. So you're stuck with whatever they choose to train and make available to you.
The API allows you to fine-tune existing models on your own dataset [1]
[1] Cf second paragraph of https://openai.com/blog/openai-api/
Re: OpenAI API
#155Earlier quoted context omitted.
The reasons why MS windows and GPT-3 cannot easily be modified by anyone are different, but the result is the same: you're stuck with what you're sold. To clarify: MS windows is closed source, but you can't very well train a large GPT model unless you're someone with the resources of OpenAI. So you're stuck with whatever they choose to train and make available to you.
The API allows you to fine-tune existing models on your own dataset [1] [1] Cf second paragraph of https://openai.com/blog/openai-api/
"fine tuning", i.e. transfer learning is still limited by the training of the original model.
For instance, if the original model is trained on English text exclusively and you want to fine-tune it on Greek text you are S.O.L.
Re: OpenAI API
#156Earlier quoted context omitted.
Nobody is putting these huge models in production, even the smaller transformer models are still too expensive to run for most use cases. With the way the field is moving, GPT-3 will be old news in a month, when more advances are made and open sourced.
i don't understand. if they run it for you and you apply transfer learning and fine tuning on your specific use case that would reduce drastically the costs hence why their offer make sense
[deleted]
Re: OpenAI API
#157Earlier quoted context omitted.
I think this is generally a good answer, but keep in mind I said AGI "in text". My forecasting is that within 3 years you will be able to give arbitrary text commands and get the textual output of the equivalents of "clean my house, take care of my kids, ..." like problems. I also would contend that there is reasoning happening and that zero-shot demonstrates this. Specifically, reasoning about the intent of the prom…
Pattern recognition and matching isn’t the same thing as reasoning. Zero shot demonstrates reasoning as much as solving the quadratic equation for a new set of variables does; it’s simply the ability to create new decision boundaries leveraging the same set of classifying power and methodology. True agi isn’t bound to a medium — no one would say Helen Keller wasn’t intelligent for example. I look forward to this ping…
What exactly is the difference between pattern matching and reasoning?
Re: OpenAI API
#158Earlier quoted context omitted.
More fake news and generated AI content there is more people would stop trusting social media. It will saturate to that tipping point that humanity will need to find more genuine ways to communicate. So I say bring it on.
Yeah bring the noise floor up so high that signal is lost. Burn all social media to the ground, I say.
You replied within an hour and now Im seeing it in 20 hours.
We exchange short texts, we both dont have a context, we dont know what each of us feels at that moment and we have absolutely no feedback about each other's mental states.
THIS isnt working! The conversational part of the social media is out of sync with reality and timespan. Evolution never had to optimize for this b/c there was never a need for it.
We struggle to understand, we just throw words at each other in passing and fill in the dots in our own minds - which is terrible b/c those dots are too far apart.
Signal is a needle in a haystack. Its not worth trying to keep fixing and reshaping the haystack so we dont keep loosing the needle. Lets just admit this tool isnt working and move on to better alternatives.
Edit: Clarifications (Need for post-editing also supports my point btw. )
Re: OpenAI API
#159Whoa -- Speech to bash commands? That's a pretty novel idea to me with my limited awareness of NLP. I could see this same idea in a lot of technical applications -- Provisioning cloud infrastructure, creating a database query.. Very cool!
It is not a novel idea and I don't think it is practical. If the natural language was practical for bash we would already have already "list directory" instead of "ls" and so on. "ls" is just 3 keystrokes while the natural language option is 15, 5 times more.
It could be good for programmers who can't type and have to use voice. I couldn't use my hands much for a couple months, and the state of the art for voice access.. leaves something to be desired. If I lose function for longer in the future I'd need something like that.
Re: OpenAI API
#160Earlier quoted context omitted.
Zero shot and few-shot learning in GPT-3 and lack of significant diminishing returns in scaling text models. Zero-shot learning is equivalent to saying "i'm just going to ask the model something that it was not trained to do"
And how do we get from zero shot to AGI? You're making gigantic leaps here.
For those who are wondering about reasoning behind this being the path to full AGI I recommend this Gwern post that goes into detail: https://www.gwern.net/newsletter/2020/05
From what I understand, its not just that the GPT-3 has impressive performance but more what is signifies and that is the fact that massive scaling didn't produce diminishing return, and if this pattern persists, it can get them to the finish line.