Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI API

beta.openai.com

Post reply on HN
Print view View original

91–100 of 165 posts

Re: OpenAI API

#91

Post by FeepingCreature »

Barrin92 wrote: post #88
dmvaldman wrote: post #79

Earlier quoted context omitted.

i think you are assuming that what is happening under the hood is that a human-inputted sentence is being parsed into a grammar. it is not.

I know that it isn't. That's part of the problem. There is no attempt to generate some sort of structure that can be interpreted semantically and reasoned about by the model. The model just operates on the input superficially and statistically. That's why there has been virtually no progress on trivial tasks such as answering: "I took the water bottle out of the backpack so that it would be [lighter/handy]" What is l…

Have you tried feeding this to GPT and seeing if it continues it in a way that reveals understanding?

It sounds like you're saying "It doesn't work because it can't work", but you haven't actually shown that it doesn't work.

Re: OpenAI API

#92

Post by pphysch »

mrfusion wrote: post #81
eggsnbacon1 wrote: post #65

Earlier quoted context omitted.

They redefined the org from non-profit to "capped profit", whatever that means. They're directly selling GPT 3 even though they originally said they wouldn't release it because of potential bad uses. They paid MS a ton of money for hardware and got a huge equity investment from them. And lets be honest here, the easiest and most straight-forward use of GPT3 is generating spam and low quality clickbait. Its the only u…

Couldn’t you generate fake support for issues on social media with this?

Already being done on an industrial scale, though this is further progress.

Re: OpenAI API

#93

Post by eggsnbacon1 »

sailingparrot wrote: post #73
eggsnbacon1 wrote: post #47

OpenAI started as a non-profit, went for-profit. Still owned by the big players.... Something isn't right. Is OpenAI just a submarine so the tech giants can do unethical research without taking blame??? Its textbook misdirection, nonprofit and "Open" in the name, hero-esque mission statement. How do you make the mental leap from "we're non-profit and we won't release things too dangerous" to "JK we're for-profit and…

> so the tech giants can do unethical research I know it's trendy (and partly justified) to look down on OpenAI, but can you actually give any basis for this claim? What kind of research is OpenAI doing that all the other big AI players (Google/DeepMind, FB, Microsoft) aren't also invested in? And even if others are doing the same, what part of OpenAI's research do you consider unethical? > It may just be a conspirac…

> What kind of research is OpenAI doing that all the other big AI players (Google/DeepMind, FB, Microsoft) aren't also invested in? And even if others are doing the same, what part of OpenAI's research do you consider unethical?

I believe all of them are doing unethical research, especially facial recognition. Notice the public backpedaling this week from all the big tech companies on this too. By directing their cash through OpenAI they can avoid whatever fallout comes from unleashing things like GPT3 on the world.

The most straightforward use case for GPT3 is generating fake but believable text. AKA spam. That's what it was designed to do. If you think fake news is a problem now, wait till someone is generating a dozen fake but believable news articles per minute by seeding GPT3 with a few words and hitting a button.

Its a conspiracy theory with some circumstantial evidence. We will probably never know either way, because who would admit to it if it was true.

Re: OpenAI API

#94

Post by denster »

wildermuthn wrote: post #36

In one of their examples, they note “They saw ratings hover around 60% with their original, in-house tech — this improved by 7-8% with GPT-2 — and is now in the 80-90% range with the API.” Bloomberg reports the API is based on GPT-3 and “other language models”. If that’s true, this is a big deal, and it epitomizes OpenAI’s namesake. The largest NLP models require vast corporate resources to train, let alone put into…

Agreed on the democratization front!

We saw this OpenAI demo: https://player.vimeo.com/video/427943452

and were just blown away. Very cool!!

I guess a spreadsheet is never too old [1] to learn new tricks :)

[1] Founder of https://mintdata.com here, so a bit biased (& opinionated about) spreadsheets, take the above with a pound or 10 of salt.

[2] I've sent them this example how we'd invoke their APIs, hopefully they'll let us into the beta, fingers crossed :) https://mintdata.com/docs/learn/core-mechanics/work-with-dat...

Re: OpenAI API

#95

Post by Barrin92 »

FeepingCreature wrote: post #91
Barrin92 wrote: post #88

Earlier quoted context omitted.

I know that it isn't. That's part of the problem. There is no attempt to generate some sort of structure that can be interpreted semantically and reasoned about by the model. The model just operates on the input superficially and statistically. That's why there has been virtually no progress on trivial tasks such as answering: "I took the water bottle out of the backpack so that it would be [lighter/handy]" What is l…

Have you tried feeding this to GPT and seeing if it continues it in a way that reveals understanding? It sounds like you're saying "It doesn't work because it can't work", but you haven't actually shown that it doesn't work.

yes, I have. You can paste these into the website of the Allen Institute for AI, yourself here. (https://demo.allennlp.org/reading-comprehension/MjE1MzE1Mg==)

In the example above it guesses wrongly, but again this is not surprising because it can't possibly get the right answer (other than by chance). The solution here cannot be found by correlating syntax, you can only answer the question if you understand the meaning of the sentence. That's what these schemas are constructed for.

Re: OpenAI API

#96

Post by m_ke »

dang wrote: post #87
m_ke wrote: post #4

I guess Sama plans on manufacturing growth metrics by forcing YC companies to pretend that they're using this. Generic machine learning APIs are a shitty business to get into unless you plan on hiring a huge sales team and selling to dinosaurs or doing a ton of custom consulting work, which doesn't scale the way VCs like it to. Anybody who will have enough know how to use their API properly can jus grab an open sourc…

> I guess Sama plans on manufacturing growth metrics by forcing YC companies to pretend that they're using this. That's wrong in almost too many ways to list. Sam left YC over a year ago, nor would he do such a thing. Nor does YC have that kind of power over companies, nor would it use it that way if it did. That would be wrong and also dumb.

Sorry, that was supposed to be sarcastic. What I meant to say is that Sam has a huge network and is a phone call away from pitching any CEO in the valley. One of the biggest benefits of YC these days is the huge network of companies in your portfolio, which makes getting intros and pilots a lot easier, leading to "traction" and more VC dollars.

Re: OpenAI API

#97

Post by grizzlemeelmo »

wildermuthn wrote: post #36

In one of their examples, they note “They saw ratings hover around 60% with their original, in-house tech — this improved by 7-8% with GPT-2 — and is now in the 80-90% range with the API.” Bloomberg reports the API is based on GPT-3 and “other language models”. If that’s true, this is a big deal, and it epitomizes OpenAI’s namesake. The largest NLP models require vast corporate resources to train, let alone put into…

Here's the Bloomberg story https://www.bloomberg.com/news/articles/2020-06-11/trillions...

Re: OpenAI API

#98

Post by sailingparrot »

eggsnbacon1 wrote: post #93
sailingparrot wrote: post #73

Earlier quoted context omitted.

> so the tech giants can do unethical research I know it's trendy (and partly justified) to look down on OpenAI, but can you actually give any basis for this claim? What kind of research is OpenAI doing that all the other big AI players (Google/DeepMind, FB, Microsoft) aren't also invested in? And even if others are doing the same, what part of OpenAI's research do you consider unethical? > It may just be a conspirac…

> What kind of research is OpenAI doing that all the other big AI players (Google/DeepMind, FB, Microsoft) aren't also invested in? And even if others are doing the same, what part of OpenAI's research do you consider unethical? I believe all of them are doing unethical research, especially facial recognition. Notice the public backpedaling this week from all the big tech companies on this too. By directing their cas…

> I believe all of them are doing unethical research, especially facial recognition.

Yes all of them are doing facial recognition research, except... OpenAI, so how exactly is OpenAI used as a scapegoat to be able to do that kind of research without public backlash?

> By directing their cash through OpenAI they can avoid whatever fallout comes from unleashing things like GPT3 on the world.

GPT-3 si not unethical research. It is what you decide to with it and how you decide to release it that can potientially be unethical.

Also, OpenAI is just ahead of other labs because they have an insane compute budget and really talented people, but if you have been following a little bit the NLP news, you will see that your theory of OpenAI being a front for unethical research just makes no sense. OpenAI release GPT-2, 1.5B billions parameters, then NVIDIA realeased Megatron, 8B parameters, Google released T5 at 11B and recently Microsoft did turing-nlg at 17B. So they are clearly working on this in their own names and very much publicizing their work.

Re: OpenAI API

#99

Post by [deleted] »

madcowd wrote: post #86
azinman2 wrote: post #60

Earlier quoted context omitted.

It's only Open™️ if I can run the API on my own machines.

If big LM's are the future then even if you had the model you couldn't run it on your own machines without having a DGX or two laying around.

[deleted]

Re: OpenAI API

#100

Post by dmvaldman »

azinman2 wrote: post #89
dmvaldman wrote: post #82

Earlier quoted context omitted.

what is the difference between zero-shot learning in text and AGI? not saying there isn't one, but can you state what it is?you can express any intent in text (unlike other media). to solve zero-shot in text is equivalent to the model responding to all intents. many people have different definitions for AGI though. for me it clicked when i realized that text has this universality property of capturing any intent.

Zero-shot learning is a way of essentially building classifiers. There's no reasoning, there's no planning, there's no commonsense knowledge (not in a comprehensive, deep way that we would look for it call it that), and there's no integration of these skills to solve common goals. You can't take GPT and say ok turn that into a robot that can clean my house, take care of my kids, cook dinner, and then be a great dinne…

I think this is generally a good answer, but keep in mind I said AGI "in text". My forecasting is that within 3 years you will be able to give arbitrary text commands and get the textual output of the equivalents of "clean my house, take care of my kids, ..." like problems.

I also would contend that there is reasoning happening and that zero-shot demonstrates this. Specifically, reasoning about the intent of the prompt. The fact that you get this simply by building a general-purpose text model is a surprise to me.

Something I haven't seen yet is a model simulate the mind of the questioner, the way humans do, over time (minutes, days, years).

In 3 years, I'll ping you :) Already made a calendar reminder

Post reply on HN