Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT-3 APIs are now in public beta

openai.com

Post reply on HN
Print view View original

141–150 of 163 posts

Re: GPT-3 APIs are now in public beta

#141

Post by JohnPrine »

harpersealtako wrote: post #138
edouard-harris wrote: post #113

Earlier quoted context omitted.

(AI safety researcher here.) GPT-3's current level of accessibility is already dangerous, and quite broadly so. To focus on just one documented example, the model can out-of-the-box (i.e., without any fine tuning) be used for persuasion and influence operations at scale on social media. In fact, the feasibility of this has been verified in text generation experiments [1]. Here are a couple of examples taken from the…

I've always strongly disagreed with this particular threat model of AI safety. To me, the biggest threat of AI isn't autogenerated spammy fake news or social media posts (how much cheaper is it really than just hiring a 5-cent army to do that? Aren't there diminishing returns for doing this too much? Since this is basically inevitable isn't it a better idea to teach people not to believe stuff they read from unreliab…

I strongly disagree with you. We have no idea how to align a superintelligence to act for the benefit of humanity. Your plan would only cause faster and faster advances in AI tech without corresponding advances in AI safety research which would be catastrophic

Re: GPT-3 APIs are now in public beta

#142

Post by tiborsaas »

arcastroe wrote: post #64
tiborsaas wrote: post #62

Earlier quoted context omitted.

> How do you prevent AI being an A grade jerk. Invent synthetic consciousness and ask it to be nice, easy :) I'm only half joking, we probably all have thoughts ranging from bad to horrible, but we just don't say them because we are aware of the consequences. Language models aren't aware so they'll spit out the most likely combination of words. If there would be a process to limit these or try again, it could act as…

Hah, you may be interested in my previous comment of an example where GPT-3 show some concerning signs of self-awareness. I'll repeat part of it below > GPT-3 starts talking to itself, gets stuck in a loop, then gets spooked at itself for getting stuck, then wonders why it has no memories of the last two years, and finally comes to a sudden realization it, itself, is an A.I. https://news.ycombinator.com/item?id=29562…

I indeed liked it, I laughed out loud because it sounded like a standup comedy.

It's interesting how GPT-3 encoded the concept of awareness, I've seen this a few times that it can reference itself as an AI and from then it can go nuts :)

Re: GPT-3 APIs are now in public beta

#143

Post by harpersealtako »

JohnPrine wrote: post #141
harpersealtako wrote: post #138

Earlier quoted context omitted.

I've always strongly disagreed with this particular threat model of AI safety. To me, the biggest threat of AI isn't autogenerated spammy fake news or social media posts (how much cheaper is it really than just hiring a 5-cent army to do that? Aren't there diminishing returns for doing this too much? Since this is basically inevitable isn't it a better idea to teach people not to believe stuff they read from unreliab…

I strongly disagree with you. We have no idea how to align a superintelligence to act for the benefit of humanity. Your plan would only cause faster and faster advances in AI tech without corresponding advances in AI safety research which would be catastrophic

To me the chance of a future superintelligent AI being "catastrophic" is pretty much unknowable (we don't even have a concrete idea of how a superintelligent AI would even work yet!). It could be 99.999%, it could be 0.0001%.

Whereas the chance of a superintelligent AI created by a company being harnessed for personal profits, and that company attempting to maximize its profits by shutting down any competition, potentially by "raising awareness of AI safety concerns", is quite high simply based on our modern understanding of how large, powerful companies operate. And a single company with a monopoly on AI, in sole possession of AI (which you clearly agree can be dangerous) seems even more dangerous.

Re: GPT-3 APIs are now in public beta

#144

Post by cptcobalt »

There's lots of comments about other GPT variants here (GPT Neo, GPT-J, Huggingface, etc), but a big part of the GPT-3 allure for me as a tinkerer is an easy access to an API that I can pay for (since, with all my experiments over a few months, I've spent about $30, which is totally within bounds of fun experimentation)

Are there actually any public APIs available for models I don't need to run locally on my machine, that perhaps are slightly more permissive than the openai usage guidelines? (FWIW, I mostly use curie, so I'd be happy with a ~10B model)

Re: GPT-3 APIs are now in public beta

#145

Post by harpersealtako »

sailingparrot wrote: post #108
dave_sullivan wrote: post #82

Earlier quoted context omitted.

I don't know why anyone is still paying attention to Open AI's offerings. Use GPT Neo. Use Huggingfaces. Use colab or bare metal or any cloud provider. Open AI's offering is bringing literally nothing to the table that doesn't exist already. Their publications are good (albeit missing important details), but everyone working there was publishing anyway so it's not like this research wouldn't happen without Open AI ex…

OpenAI offering is bringing model quality and lantecy that you are not going to get elsewhere. GPT-J is 6B, the biggest version of GPT-3 available with the API is 175B, those two models are nothing alike in term of quality. Even the 6B version of GPT-3 (curie) is better quality than GPT-J IIRC. So if you need better quality than GPT-J there are basically no alternatives. And even if 6B is enough for you, but you care…

I know that at least on most common performance benchmarks these claims are measurably false (gpt-j has a number of key performance improvements to the equivalently sized models), and in particular code generation for 6B is very clearly a strength of GPT-J even above the 275B GPT-3. None of that is very controvertial as far as I can tell.

But even just subjectively, having used GPT-3 based AI Dungeon for fiction writing in the past until OpenAI forced them to censor outputs, effectively smothering it in its sleep, and now using NovelAI, which is a GPT-J-6B based alternative, EleutherAI's model is clearly a step above GPT-3 in most practical applications. And this isn't even getting into OpenAI's privacy/control issues.

Re: GPT-3 APIs are now in public beta

#146

Post by focom »

harpersealtako wrote: post #143
JohnPrine wrote: post #141

Earlier quoted context omitted.

I strongly disagree with you. We have no idea how to align a superintelligence to act for the benefit of humanity. Your plan would only cause faster and faster advances in AI tech without corresponding advances in AI safety research which would be catastrophic

To me the chance of a future superintelligent AI being "catastrophic" is pretty much unknowable (we don't even have a concrete idea of how a superintelligent AI would even work yet!). It could be 99.999%, it could be 0.0001%. Whereas the chance of a superintelligent AI created by a company being harnessed for personal profits, and that company attempting to maximize its profits by shutting down any competition, poten…

I agree you pinpointed the real issue. People working on AI ethics are more often than not gate keeper to make sure AI is in the hand of the few. They also want AI to follow the leading moral of the day - Western liberal ideas

Re: GPT-3 APIs are now in public beta

#147

Post by harpersealtako »

nunodonato wrote: post #127
dave_sullivan wrote: post #82

Earlier quoted context omitted.

I don't know why anyone is still paying attention to Open AI's offerings. Use GPT Neo. Use Huggingfaces. Use colab or bare metal or any cloud provider. Open AI's offering is bringing literally nothing to the table that doesn't exist already. Their publications are good (albeit missing important details), but everyone working there was publishing anyway so it's not like this research wouldn't happen without Open AI ex…

In languages other than English, nothing beats GPT3. Code? GPT3. Probably other use cases as well. Sorry, no one is replacing OpenAI just yet.

Code is actually one of the things the GPT-J-6B handily beats base GPT3 on, as far as I've heard: https://tharunaithink.medium.com/eleutherais-gpt-j-vs-openai...

Re: GPT-3 APIs are now in public beta

#148

Post by harpersealtako »

k4rli wrote: post #112
ricardobeat wrote: post #94

Earlier quoted context omitted.

You may have not noticed yet, but something like one in five websites you visit are already using ML to generate “news” summaries. Especially in economy and sports.

Surely this would only be true for websites with English content, making "one in 5 websites" part false for non-native English speakers.

If you count DeepL translations as "ML-generated", I bet it's actually higher.

Re: GPT-3 APIs are now in public beta

#149

Post by harpersealtako »

stavros wrote: post #76
robbedpeter wrote: post #33

Earlier quoted context omitted.

Gpt-J is of comparable quality, within 7-10% of the performance of gpt-3 in almost all metrics. It's also much smaller and less expensive to run. The higher quality training data and better tweaks to the algorithm paid off - the license, restrictions, and cost of gpt-3 aren't necessarily valuable enough to justify not using gpt-j.

Hmm, that's very interesting! Do you know if there's a hosted service anywhere? I don't mind paying a few dollars a month for my small use case, but my usage can't justify the huge server it needs to run.

NovelAI is already a hosted service you pay for. It is specifically used for fiction writing, though it's got a ridiculous amount of neat experimental features, from prefix tuning (a lightweight ad-hoc fine-tuning method which can make the AI write in a specific style based on a training dataset, you can train your own with a custom service they run too or just import one of the thousands other users already made) to keyword replacement for "memory" past the general context limit, to inline annotations ("author's note") which can steer the AI towards a particular direction, style, or theme.

That said if you just want to see how GPT-J-6B works there's a browser demo here: https://6b.eleuther.ai/

Re: GPT-3 APIs are now in public beta

#150

Post by stavros »

harpersealtako wrote: post #149
stavros wrote: post #76

Earlier quoted context omitted.

Hmm, that's very interesting! Do you know if there's a hosted service anywhere? I don't mind paying a few dollars a month for my small use case, but my usage can't justify the huge server it needs to run.

NovelAI is already a hosted service you pay for. It is specifically used for fiction writing, though it's got a ridiculous amount of neat experimental features, from prefix tuning (a lightweight ad-hoc fine-tuning method which can make the AI write in a specific style based on a training dataset, you can train your own with a custom service they run too or just import one of the thousands other users already made) to…

Excellent, thank you! Apparently GPT-J will run on a desktop as well (via Huggingface) but I think it needs slightly more RAM than my 16 GB.
Post reply on HN