Welcome to the end of the internet as we know it, where affiliate marketers and ad sites generate content to flood search engines with results, rendering it near impossible to find quality content.
GPT-3 APIs are now in public beta
101–110 of 163 posts
Re: GPT-3 APIs are now in public beta
#102Earlier quoted context omitted.
Yes, but they didn't release the model after training and you can't take your weights with you if you finetune their model. GPT Neo was trained at similar expense, and they released the weights. Use that.
First part is correct, the second part is not. GPT Neo is a 2.7B param model, the largest GPT is 175B (they have various flavours, up to 175B). I appreciate the sentiment and what ElutherAI is doing with GPT Neo, but there is no open source equivlenet of the full GPT-3 available for the public to use. Hopefuly it's just a matter of time.
GPT-J is 6B and comes pretty close. Also practically I haven’t noticed a difference.
Keep in mind there are also closed source alternatives: for example, AI21’s Jurassic-1 models are comparable, cheaper, and technically larger (albeit somewhat comically, 178B instead of 175B parameters).
Re: GPT-3 APIs are now in public beta
#103Earlier quoted context omitted.
Please see my other response: https://news.ycombinator.com/item?id=29576981 As a summary, generating content isn't new and just increases your moderation load. AI is not something "special". It's just software, at the end.
You are comparing apples and oranges. "generating content" does not yet say HOW the data is generated. Here we are specifically talking about probabilistic generative model, which are inherently unsuited for mission critical purposes, as they are engineered right now. I don't need to do this discussion, this is basic introductory education and consent in University courses about AI and especially about system enginee…
> I don't need to do this discussion
Then why are you on HN, replying to exactly that discussion?
We are not teenagers, I may be wrong, I would be okay with it.
> Here we are specifically talking about probabilistic generative model, which are inherently unsuited for mission critical purposes, as they are engineered right now
In what qualitative way does that differ from any kind of data generation?
Re: GPT-3 APIs are now in public beta
#104Earlier quoted context omitted.
Please see my other response: https://news.ycombinator.com/item?id=29576981 As a summary, generating content isn't new and just increases your moderation load. AI is not something "special". It's just software, at the end.
That’s a bit like saying machine guns aren’t an improvement over bows and arrows because they both throw projectiles, isn’t it?
I'm not talking about improvements.
You need gun control with bows and arrows, and you still need gun control with machine guns.
That's what I'm saying.
Re: GPT-3 APIs are now in public beta
#105Welcome to the end of the internet as we know it, where affiliate marketers and ad sites generate content to flood search engines with results, rendering it near impossible to find quality content.
Sadly you are probably right. Hidden affiliate marketers are the parasites of search results and of the internet in general
Re: GPT-3 APIs are now in public beta
#106Earlier quoted context omitted.
That’s a bit like saying machine guns aren’t an improvement over bows and arrows because they both throw projectiles, isn’t it?
I'm not talking about improvements. You need gun control with bows and arrows, and you still need gun control with machine guns. That's what I'm saying.
And what we're saying is stuff like GPT-3 may be improvements on the level of archery --> machine gunning when it comes to online trolling, misinformation, and the like.
Casually dismissing it with "well it's just extra moderation load" is a mistake, I think.
Re: GPT-3 APIs are now in public beta
#107This is cool but I'm hesitant to invest any real amount of time building something on this. How do I know that if I build something massively successful they wont shut off my access and build a competitor?
Considering OpenAI's notorious history of apparently pivoting 180 to Closed (see HN references ad nauseum ), you might be wise to be hesitant. This is perhaps an organisation with more fame than trust.
>OpenAI's notorious history of apparently pivoting 180 to Closed
would you mind providing discussion on HN about this?
Re: GPT-3 APIs are now in public beta
#108Looks neat, but note that OpenAI is basically an App Store model - whatever you make with it cannot be shown to more than five people unless OpenAI has reviewed and explicitly approved your project. And their usage guidelines are pretty narrow - "open-ended chatbots" are explicitly disallowed, for example. > https://beta.openai.com/docs/going-live > https://beta.openai.com/docs/usage-guidelines/use-case-guide...
I don't know why anyone is still paying attention to Open AI's offerings. Use GPT Neo. Use Huggingfaces. Use colab or bare metal or any cloud provider. Open AI's offering is bringing literally nothing to the table that doesn't exist already. Their publications are good (albeit missing important details), but everyone working there was publishing anyway so it's not like this research wouldn't happen without Open AI ex…
OpenAI offering is bringing model quality and lantecy that you are not going to get elsewhere.
GPT-J is 6B, the biggest version of GPT-3 available with the API is 175B, those two models are nothing alike in term of quality. Even the 6B version of GPT-3 (curie) is better quality than GPT-J IIRC.
So if you need better quality than GPT-J there are basically no alternatives.
And even if 6B is enough for you, but you care about latency, OpenAI has the best inference runtime by far, and you are not going to replicate that on your cloud/bare-metal. Unless your scenario specifically benefits from your API and your other servives to be colocated.
Edit: I forgot about finetuning. OpenAI gives you the ability to finetune all of their variants. Maybe you already have the knowledge to finetune something like GPT-J yourself, but I would guess that most potential users of the API do not have it.
Re: GPT-3 APIs are now in public beta
#109Earlier quoted context omitted.
Yea, this made me look to alternatives and I mainly use Huggingface instead. I don't want to wake up one day and learn that a side project got rejected by a higher power and I have to write a HN post to get my account unlocked.
What model do you use with Huggingface? Huggingface is just a wrapper.
Blenderbot and distillery and I'm looking into use others also.
GPT-3 has multiple models associated with it now so it has become a wrapper also.
Re: GPT-3 APIs are now in public beta
#110Earlier quoted context omitted.
I don't know why anyone is still paying attention to Open AI's offerings. Use GPT Neo. Use Huggingfaces. Use colab or bare metal or any cloud provider. Open AI's offering is bringing literally nothing to the table that doesn't exist already. Their publications are good (albeit missing important details), but everyone working there was publishing anyway so it's not like this research wouldn't happen without Open AI ex…
> "Oh, we can't share this knowledge because it's too dangerous" Every time I hear someone say this, I always think, "Dangerous... to whom ?"
I can imagine a effective scamming situation where you just use gpt to be an unhelpful support staff. It doesn't matter too much what it says, so long as it sounds vaguely like a human.
It adds artificial legitimacy to something that might be without it