Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The makers of Eleuther hope it will be an open source alternative to GPT-3

wired.com

Post reply on HN
Print view View original

81–90 of 100 posts

Re: The makers of Eleuther hope it will be an open source alternative to GPT-3

#81

Post by whimsicalism »

nl wrote: post #75
whimsicalism wrote: post #62

Earlier quoted context omitted.

Looking back, I think freshman me was perhaps a bit harsh in my assessment of my peers. Here are two excerpts, one my own writing and one from a peer. Rereading them, I am not sure GPT3 could recreate either of these, but you can judge: Peer: > The Gaza Conflict Gave Hamas what they needed to build an even deeper anti US narrative and anti-israeli narrative. The reasons that Israel was able to act the way it did so d…

Here's some samples using the small GPT-2 (emphasis: small GPT-2!) in AiTextGen, seeded using "The Gaza Conflict Gave Hamas what they needed " (1st 3 samples, not cherry picked): > The Gaza Conflict Gave Hamas what they needed to survive and that's not something Israel can afford. > The Gaza Conflict Gave Hamas what they needed to take control of Gaza. In the past few days I have had a few conversations with people w…

From reading these (esp. the last), you would think the US is allied with Palestine against Israel!

Re: The makers of Eleuther hope it will be an open source alternative to GPT-3

#82

Post by wongarsu »

mrkramer wrote: post #66

Couldn't Google build the the world's most powerful NLP AI? They scraped the whole web and have DeepMind to pull it off on top of Google's powerful and massive data centers.

They probably could, but what for?

They did develop BERT and use (used?) it for parsing search queries [1]. They probably use NLP models in the ranking algorithm too. But those use cases are about getting the best result possible within the throughput/latency requirements, which necessarily makes them less "powerful" than models like GPT that pay little attention to performance.

https://blog.google/products/search/search-language-understa...

Re: The makers of Eleuther hope it will be an open source alternative to GPT-3

#83

Post by mrkramer »

wongarsu wrote: post #82
mrkramer wrote: post #66

Couldn't Google build the the world's most powerful NLP AI? They scraped the whole web and have DeepMind to pull it off on top of Google's powerful and massive data centers.

They probably could, but what for? They did develop BERT and use (used?) it for parsing search queries [1]. They probably use NLP models in the ranking algorithm too. But those use cases are about getting the best result possible within the throughput/latency requirements, which necessarily makes them less "powerful" than models like GPT that pay little attention to performance. https://blog.google/products/search/se…

I forgot about BERT thx for reminding me.

Google for example can monetize NLP AI framework just like they are monetizing Kubernetes: https://cloud.google.com/kubernetes-engine

If OpenAI and Microsoft are licensing and monetizing GPT-3 why Google wouldn't want to compete with them when they have more data than them. Nothing can beat the amount of data that Google and Google Search have gathered over the last 20 years.

Re: The makers of Eleuther hope it will be an open source alternative to GPT-3

#84

Post by nutanc »

starik36 wrote: post #68
nutanc wrote: post #46

I have recently started an experiment to generate an AI generated newsletter[1]. All posts are generated by GPT-3. I work as the editor. It works well for some topics and not so well for some topics. Since I curate the content, I dont publish topics which are not done well. For example, I tried to make it generate a nice article on the Suez canal crises. But it was harder than I thought it would be. It generates buzz…

Are you using OpenAI API to generate these?

Yes.

Re: The makers of Eleuther hope it will be an open source alternative to GPT-3

#85

Post by mylons »

burlesona wrote: post #15
everdrive wrote: post #2

People already believe garbage at a pretty alarming rate. It's easy to guess at a number of possible outcomes here: - More junk text moves the public to doubt legitimate information even further than they currently do. - There is so much human-generated junk text that adding more of it via AI actually doesn't have much of an effect. - People return to lean on experts, perhaps even more than before. (just as a number…

I think we may come to see the era of roughly 1990-2010 as the golden age of information: relative abundance creating new opportunity, before the noise drowned it all out. I suspect that in the future people will, ironically, return more strictly to tribal knowledge, as the media and the internet will be (already is) a vast ocean from which you can pull anything you want to believe. Thus nothing you see or hear from…

neal stephenson has a cool prediction on how to handle this in one of his latest books: curators. people of means will have a curated view of the internet and information that costs $$$

Re: The makers of Eleuther hope it will be an open source alternative to GPT-3

#86

Post by vsareto »

Der_Einzige wrote: post #42
spideymans wrote: post #4

It'll be interesting to see how colleges and universities react to GPT-3. Students will surely use it to write entire assignments.

Anecdotally, but I know a number of people who do university assignments for money. Many of the clients for the folks I know are at the university level are folks with poorer then average English language skills and are usually in intro writing courses. I'd be terrified if I were one of them right now. GPT-3 would be a godsend for cheaters, but still requires a human to jump in and rewrite whole sections. No, if you…

I feel like we're overestimating the value of generating text and selectively picking out things that sound good. This is up for debate, but you'd probably do better to read actual authors and pick things up from them and maybe making your own changes. The time spent parsing the output to see if it's good could be spent on thinking about something to write?

Would be neat to try and publish books with a percentage of AI generated text and see how well they do. Maybe there's a sweet spot for productivity.

Re: The makers of Eleuther hope it will be an open source alternative to GPT-3

#87

Post by burlesona »

mylons wrote: post #85
burlesona wrote: post #15

Earlier quoted context omitted.

I think we may come to see the era of roughly 1990-2010 as the golden age of information: relative abundance creating new opportunity, before the noise drowned it all out. I suspect that in the future people will, ironically, return more strictly to tribal knowledge, as the media and the internet will be (already is) a vast ocean from which you can pull anything you want to believe. Thus nothing you see or hear from…

neal stephenson has a cool prediction on how to handle this in one of his latest books: curators. people of means will have a curated view of the internet and information that costs $$$

It’s already here. How many people are paying for newsletter subscriptions etc, and how quickly is that growing? :)

Re: The makers of Eleuther hope it will be an open source alternative to GPT-3

#88

Post by fapjacks »

rib3ye wrote: post #32
minimaxir wrote: post #14

As someone who works on a Python library solely devoted to making AI text generation more accessible to the normal person ( https://github.com/minimaxir/aitextgen ) I think the headline is misleading. Although the article focuses on the release of GPT-Neo, even GPT-2 released in 2019 was good at generating text, it just spat out a lot of garbage requiring curation, which GPT-3/GPT-Neo still requires albeit with a bet…

Totally off topic: can you fix the pip3 installer for aitextgen? I just filed an issue on GH issue tracker.

I don't know when it started (as it's been years since I wrote anything in Python) but the last couple of years I've been seeing way way more of these generic Python environment / configuration errors around the internet that are hard to diagnose and debug. Has something happened over the last few years with Python and its configuration and dependency management?

Re: The makers of Eleuther hope it will be an open source alternative to GPT-3

#89

Post by fapjacks »

mylons wrote: post #85
burlesona wrote: post #15

Earlier quoted context omitted.

I think we may come to see the era of roughly 1990-2010 as the golden age of information: relative abundance creating new opportunity, before the noise drowned it all out. I suspect that in the future people will, ironically, return more strictly to tribal knowledge, as the media and the internet will be (already is) a vast ocean from which you can pull anything you want to believe. Thus nothing you see or hear from…

neal stephenson has a cool prediction on how to handle this in one of his latest books: curators. people of means will have a curated view of the internet and information that costs $$$

In many ways, predicting the future is just about looking at how to apply past trends to our modern world. People and organizations with the resources to do so are actively working to cram their "curation" into the public sphere, to try and sway the masses. That's advertising. It's the same thing it was in the 20th century, just more personal and using more sinister means of hogging your attention span. Peoples' intuition about "advertising" is as a commercial enterprise, but I'm certain all the textbooks that talk about the 2010s and 2020s will mention the role of mass bot/shill campaigns to sway the public by governments, companies, organizations, individual people.

Re: The makers of Eleuther hope it will be an open source alternative to GPT-3

#90

Post by fapjacks »

whimsicalism wrote: post #29
everdrive wrote: post #24

Earlier quoted context omitted.

In the early-ish days of the consumer internet, consumers had a new and huge information advantage over companies. People moved from relying on brand name, to reading online reviews. Often finding niche brands which they had otherwise not heard of. Now, in 2021, that experience is flipped on its head. Amazon reviews are gamed and cannot be trusted. Companies build niche brands like fly-by-night companies, and the les…

I just rely on reviewers like NYT Wirecutter and then buy whatever the reviewers suggest (and is cheap) on Amazon.

Why do you imagine the NYT isn't taking a bigger paycheck for saying the same thing the fake Amazon reviews are saying? We live in a world where brands are trash. They're built by the lowest bidder with the absolute cheapest parts ("the Toyota Way", heavy airquotes), and packaged with lies (marketing). Where did this idea come from that it's only the foreign resellers of Chinese products that are posting fake Amazon reviews, when it is absolutely the same case with household brand name companies? Go to Fakespot and look up random brand name companies you would normally "trust" and see what how Fakespot rates the company. We are WAY past the point where the vast majority of companies (governments, organizations, etc) aren't using these types of mass propaganda campaigns. A brand name in the general case is no more trustworthy than some imitator.
Post reply on HN