Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The makers of Eleuther hope it will be an open source alternative to GPT-3

wired.com

Post reply on HN
Print view View original

71–80 of 100 posts

Re: The makers of Eleuther hope it will be an open source alternative to GPT-3

#71

Post by I_Byte »

nutanc wrote: post #46

I have recently started an experiment to generate an AI generated newsletter[1]. All posts are generated by GPT-3. I work as the editor. It works well for some topics and not so well for some topics. Since I curate the content, I dont publish topics which are not done well. For example, I tried to make it generate a nice article on the Suez canal crises. But it was harder than I thought it would be. It generates buzz…

How do you go about generating these posts? I think I would like to play around with something like this but I am not sure where to start.

Re: The makers of Eleuther hope it will be an open source alternative to GPT-3

#72

Post by sodality2 »

worik wrote: post #64

I thought that the important barrier to building these sorts of systems is the cost of, indirectly the energy required for, training the model. Is that still correct? How does a Free Software or "Open Source" project get around that?

Distributing the trained models.

Re: The makers of Eleuther hope it will be an open source alternative to GPT-3

#74

Post by vmception »

spideymans wrote: post #4

It'll be interesting to see how colleges and universities react to GPT-3. Students will surely use it to write entire assignments.

Wolfram Alpha has been solving calculus problems for 12 years and it is barely a footnote in how the college experienced has changed

So I would say this likely will just be there. It just is. Wont change anything, universities will acknowledge it, a headline or two will occur when its use was discovered in a paper that a student didnt even skim to make less obvious, and most papers will fly under the radar.

Other kinds of assessments will still do their job.

Re: The makers of Eleuther hope it will be an open source alternative to GPT-3

#75

Post by nl »

whimsicalism wrote: post #62
bluetwo wrote: post #52

Earlier quoted context omitted.

Examples?

Looking back, I think freshman me was perhaps a bit harsh in my assessment of my peers. Here are two excerpts, one my own writing and one from a peer. Rereading them, I am not sure GPT3 could recreate either of these, but you can judge: Peer: > The Gaza Conflict Gave Hamas what they needed to build an even deeper anti US narrative and anti-israeli narrative. The reasons that Israel was able to act the way it did so d…

Here's some samples using the small GPT-2 (emphasis: small GPT-2!) in AiTextGen, seeded using "The Gaza Conflict Gave Hamas what they needed " (1st 3 samples, not cherry picked):

> The Gaza Conflict Gave Hamas what they needed to survive and that's not something Israel can afford.

> The Gaza Conflict Gave Hamas what they needed to take control of Gaza. In the past few days I have had a few conversations with people who have been in Gaza who are very critical of the IDF. These people are often very, very angry. They are also very angry about the way the IDF is doing what it is supposed to be doing and what it is supposed to be doing. So in my mind, the issue of Hamas being in control of Gaza is not something that should go away. It is something that needs to be addressed. Q: What have you been hearing about since the war started? A: The IDF has been saying that it is going to allow Hamas to take over Gaza. Q: What has been particularly troubling to you lately? A: I have been hearing about this as well. Q: What has been more troubling to you recently? A: My friends have been saying that they want to call it Operation Protective Edge. Q: And, you know, what do they mean by that? A: They mean that they want to use it to make sure that they have enough force to take control of Gaza. Q: And that means that they want to make sure that they have enough

> The Gaza Conflict Gave Hamas what they needed?"

The UN is now considering whether to send military advisers to Gaza to assist the country's military forces. The Security Council is considering whether to send the equipment, along with the training, to a military operation in the region. The US is also considering sending a team of US special forces to assist the Palestinian armed forces in the conflict.

The United Nations is now considering whether to send military advisers to Gaza to assist the country's military forces.

Kerry's comments come as the US has been in touch with the Palestinians to offer support in exchange for a full ceasefire, and as the US continues to support the PA and Hamas, the two groups have been engaged in a long-running conflict with Israel in the Gaza Strip.

In January, Kerry condemned Israel's "continued offensive against Gaza," saying the blockade was the "worst violation of international law on the part of the Israeli government and the civilian population of Gaza."

The US is now considering whether to send military advisers to Gaza to assist the country's military forces. According to Reuters, the US Secretary of State John Kerry said this week that "there is no guarantee" that the US will send special forces "to the Gaza Strip

So yeah - not fantastic, but interestingly not terrible either. The non-factual but coherent nature of it is very troubling.

Re: The makers of Eleuther hope it will be an open source alternative to GPT-3

#76

Post by caslon »

doesnotexist wrote: post #48

How many internet forum prophecy cults (you know like the q one) are or will be powered by these language models? It's often assumed or at least easier to imagine the evaluator in Turing's test is a rational actor that possesses a high-degree of skepticism. But it seems that a lot of the human population is ready and willing to believe wild claims with little or no evidence and many people seek out information that c…

This thought process is something I think is a common misconception with how cults work.

A machine to autogenerate cult-ish nonsense isn't needed. Humans are already incredibly good at doing this on their own.

Not only this, but another thing about this is that cults generally fine-tune themselves to fit their members.

A machine generating convincing lies still wouldn't meaningfully do as much as a human-operated, human-targeted attempt at a cult. Creating one is something basically any human can do; the required skillset is something most people possess.

Re: The makers of Eleuther hope it will be an open source alternative to GPT-3

#77

Post by pedalpete »

spideymans wrote: post #4

It'll be interesting to see how colleges and universities react to GPT-3. Students will surely use it to write entire assignments.

A friend started a AI to improve writing (https://outwrite.com) and when the initially started, they had a detect plagiarism feature that teachers could use, I think they stopped developing that eventually.

If I recall correctly, the way it worked was to build up a model of this persons writing, and how it compared to to other people, and then would measure the likelihood that sentences and paragraphs matched the rest of the writing.

I suspect something similar could be done with GPT-x

Re: The makers of Eleuther hope it will be an open source alternative to GPT-3

#78

Post by worik »

sodality2 wrote: post #72
worik wrote: post #64

I thought that the important barrier to building these sorts of systems is the cost of, indirectly the energy required for, training the model. Is that still correct? How does a Free Software or "Open Source" project get around that?

Distributing the trained models.

I should have read the article more carefully!!

The Eleuther project makes use of distributed computing resources, donated by cloud company CoreWeave as well as Google, through the TensorFlow Research Cloud, an initiative that makes spare computer power available, according to members of the project

Re: The makers of Eleuther hope it will be an open source alternative to GPT-3

#79

Post by nl »

6gvONxR4sf7o wrote: post #65
stellaathena wrote: post #56

Earlier quoted context omitted.

Hi! I’m the EAI person who your criticism of the Pile is most directed at. I’m curious if you read Sections 6.5 and 7 of the Pile working paper and, if so, what your response to it is. As you note, virtually everyone trains on copyright data and just ignores any implications of that fact. I feel that our paper is very upfront about this though, going as far as to have a table that explicitly lists which subsets conta…

Hi again! We had a back-and-forth about this a while back regarding the paper and I think we didn't end up on the same page regarding the "public data" definition in the paper (found it! [0]). I love that you're upfront in the paper, because it's silly how most people just don't acknowledge it (though they usually don't redistribute it publicly like the pile does). I think the gist was us disagreeing about the releva…

I don't have a dog in this fight, but I think you should re-read this: data which cannot be easily obtained but can be obtained, e.g. through a torrent or on the dark web.

It's an extra piece of engineering to reliably scrape torrents and the dark web and exclude spam traps. "Easily obtained" is probably as much about this vs the copyright aspects.

The person you are replying to is correct in saying that most people train on the "public web" (eg, common crawl data). The copyright implications of this haven't been tested in court as yet.

It is worth noting that common-crawl data is widely distributed and would seem to raise the same issues you are identifying here.

Re: The makers of Eleuther hope it will be an open source alternative to GPT-3

#80

Post by claudiawerner »

robotresearcher wrote: post #49
IgorPartola wrote: post #6

Earlier quoted context omitted.

What are you on about? There are AI driven chat bots. When they aren’t used it’s not because they might say something offensive. General AI is not here by any definition or redefinition of General AI. We are not fragile.

AI driven chat bots are routinely deployed, yes. But it's also true that at least one bot generated content that spooked its owner: "The AI chatbot Tay is a machine learning project, designed for human engagement. As it learns, some of its responses are inappropriate and indicative of the types of interactions some people are having with it. We're making some adjustments to Tay." (Microsoft statement) https://www.the…

This is because Twitter users, some coordinated on 4chan's /pol/ board, decided to train the bot on extreme racist input:

https://en.wikipedia.org/wiki/Tay_(bot)#Initial_release

Post reply on HN