Live data from Hacker News

ChatGPT generates fake data set to support scientific hypothesis

nature.com

101–110 of 159 posts

Re: ChatGPT generates fake data set to support scientific hypothesis

#101
post #22
post #16

>“It will make it very easy for any researcher or group of researchers to create fake measurements on non-existent patients, fake answers to questionnaires or to generate a large data set on animal experiments.” Perhaps I'm naive, but I think the people that want to fake data were already doing it without tools like chatgpt. Especially since a ton of biological data is normally distributed, so it's exceedingly easy t…

Exactly. How is this news? What would be surprising is if ChatGPT _could_ generate fake data that passed analysis.

[deleted]

Re: ChatGPT generates fake data set to support scientific hypothesis

#102
post #12

For anyone jumping in who hasn't read the article, this is not about hallucinations and some researchers being surprised that GPT-4 doesn't always respond with absolute truth. Instead it's about how easily it can be used to generate plausible looking datasets that would confirm a hypothesis. It's a warning note to journals about how fake data can more easily be created.

[deleted]

Re: ChatGPT generates fake data set to support scientific hypothesis

#103

It's awesome to know that the scientific process that's been in use for centuries, based on peer review and the importance of replication, is still so powerful that AI generated fake data can't do anything to undermine it. Reality doesn't care how the data was faked when you replicate an experiment! Now, if only scientists and institutions would actually bother using those tools we developed centuries ago. Unfortunat…

[deleted]

Re: ChatGPT generates fake data set to support scientific hypothesis

#105

It's only a matter of time until someone comes up with a GPT that takes whatever off-axis theories a research paper writer wishes to promulgate, and searches the entire corpus of academic literature for references that can be strung together in such a way as to support any argument one likes. A quack's dream come true, substantiating an argument by backsolving from its feeble or malevolent conclusion to a set of well…

Have you been watching American politics for the last 20 years? Tea Party to Q/Maga, they will cite any quack economist, quack scientist, or quack doctor that provides a theory that enforces their narrative, data be damned.

Re: ChatGPT generates fake data set to support scientific hypothesis

#106

Earlier quoted context omitted.

ChatGPT will unlock new levels of both good and bad. The question is what the ratio between the two is going to be.

If you think about it, it will make 99% of ChatGPT answer technically INCORRECT. It is harder to keep bad data out than it is to keep it filled with good data.

Reminds me of the “asymmetric bullshit principle”

Re: ChatGPT generates fake data set to support scientific hypothesis

#107
post #57

Earlier quoted context omitted.

So we’re pretending making something an order of magnitude easier makes no difference? Ok.

Are we pretending Faker hasn't been a staple library in software testing for years?

Are we pretending GPT isn’t a leagues better data-faker than Faker?

Re: ChatGPT generates fake data set to support scientific hypothesis

#108
post #57

Earlier quoted context omitted.

Are we pretending Faker hasn't been a staple library in software testing for years?

Are we pretending GPT isn’t a leagues better data-faker than Faker?

As always, it depends on your requirements.

GPT definitely wins out if you want more novelty/variety in your fake data and are willing to accept extraordinarily higher cost, less rigor, and less reliability. I'm sure there's some occasion when those criteria win out, but Faker's pretty decent most of the time.

Re: ChatGPT generates fake data set to support scientific hypothesis

#109
post #96
post #16

>“It will make it very easy for any researcher or group of researchers to create fake measurements on non-existent patients, fake answers to questionnaires or to generate a large data set on animal experiments.” Perhaps I'm naive, but I think the people that want to fake data were already doing it without tools like chatgpt. Especially since a ton of biological data is normally distributed, so it's exceedingly easy t…

Being able to do it at scale is an issue because of paper mills.

`np.random.normal(0, 1)`. You too can generate infinite amount of normal distribution data to use for fake datasets, free of AI.
Post reply on HN