Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT-4 Outperforms Elite Crowdworkers, Saving Researchers $500k and 20k hours

artisana.ai

Post reply on HN
Print view View original

81–90 of 151 posts

Re: GPT-4 Outperforms Elite Crowdworkers, Saving Researchers $500k and 20k hours

#81

Post by dragonwriter »

SoftTalker wrote: post #69
dragonwriter wrote: post #30

Earlier quoted context omitted.

Shotspotter is a thing and it already has been doing that for years (both on its own and on law enforcement request.)

Shotspotter isn't an AI though is it? I thought it was just triangulation of gunshot locations using microphones and synchronized clocks?

> Shotspotter isn’t an AI though is it?

Shotspotter has been billed as a “system of sensors, software, AI and expert human review that accurately detects, locates and alerts police to gunfire”, and the company behind it (formerly “Shotspotter” was the company name, its recently been renamed “Soundthinking”) has a number of other AI-involved law enforcement products now, as well.

Re: GPT-4 Outperforms Elite Crowdworkers, Saving Researchers $500k and 20k hours

#82

Post by kerkeslager »

fshbbdssbbgdd wrote: post #52
troops_h8r wrote: post #7

I don't think I see enough discussion about what this means for privacy. There was some protection in the fact that it was prohibitively expensive to get someone to listen to every single one of our phonecalls/read all our emails/etc. Worrying that this will no longer be the case.

With regard to privacy, what’s the difference between your email’s text stored on a server, and your email’s text alongside the output of the text processed through a LLM? If “they” can already look at the text, what more privacy is there to lose?

There's a great deal of privacy in simply being a needle in a haystack. Part of the processing that's possible with an LLM is filtering.

Imagine you've sent an email about transporting a friend's daughter across state lines to get a medically-necessary abortion. Or if you prefer, imagine you've arranged via email to "lose" some firearms which don't comply with your state's new assault weapons ban.

Pre-LLMs, trying to find these sorts of emails was very hard. A simple text search for "abortion" or "gun" is going to come up with far more emails where two family members got into a political debate, than emails about lawbreaking. Big Brother will find a few such emails here and there by chance, but the vast majority of such incriminating emails will simply be lost in the pile.

Enter LLMs, and Big Brother can feed some of the incriminating emails found my chance into a training dataset along with a bunch of non-incriminating emails, and teach the AI to find incriminating emails, and then apply the model to the entire list of emails and get a nicely filtered list of only the emails which are incriminating, further tuning the model by adding emails it gets wrong to the training dataset when they are found.

Re: GPT-4 Outperforms Elite Crowdworkers, Saving Researchers $500k and 20k hours

#83

Post by wefarrell »

alphanumeric0 wrote: post #18
troops_h8r wrote: post #7

I don't think I see enough discussion about what this means for privacy. There was some protection in the fact that it was prohibitively expensive to get someone to listen to every single one of our phonecalls/read all our emails/etc. Worrying that this will no longer be the case.

Why do you think there was a mass surveillance of American domestic communications since forever ago, as leaked by Snowden? This technology has been available since then and can effectively summarize millions of pieces of communication.

I think that was metadata and not actual audio of conversations.

Re: GPT-4 Outperforms Elite Crowdworkers, Saving Researchers $500k and 20k hours

#84

Post by 876978095789789 »

Nifty3929 wrote: post #74
876978095789789 wrote: post #50

Great to see this tech and the money invested in it being used to take low-paying jobs away from people with limited options, instead of something like drug discovery or cancer biology.

This assumes those people really do have no other way to contribute. I don’t believe that’s the case. Do you? I believe people can contribute in many different ways. When technology enables us to get my work output without me, that frees me up to produce other things for society.

The issue is not can people contribute in some other way, but can they convince someone else to pay them a living wage for doing so, which is going to prove progressively more difficult as this technology advances.

Re: GPT-4 Outperforms Elite Crowdworkers, Saving Researchers $500k and 20k hours

#85

Post by ilaksh »

ftxbro wrote: post #79
nr2x wrote: post #66

Earlier quoted context omitted.

Yes, but GPT-5 will be better and the humans won’t. It’s very troubling.

I agree, but there are questions about GPT-N successors. Surprisingly (to me) many people think that GPT-N will never exceed human level intelligence because it was trained on the internet. I think that argument is obviously wrong. Another is that I am sure a large chunk of people will never concede that the AI is smarter than them. Literally never, no matter how smart the bot gets. I mean, probably a lot of people t…

GPT4 is already better than most if not all humans in some metrics.

I think it's a hard sell to say that nothing better than human is allowed.

But the existential concern comes from supposed exponential takeoff. To me it should be easy to convince people that something 10 or 100 times faster or smarter than humans should not be allowed.

Weirdly you don't see people talking about regulating autonomy which is also part of it.

I find the argument that Eliezer Yudkowsky makes with the super slow aliens to be very compelling especially in the context of fully autonomous AI that people have stupidly designed to imitate human (animal) characteristics like survival instincts. I suspect that that regulators will ban extremely high performing AI. But unfortunately the prediction is that this can't be contained which means it's quite probable that militaries will cause the end of the world just like we have been expecting but in a new way. Since they will likely be excluded from the ban at least secretly.

Re: GPT-4 Outperforms Elite Crowdworkers, Saving Researchers $500k and 20k hours

#86

Post by bequanna »

hnaouesteuho wrote: post #58

From reading the paper, GPT-4 also outperformed the researchers themselves in many categories, despite the researchers being the ones who created the dataset being used to perform the comparison. In other words, the metrics are biased in the researchers’ favor — so GPT-4 would have beat them even more often (probably a majority of the time based on the numbers), if someone else had created the guidelines and golden l…

With the current unskilled labor shortage driving wage increases which pushes inflation up, this seems to be arriving just in the time.

Re: GPT-4 Outperforms Elite Crowdworkers, Saving Researchers $500k and 20k hours

#87

Post by mztwo »

m3kw9 wrote: post #65

“ Employing Surge AI's top-tier human annotators at a rate of $25 per hour would have cost $500,000 for 20,000 hours of work”. That’s a wrap for Surge AI

Lots of immediate business from companies needing humans to spin up their models though... but as LLMs get more advanced it's anyone's guess what will happen here.

Re: GPT-4 Outperforms Elite Crowdworkers, Saving Researchers $500k and 20k hours

#88

Post by somsak2 »

Nifty3929 wrote: post #74
876978095789789 wrote: post #50

Great to see this tech and the money invested in it being used to take low-paying jobs away from people with limited options, instead of something like drug discovery or cancer biology.

This assumes those people really do have no other way to contribute. I don’t believe that’s the case. Do you? I believe people can contribute in many different ways. When technology enables us to get my work output without me, that frees me up to produce other things for society.

like what?

Re: GPT-4 Outperforms Elite Crowdworkers, Saving Researchers $500k and 20k hours

#89

Post by Teever »

Roritharr wrote: post #20
Teever wrote: post #16

Earlier quoted context omitted.

Yeah, I think the NSA is going to get their money's worth for that Utah Datacenter that they started building like 20 years ago.

Looking at this from far away, with the Snowden revelations in mind I'd think it's not tinfoil hat territory to assume that some of the progress at OpenAI got achieved with some help from well ressourced folks in the USG/Three Letter Agencies.

Yeah, I've started wondering similar things about that too, like how far ahead is the NSA on this stuff? And how does that tie in to the recent policy of denying China semiconductors?

Perhaps history will show that the NSA made algorithmic breakthroughs a few years ago and realized what was coming, so political policy was crafted to stymie Chinese progress in this field, and what we're seeing in the public sphere from companies like openAI is a managed release of the technology into the public, openAI at least managing to independently discover the same breakthroughs that the NSA made a a few years ago.

Re: GPT-4 Outperforms Elite Crowdworkers, Saving Researchers $500k and 20k hours

#90

Post by roywiggins »

talesvin wrote: post #26
SketchySeaBeast wrote: post #19

Earlier quoted context omitted.

Man, I can't wait for the AI to start hallucinating crimes.

I mean, if we get to the point AI is pointing the finger at someone i hope that a human will double check it at least.

Turns out having humans in the loop doesn't help that much:

https://www.cnn.com/2021/04/29/tech/nijeer-parks-facial-reco...

Post reply on HN