Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT-4 Outperforms Elite Crowdworkers, Saving Researchers $500k and 20k hours

artisana.ai

Post reply on HN
Print view View original

61–70 of 151 posts

Re: GPT-4 Outperforms Elite Crowdworkers, Saving Researchers $500k and 20k hours

#61

Post by nr2x »

justinjlynn wrote: post #39
malux85 wrote: post #23

Earlier quoted context omitted.

Yeah but have you seen the leaked slides? It's clear that they have only the ability to analyse 10% or less of the data they are storing . GPT-like systems will close that gap, and then comes all of the problems of automated law enforcement - Extrapolation from incomplete data, false positives from coindicences, interpretation errors, all that annoying stuff

Just wait until they go back and have the bandwidth to finally analyse all the historical data they've accumulated...

I’m guessing they are, and have been, for a while.

Re: GPT-4 Outperforms Elite Crowdworkers, Saving Researchers $500k and 20k hours

#62

Post by adastra22 »

876978095789789 wrote: post #50

Great to see this tech and the money invested in it being used to take low-paying jobs away from people with limited options, instead of something like drug discovery or cancer biology.

Better get used to it. That's basically tech's entire value proposition.

Re: GPT-4 Outperforms Elite Crowdworkers, Saving Researchers $500k and 20k hours

#63

Post by nr2x »

Roritharr wrote: post #20
Teever wrote: post #16

Earlier quoted context omitted.

Yeah, I think the NSA is going to get their money's worth for that Utah Datacenter that they started building like 20 years ago.

Looking at this from far away, with the Snowden revelations in mind I'd think it's not tinfoil hat territory to assume that some of the progress at OpenAI got achieved with some help from well ressourced folks in the USG/Three Letter Agencies.

I don’t think they helped them. Now, did they train off the same data sources? Well, since OpenAI isn’t saying what GPT-4 is trained on, and the NSA can hoover up all kinds of non-public data, it stands to reason they may both be doing something slightly shady with emails, texts, and the like.

Re: GPT-4 Outperforms Elite Crowdworkers, Saving Researchers $500k and 20k hours

#64

Post by nr2x »

876978095789789 wrote: post #54
nr2x wrote: post #53

Earlier quoted context omitted.

In this case it is academic research released to the public, which is why anybody knows about it. This is a fairly good thing compared to any alternative I can think of.

> In this case it is academic research released to the public That's going to be the exception, not the rule. The benefits from automating crowdwork will disproportionately accrue to corporate profits.

Exactly, which is why this research is good to release. That’s my point.

Re: GPT-4 Outperforms Elite Crowdworkers, Saving Researchers $500k and 20k hours

#66

Post by nr2x »

ftxbro wrote: post #57

If you look at the table, the GPT-4 model has better correlation with the expert ensemble than the crowd does, but only on some criteria. The GPT-4 model is closer for all of the ethics questions, but the crowd is closer for the utility level and economic impact questions.

Yes, but GPT-5 will be better and the humans won’t. It’s very troubling.

Re: GPT-4 Outperforms Elite Crowdworkers, Saving Researchers $500k and 20k hours

#67

Post by lovvtide »

HPMOR wrote: post #11
lovvtide wrote: post #9

Earlier quoted context omitted.

Now that is something I hadn't considered. Woah.

Not to sound condescending but really? How is this not immediately your mind goes? Every piece of information ever recorded can now be summarized and cross-linked efficiently. Privacy is beyond dead. Soon every authoritarian government (and Democratic ones albeit secretly) will have integrated platforms that track every single one of your movements, known contacts, internet usage, financial data, and correspondence.…

I know right! It's so obvious.

Re: GPT-4 Outperforms Elite Crowdworkers, Saving Researchers $500k and 20k hours

#68

Post by rossdavidh »

So, uh, GPT-4 outperforms at labeling. What is that labeling used for?

"Employing Surge AI's top-tier human annotators at a rate of $25 per hour would have cost $500,000 for 20,000 hours of work, an excessive amount to invest in the research endeavor. Surge AI is a venture-backed startup that performs the human labeling for numerous AI companies including OpenAI, Meta, and Anthropic."

What could go wrong? Using GPT-4 to perform labeling used by OpenAI in order to train...uh, wait.

Re: GPT-4 Outperforms Elite Crowdworkers, Saving Researchers $500k and 20k hours

#69

Post by SoftTalker »

dragonwriter wrote: post #30
SketchySeaBeast wrote: post #19

Earlier quoted context omitted.

Man, I can't wait for the AI to start hallucinating crimes.

Shotspotter is a thing and it already has been doing that for years (both on its own and on law enforcement request.)

Shotspotter isn't an AI though is it? I thought it was just triangulation of gunshot locations using microphones and synchronized clocks?

Re: GPT-4 Outperforms Elite Crowdworkers, Saving Researchers $500k and 20k hours

#70

Post by rossdavidh »

rossdavidh wrote: post #68

So, uh, GPT-4 outperforms at labeling. What is that labeling used for? "Employing Surge AI's top-tier human annotators at a rate of $25 per hour would have cost $500,000 for 20,000 hours of work, an excessive amount to invest in the research endeavor. Surge AI is a venture-backed startup that performs the human labeling for numerous AI companies including OpenAI, Meta, and Anthropic." What could go wrong? Using GPT-4…

https://en.wikipedia.org/wiki/Positive_feedback
Post reply on HN