Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT-4 Outperforms Elite Crowdworkers, Saving Researchers $500k and 20k hours

artisana.ai

Post reply on HN
Print view View original

131–140 of 151 posts

Re: GPT-4 Outperforms Elite Crowdworkers, Saving Researchers $500k and 20k hours

#131

Post by hex4def6 »

alphanumeric0 wrote: post #18
troops_h8r wrote: post #7

I don't think I see enough discussion about what this means for privacy. There was some protection in the fact that it was prohibitively expensive to get someone to listen to every single one of our phonecalls/read all our emails/etc. Worrying that this will no longer be the case.

Why do you think there was a mass surveillance of American domestic communications since forever ago, as leaked by Snowden? This technology has been available since then and can effectively summarize millions of pieces of communication.

Collection an ocean of data from everyone isn't the same as actually painstakingly tying all the pieces together for everyone.

They've created a huge library of unorganized data. The difference here is they now can spawn a million untiring AI private investigators / librarians to organize this information into coherent "case files".

At least for me, until this point I've had a feeling of anonymity in the idea that, while my data is being slurped up, I'm just one data point in a sea of other 'normal' people. There would be little value in spending government time and effort tying all of the web detritus together for me. The juice would definitely not be worth the squeeze.

However, when the cost of this effort is nearly zero, that now becomes a different story. The balance of power between government and the people it rules is going to radically shift.

Re: GPT-4 Outperforms Elite Crowdworkers, Saving Researchers $500k and 20k hours

#132

Post by pixl97 »

pseudo0 wrote: post #114
lumost wrote: post #101

Earlier quoted context omitted.

The top use case I've been hearing is in legal discovery. Law firms used to play games with diligence by disclosing TBs of email and making it cost prohibitive to find relevant emails. This task would normally require a $60-100/hr paralegal or lawyer. GPT-4 can do that task for fractions of a penny per email now. It doesn't have to be perfect if its competing with nothing. I expect we'll see similar shops for any oth…

Is there a solution to the issue of data stewardship yet? I'd imagine it typically would not be permissible to send a bunch of proprietary legal documents off to OpenAI. What I'd really love to implement is a way for GPT-4 to answer questions based on a corpus of "all our Confluence pages plus random other sources of documentation." Like with the legal document issue, it's a bit of a nonstarter right now given the pr…

GPT-4 based searchability of my works Confluence would save me a ton of time.

"Hey, has anyone worked on Problem X, and what was the outcome of their project"

Re: GPT-4 Outperforms Elite Crowdworkers, Saving Researchers $500k and 20k hours

#133

Post by pixl97 »

two_in_one wrote: post #96
alphanumeric0 wrote: post #18

Earlier quoted context omitted.

Why do you think there was a mass surveillance of American domestic communications since forever ago, as leaked by Snowden? This technology has been available since then and can effectively summarize millions of pieces of communication.

Not exactly. Gov had to be selective because its surveillance required a lot of resources per person/call. New technology allows it cheap and en mass. Voice calls can be recorded, then converted to text, then filtered. And humans will only analyze something of interest. Like we did have alphabet and books, and newspapers for hundreds of years. But only with internet we got the ability to process them easily.

Not only converted to text, it seems likely that we can document sentiment around the persons speech. For example if you're a low priority target that's still on the radar, but not high enough on the list to get a human handler I could see something like, not only what you said, but were you laughing, angry, crying. The the tone of your voice indicate the likelihood of action in a short time frame?

Re: GPT-4 Outperforms Elite Crowdworkers, Saving Researchers $500k and 20k hours

#134

Post by pixl97 »

111111101101 wrote: post #127
stevenhuang wrote: post #42

Earlier quoted context omitted.

> This technology has been available since then No, it hasn't lol.

I'm not so sure about that. I mean, maybe not since the Snowden leaks but how do we know that governments haven't been running their own LLMs for the last five years or so? We know that they're using sockpuppets[1]. We know that they're astroturfing[2]. Integrating LLMs into their toolkits seems like an obvious move, so obvious that they would be stupid not to do it. [1] https://www.theguardian.com/technology/2011/ma…

>haven't been running their own LLMs for the last five years

Because the hardware has not existed.

This said by accident I've seen hardware that was brought to a testing company by federal marshals that was massively parallel custom hardware that was likely for signal processing a lot of channels at once. So there is plenty of custom hardware out there, but these items have not been produced at the scale needed (from what anyone can tell) and, again from what we can tell, they don't have the general processing capability that GPU/TPU driven LLMs have.

Re: GPT-4 Outperforms Elite Crowdworkers, Saving Researchers $500k and 20k hours

#135

Post by realusername »

rossdavidh wrote: post #68

So, uh, GPT-4 outperforms at labeling. What is that labeling used for? "Employing Surge AI's top-tier human annotators at a rate of $25 per hour would have cost $500,000 for 20,000 hours of work, an excessive amount to invest in the research endeavor. Surge AI is a venture-backed startup that performs the human labeling for numerous AI companies including OpenAI, Meta, and Anthropic." What could go wrong? Using GPT-4…

We're also using computers to build better computers, it makes sense

Re: GPT-4 Outperforms Elite Crowdworkers, Saving Researchers $500k and 20k hours

#136

Post by pixl97 »

talesvin wrote: post #26
SketchySeaBeast wrote: post #19

Earlier quoted context omitted.

Man, I can't wait for the AI to start hallucinating crimes.

I mean, if we get to the point AI is pointing the finger at someone i hope that a human will double check it at least.

How long before the AI built in the US figures out its reward function is 80% more likely to be satisfied if it points its digital finger at someone black rather than the most likely subject?

Re: GPT-4 Outperforms Elite Crowdworkers, Saving Researchers $500k and 20k hours

#137

Post by pixl97 »

Teever wrote: post #89
Roritharr wrote: post #20

Earlier quoted context omitted.

Looking at this from far away, with the Snowden revelations in mind I'd think it's not tinfoil hat territory to assume that some of the progress at OpenAI got achieved with some help from well ressourced folks in the USG/Three Letter Agencies.

Yeah, I've started wondering similar things about that too, like how far ahead is the NSA on this stuff? And how does that tie in to the recent policy of denying China semiconductors? Perhaps history will show that the NSA made algorithmic breakthroughs a few years ago and realized what was coming, so political policy was crafted to stymie Chinese progress in this field, and what we're seeing in the public sphere fro…

You're seeing the government entity as separate from the corporate entity, but quite often in the US its the other way around. The government entity is a rather hollow shell, and the 'brains' of the operation is contracted out to the corporation. The government entity would almost cease to exist if the corporation under it magically disappeared.

Re: GPT-4 Outperforms Elite Crowdworkers, Saving Researchers $500k and 20k hours

#138

Post by pixl97 »

mclightning wrote: post #119
rossdavidh wrote: post #68

So, uh, GPT-4 outperforms at labeling. What is that labeling used for? "Employing Surge AI's top-tier human annotators at a rate of $25 per hour would have cost $500,000 for 20,000 hours of work, an excessive amount to invest in the research endeavor. Surge AI is a venture-backed startup that performs the human labeling for numerous AI companies including OpenAI, Meta, and Anthropic." What could go wrong? Using GPT-4…

This is a bigger problem than people realise. Think about it, how many millions of articles are posted online produced by OpenAI's GPTs to date... Good luck clearing out the training data for GPT-5. True human content will get gradually scarce. We steer it for sure for our posts, but it is still GPTs that do the heavy lifting. OpenAI's own classifier fails to detect GPT-4 generated text at the moment.

>classifier fails to detect GPT-4 generated text

That's because beyond the 'As an AI language model' and a few key words it can be nearly impossible to detect GPT-4 especially if any prompt is used to intentionally keep it from being detected.

Human like text is a solved problem. There is no more getting better at detecting AI written text, there is only classifying more humans incorrectly at this point.

Re: GPT-4 Outperforms Elite Crowdworkers, Saving Researchers $500k and 20k hours

#139

Post by pixl97 »

nr2x wrote: post #71
courseofaction wrote: post #24

We need new political arrangements to distribute the gains of AI or things are going to get very bad very quickly.

AI isn’t the problem, capitalism is. I’m not worried about rogue AI taking over the nukes. I’m worried the same people who think it’s a great idea to charge so much for insulin that people start dying are the ones who will be using AI to hurt people. Hell, give me a slightly evil AI run amok over any pharma CEO doing their job.

So China/Russia/Wherever-stan is going to use AI more responsibly?

Human greed is the problem. Authoritarianism and capitalism are just subcategories of the greed problem.

What we don't have an answer for yet, is will AGI be greedy?

Re: GPT-4 Outperforms Elite Crowdworkers, Saving Researchers $500k and 20k hours

#140

Post by pixl97 »

selectodude wrote: post #55
boringuser2 wrote: post #31

Earlier quoted context omitted.

If money could solve this problem, China would lead AI, not OpenAI.

200,000 wafers per month is a lot of GPUs.

Mostly meaningless unless every other part of the supply chain exists to turn them into A100s. Being that supply chain is very difficult and expensive to extend, it's something that can take years to build out, even in a hurry.
Post reply on HN