Live data from Hacker News

GPT-4 Outperforms Elite Crowdworkers, Saving Researchers $500k and 20k hours

artisana.ai

71–80 of 151 posts

Re: GPT-4 Outperforms Elite Crowdworkers, Saving Researchers $500k and 20k hours

#71

We need new political arrangements to distribute the gains of AI or things are going to get very bad very quickly.

AI isn’t the problem, capitalism is.

I’m not worried about rogue AI taking over the nukes. I’m worried the same people who think it’s a great idea to charge so much for insulin that people start dying are the ones who will be using AI to hurt people.

Hell, give me a slightly evil AI run amok over any pharma CEO doing their job.

Re: GPT-4 Outperforms Elite Crowdworkers, Saving Researchers $500k and 20k hours

#72

Earlier quoted context omitted.

This is a problem that money can solve.

If money could solve this problem, China would lead AI, not OpenAI.

And 9 women would output a baby in 1 month!

Re: GPT-4 Outperforms Elite Crowdworkers, Saving Researchers $500k and 20k hours

#73

We need new political arrangements to distribute the gains of AI or things are going to get very bad very quickly.

We are fucked. I have no hope in humanity managing this technology responsibly and no hope in my future. The months since ChatGPT's release have been some of the worst in my life, mental-health-wise.

Re: GPT-4 Outperforms Elite Crowdworkers, Saving Researchers $500k and 20k hours

#74

Great to see this tech and the money invested in it being used to take low-paying jobs away from people with limited options, instead of something like drug discovery or cancer biology.

This assumes those people really do have no other way to contribute. I don’t believe that’s the case. Do you?

I believe people can contribute in many different ways. When technology enables us to get my work output without me, that frees me up to produce other things for society.

Re: GPT-4 Outperforms Elite Crowdworkers, Saving Researchers $500k and 20k hours

#75

So, uh, GPT-4 outperforms at labeling. What is that labeling used for? "Employing Surge AI's top-tier human annotators at a rate of $25 per hour would have cost $500,000 for 20,000 hours of work, an excessive amount to invest in the research endeavor. Surge AI is a venture-backed startup that performs the human labeling for numerous AI companies including OpenAI, Meta, and Anthropic." What could go wrong? Using GPT-4…

Yep - you highlighted exactly what raised my eyebrow as I was writing the article.

Re: GPT-4 Outperforms Elite Crowdworkers, Saving Researchers $500k and 20k hours

#76

Great to see this tech and the money invested in it being used to take low-paying jobs away from people with limited options, instead of something like drug discovery or cancer biology.

PC and Xerox eliminated the secretarial pool, and working women since then have been working on much more meaningful things.

Re: GPT-4 Outperforms Elite Crowdworkers, Saving Researchers $500k and 20k hours

#77
post #56

Earlier quoted context omitted.

> This technology has been available since then No, it hasn't lol.

How sure are you about that? The basic theories have been around since the 70s, have been proven at scale in the last decade, and the NSA has more data and compute than anybody else. I’d be shocked if they aren’t very far along in solving many problems.

This isn’t really a “throw money at it” problem like the government is good at.

Take drones for example. The government got really good at those because they made them jet-powered (lol) and blew a bunch of money on server-grade FPGA’s in each one of them.

You can’t really just buy a lot of GPUs to make an LLM work, you need iterative development of architecture and training methods.

Like maybe the government invented self-attention before 2017, but if they didn’t, then the constraint is training time, and the government has the same number of seconds as the rest of us.

Re: GPT-4 Outperforms Elite Crowdworkers, Saving Researchers $500k and 20k hours

#78

Earlier quoted context omitted.

> This technology has been available since then No, it hasn't lol.

Yes, it has. Consumer-wise, we've had Dragon Naturally Speaking since the late 90s. It's pretty simple to have a script read what it outputs text-wise and look for key words. No AI is even needed to do this.

Gaussian transcription models are old, but they also are AI.

They are not deep learning/neural nets.

Also fun fact as a pedant tax: Symantec is so named because they started out as transcription software, hit a wall, and pivoted to security SW.

Re: GPT-4 Outperforms Elite Crowdworkers, Saving Researchers $500k and 20k hours

#79
post #66
post #57

If you look at the table, the GPT-4 model has better correlation with the expert ensemble than the crowd does, but only on some criteria. The GPT-4 model is closer for all of the ethics questions, but the crowd is closer for the utility level and economic impact questions.

Yes, but GPT-5 will be better and the humans won’t. It’s very troubling.

I agree, but there are questions about GPT-N successors.

Surprisingly (to me) many people think that GPT-N will never exceed human level intelligence because it was trained on the internet. I think that argument is obviously wrong.

Another is that I am sure a large chunk of people will never concede that the AI is smarter than them. Literally never, no matter how smart the bot gets. I mean, probably a lot of people think they are as smart as anyone else. They won't agree that someone else is smarter than them, and they certainly won't agree that some bot is smarter than them. It's also a loaded assessment, like they will think that if they agree to that, then they are also implicitly agreeing to cede their personal agency to the bot.

Another possibility is that GPT-N successors that surpass human level cognition will be banned by regulation, like some drugs or nuclear explosives or bio weapons. They could even be pre-emptively banned at some level below human level, and maybe it would never be publicly acknowledged that it's technically possible to go above human level.

Post reply on HN