Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT-4 Outperforms Elite Crowdworkers, Saving Researchers $500k and 20k hours

artisana.ai

Post reply on HN
Print view View original

121–130 of 151 posts

Re: GPT-4 Outperforms Elite Crowdworkers, Saving Researchers $500k and 20k hours

#121

Post by lumost »

pseudo0 wrote: post #114
lumost wrote: post #101

Earlier quoted context omitted.

The top use case I've been hearing is in legal discovery. Law firms used to play games with diligence by disclosing TBs of email and making it cost prohibitive to find relevant emails. This task would normally require a $60-100/hr paralegal or lawyer. GPT-4 can do that task for fractions of a penny per email now. It doesn't have to be perfect if its competing with nothing. I expect we'll see similar shops for any oth…

Is there a solution to the issue of data stewardship yet? I'd imagine it typically would not be permissible to send a bunch of proprietary legal documents off to OpenAI. What I'd really love to implement is a way for GPT-4 to answer questions based on a corpus of "all our Confluence pages plus random other sources of documentation." Like with the legal document issue, it's a bit of a nonstarter right now given the pr…

AFAIK the Azure APIs provide suitable data usage requirements. One of the most fascinating aspects of the AI world is that we've made extraordinarily expensive brute force search a valuable tool.

Re: GPT-4 Outperforms Elite Crowdworkers, Saving Researchers $500k and 20k hours

#122

Post by RC_ITR »

rozal wrote: post #116
RC_ITR wrote: post #77

Earlier quoted context omitted.

This isn’t really a “throw money at it” problem like the government is good at. Take drones for example. The government got really good at those because they made them jet-powered (lol) and blew a bunch of money on server-grade FPGA’s in each one of them. You can’t really just buy a lot of GPUs to make an LLM work, you need iterative development of architecture and training methods. Like maybe the government invented…

Did you like, forget the the military invented the nuclear bomb, semiconductors, coding, AI, NLP, the INTERNET. Everything you use is from the military. The government is good at lying, and making themselves ‘appear’ incompetent. They secretly probably have a much further advanced quantum computer. Your viewpoint is limited to mainstream technology and mainstream science.

This is such an interesting take.

The military invented the nuclear bomb yes. But Fermi did most of his thinking work in Italy before the Manhattan project. He got money thrown at him once he got here.

As for semiconductor devices, it was Bell Labs and TI.

Coding is an ambiguous concept that wasn’t really invented, but if it were, it would have first appeared in programmable looms.

The military likes to take credit for things, but really all they do is throw money at existing inventions.

I’m sure they’re throwing a bunch of money at Transformers now, but who are all these uncredited super geniuses who invent things and then let randos at Google take the credit/earn the money?

Re: GPT-4 Outperforms Elite Crowdworkers, Saving Researchers $500k and 20k hours

#123

Post by famouswaffles »

HeavyFeather wrote: post #107
two_in_one wrote: post #98

>This breakthrough saved the researchers over $500,000 and 20,000 hours of human labor. BTW, this is interesting. There is a lot of noise about AI carbon footprint. Now imagine how much humans would eat and fart for 20.000 work hours. It's about 10 man/years. Assuming 8h / 5d / 50 weeks schedule.

Indeed time to eliminate all those people I guess. /s I don’t think you can compare people’s carbon footprint because those people will exist regardless of jobs.

>I don’t think you can compare people’s carbon footprint because those people will exist regardless of jobs.

But they don't have to /s

Re: GPT-4 Outperforms Elite Crowdworkers, Saving Researchers $500k and 20k hours

#124

Post by generativeai »

mztwo wrote: post #8
AndreLock wrote: post #5

Interesting to see what the impact will be on crowdsourcing annotation companies like Scale AI, especially after reading this article: https://www.forbes.com/sites/kenrickcai/2023/04/11/how-alexa...

Anecdotally, several CTOs I know intend to lessen their use of Scale, Labelbox and more in the future. Talked to one today who already ditched MTurk for GPT-4 -- cheaper, better, faster was what he said. Labelbox does image annotating still, and one CTO said as soon as GPT-4 enabled this for him he'd have his team homebrew it from there.

Looks like Labelbox is doing something with GPT models...https://labelbox.com/blog/few-shot-learning-and-zero-shot-le...

Re: GPT-4 Outperforms Elite Crowdworkers, Saving Researchers $500k and 20k hours

#125

Post by rozal »

RC_ITR wrote: post #122
rozal wrote: post #116

Earlier quoted context omitted.

Did you like, forget the the military invented the nuclear bomb, semiconductors, coding, AI, NLP, the INTERNET. Everything you use is from the military. The government is good at lying, and making themselves ‘appear’ incompetent. They secretly probably have a much further advanced quantum computer. Your viewpoint is limited to mainstream technology and mainstream science.

This is such an interesting take. The military invented the nuclear bomb yes. But Fermi did most of his thinking work in Italy before the Manhattan project. He got money thrown at him once he got here. As for semiconductor devices , it was Bell Labs and TI. Coding is an ambiguous concept that wasn’t really invented, but if it were, it would have first appeared in programmable looms. The military likes to take credit…

I used to agree with you. I used to think the military was kinda dumb. But after doing a deeper dive into past military technology, and present - I've come to realize this is just an intelligence ruse.

They made some mistakes in the 40's and 50's to where they had nuclear secrets stolen by the Russians. And ever since has been hyper compartmentalized.

It would not surprise me if in the 90's or 00's they had an internal working LLM, considering all the puzzle pieces. You will never hear about classified tech unless it's a bomb, gets leaked. (See code breaking machines declassed after 70+ years)

In a hypothetical scenario, a military organization might want to conceal its use of a large language model (LLM) for intelligence gathering and analysis.

Another scenario is the military's current interest in everything quantum. Quantum computers for example (you wouldn't want another nation being first and pirate baying out our secrets, would you) so there is an extreme national security importance of being first.

And to be first, you need to have the smart people, which the military has. There is a reason China struggles with jet engines 80 years after their invention, and still can't make nuclear carriers. While the US navy works on things like this: https://www.navair.navy.mil/foia/sites/g/files/jejdrs566/fil...

Re: GPT-4 Outperforms Elite Crowdworkers, Saving Researchers $500k and 20k hours

#126

Post by swagasaurus-rex »

famouswaffles wrote: post #123
HeavyFeather wrote: post #107

Earlier quoted context omitted.

Indeed time to eliminate all those people I guess. /s I don’t think you can compare people’s carbon footprint because those people will exist regardless of jobs.

>I don’t think you can compare people’s carbon footprint because those people will exist regardless of jobs. But they don't have to /s

The market will determine their suitability

Re: GPT-4 Outperforms Elite Crowdworkers, Saving Researchers $500k and 20k hours

#127

Post by 111111101101 »

stevenhuang wrote: post #42
alphanumeric0 wrote: post #18

Earlier quoted context omitted.

Why do you think there was a mass surveillance of American domestic communications since forever ago, as leaked by Snowden? This technology has been available since then and can effectively summarize millions of pieces of communication.

> This technology has been available since then No, it hasn't lol.

I'm not so sure about that. I mean, maybe not since the Snowden leaks but how do we know that governments haven't been running their own LLMs for the last five years or so? We know that they're using sockpuppets[1]. We know that they're astroturfing[2]. Integrating LLMs into their toolkits seems like an obvious move, so obvious that they would be stupid not to do it.

[1] https://www.theguardian.com/technology/2011/mar/17/us-spy-op...

[2] https://boingboing.net/2015/06/22/gchqs-psy-ops-squad-target...

Re: GPT-4 Outperforms Elite Crowdworkers, Saving Researchers $500k and 20k hours

#128

Post by dwigthsomething »

two_in_one wrote: post #96
alphanumeric0 wrote: post #18

Earlier quoted context omitted.

Why do you think there was a mass surveillance of American domestic communications since forever ago, as leaked by Snowden? This technology has been available since then and can effectively summarize millions of pieces of communication.

Not exactly. Gov had to be selective because its surveillance required a lot of resources per person/call. New technology allows it cheap and en mass. Voice calls can be recorded, then converted to text, then filtered. And humans will only analyze something of interest. Like we did have alphabet and books, and newspapers for hundreds of years. But only with internet we got the ability to process them easily.

> Gov had to be selective because its surveillance required a lot of resources

not exactly. Where do you think all those budget trillions that don't have to be accounted for goes into? the FBI+NSA (=CIA but for citzens) have infinite resources.

All the overhead they have is to make sure a small subset of the citizens are not impacted. Snowden goes into this in some detail when talking about day to day operations. The norm is to extend the net as wide as possible, until you reach some politician or government agency.

Re: GPT-4 Outperforms Elite Crowdworkers, Saving Researchers $500k and 20k hours

#129

Post by RC_ITR »

rozal wrote: post #125
RC_ITR wrote: post #122

Earlier quoted context omitted.

This is such an interesting take. The military invented the nuclear bomb yes. But Fermi did most of his thinking work in Italy before the Manhattan project. He got money thrown at him once he got here. As for semiconductor devices , it was Bell Labs and TI. Coding is an ambiguous concept that wasn’t really invented, but if it were, it would have first appeared in programmable looms. The military likes to take credit…

I used to agree with you. I used to think the military was kinda dumb. But after doing a deeper dive into past military technology, and present - I've come to realize this is just an intelligence ruse. They made some mistakes in the 40's and 50's to where they had nuclear secrets stolen by the Russians. And ever since has been hyper compartmentalized. It would not surprise me if in the 90's or 00's they had an intern…

I mean, I think you’re again missing the point here.

Jet engines can be solved with money.

The actual steps of making an LLM require complex math and you can’t just pay people to make better math.

And if you could, wouldn’t those people decamp for industry and become literal trillionares?

Re: GPT-4 Outperforms Elite Crowdworkers, Saving Researchers $500k and 20k hours

#130

Post by lazyeye »

lumost wrote: post #101
troops_h8r wrote: post #7

I don't think I see enough discussion about what this means for privacy. There was some protection in the fact that it was prohibitively expensive to get someone to listen to every single one of our phonecalls/read all our emails/etc. Worrying that this will no longer be the case.

The top use case I've been hearing is in legal discovery. Law firms used to play games with diligence by disclosing TBs of email and making it cost prohibitive to find relevant emails. This task would normally require a $60-100/hr paralegal or lawyer. GPT-4 can do that task for fractions of a penny per email now. It doesn't have to be perfect if its competing with nothing. I expect we'll see similar shops for any oth…

There's also this..

https://old.reddit.com/r/ChatGPT/comments/12fiwaf/chat_gpt_w...

Post reply on HN