'Rater' is paid $10 hourly to teach Google's algorithm

Post by q-big » Sat, May 07, 2022, 11:12 PM UTC

remus wrote: post #39

fshbbdssbbgdd wrote: post #33

Earlier quoted context omitted.

Google already has similar signals: 1. Do you click on the link in the results page? 2. After clicking on that link, do you go back to the search or not? 3. How long do you stay there before coming back? Adding an explicit up/downvote would capture some additional information, but it would overlap with the above, and most users would ignore the button. I’d guess Google has at some point experimented with this and fou…

> Adding an explicit up/downvote would capture some additional information, but it would overlap with the above, and most users would ignore the button. I’d guess Google has at some point experimented with this and found it wasn’t worth keeping. Sounds pretty ripe for abuse too.

Post by kappuchino » Sat, May 07, 2022, 11:17 PM UTC

Justin_K wrote: post #17

Another prime example of how ML and AI aren't mature. With all the data google collects, they have to rely on minimum wagers to "find" the best results.

Post by reaperducer » Sat, May 07, 2022, 11:20 PM UTC

ocdtrekkie wrote: post #13

rcktmrtn wrote: post #5

On a related note, am I right in understand that the "Google Experts" [1] do it for free? [1] https://developers.google.com/community/experts/

Yes, Google has tricked a massive army of people into doing free work for one of the most profitable companies in the world. Things like "Google Local Guides" give people a gold star (and maybe a pair of branded socks) for doing unpaid labor. It's kinda hilarious, but also sad. I imagine there are Google executives in a room somewhere laughing their full heads off every time they think about how there are people who…

Post by rater266 » Sat, May 07, 2022, 11:22 PM UTC

echen wrote: post #48

ggm wrote: post #44

Surprisingly badly paid. I wonder how strongly quality would track pay? Probably they act as a CAPTCHA mutually cross checking and so QA is by volume, and not coincidentally demands they work in isolation. Sad story.

Quality control is often indeed by volume. When I was at Google / Facebook, search evaluations would typically have 5 raters each. One of the difficulties, though, is that search evaluation can be quite subjective. And honestly, most of the search engine raters we used were low quality and did a bad job (so the aggregate answer was often nonsense).

Post by kappuchino » Sat, May 07, 2022, 11:26 PM UTC

echen wrote: post #48

ggm wrote: post #44

Surprisingly badly paid. I wonder how strongly quality would track pay? Probably they act as a CAPTCHA mutually cross checking and so QA is by volume, and not coincidentally demands they work in isolation. Sad story.

Quality control is often indeed by volume. When I was at Google / Facebook, search evaluations would typically have 5 raters each. One of the difficulties, though, is that search evaluation can be quite subjective. And honestly, most of the search engine raters we used were low quality and did a bad job (so the aggregate answer was often nonsense).