Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

'Rater' is paid $10 hourly to teach Google's algorithm

finance.yahoo.com

Post reply on HN
Print view View original

81–90 of 117 posts

Re: 'Rater' is paid $10 hourly to teach Google's algorithm

#81

Post by rater266 »

vgeek wrote: post #76
rater266 wrote: post #59

Earlier quoted context omitted.

Former longtime Leapforce/RaterLabs preferred/senior rater here. There was always some exploitation going on, but it got way worse under Ruth Porat's tenure. Standard Leapforce US English raters were paid $13.50/hr. If you were lucky enough to get promoted to "Preferred Agent" (PA), you made $17.40/hr. The criteria for PA promotion was basically to be in the top 5% or so on their quality metrics, plus various subject…

Thank you for all of this information! I've roughly followed it as a marketer (YMYL ranking factors from the guidelines type stuff) and as an observer-- it is interesting that the shifts lined up with the Porat era. The two people I knew who did Leapforce also did human powered answers for ChaCha(?) where they were paid per answer versus an hourly rate, and then migrated to Leapforce at a lower date. They were both i…

There were different time allotments per task, yes. In the Raterhub era, each task had an "Average Estimated Time" (AET) set by Google. A typical 5x5 side-by-side search result comparison task would have been 9 minutes, for example. I may be forgetting something but I don't think we ever saw the individual URL rating tasks (which I think I recall were 1.5 mins? maybe it was 2. It's been so long) after the Raterhub switch.

AETs were usually reasonable until the Ruth Porat era. After Porat, they seemed to be getting pretty aggressive about reducing them as far as possible to the point of absurdity. We could release tasks for insufficient AET. I suspect they added automated tooling in the Porat era to "optimize" AET in their favor. With only a few exceptions I can remember, AETs only went down, not up.

If you were flagged by the bot, there was nothing you could do except wait for Leapforce to review you. That could take anywhere from a few hours to several days depending on how busy they were. I wouldn't say there was any meaningful appeal route; I mean, you could email their quality team, and maybe you could plead for leniency, but I don't know anyone had any real success with that. (I don't have personal experience with that; as I said, I was a PA/senior, which was a quality-based designation, so it follows that I tended to be pretty good at the job! But I always felt bad for other raters in chat who had to deal with that, seeing their livelihoods put at risk in such an arbitrary and robotic manner. If someone isn't a good fit for the job, then yes, it's not going to work out, but I do believe they should be given feedback and guidance on how to improve and a fair chance to do so.)

Re: 'Rater' is paid $10 hourly to teach Google's algorithm

#82

Post by anm89 »

bhartzer wrote: post #75
rednalexa wrote: post #54

Earlier quoted context omitted.

I would assume that if Google is paying 10$/hour for humans to rank content, someone else is paying 11$/hour to fool it with a human generating content, clicks, etc. Additionally, people writing bots learn just like Google learns, and start generating what appears to be legitimate interactions. Same algorithms that fit to critique between 'bot' and 'non-bot' class can be used to create bots that fall into the 'non-bo…

That’s what’s confusing in the articles that have come out about quality raters. Quality Raters don’t actually rank websites. Quality Raters look at websites and answer questions about them. Google then uses that data as a small part of their algorithm to adjust some factors about those sites. Quality Raters don’t actually rank pages on google.

This isn't correct. I was a quality rater. We spent almost all of our time literally ranking websites.

Re: 'Rater' is paid $10 hourly to teach Google's algorithm

#83

Post by rater266 »

vgeek wrote: post #76
rater266 wrote: post #59

Earlier quoted context omitted.

Former longtime Leapforce/RaterLabs preferred/senior rater here. There was always some exploitation going on, but it got way worse under Ruth Porat's tenure. Standard Leapforce US English raters were paid $13.50/hr. If you were lucky enough to get promoted to "Preferred Agent" (PA), you made $17.40/hr. The criteria for PA promotion was basically to be in the top 5% or so on their quality metrics, plus various subject…

Thank you for all of this information! I've roughly followed it as a marketer (YMYL ranking factors from the guidelines type stuff) and as an observer-- it is interesting that the shifts lined up with the Porat era. The two people I knew who did Leapforce also did human powered answers for ChaCha(?) where they were paid per answer versus an hourly rate, and then migrated to Leapforce at a lower date. They were both i…

I'll add that even though the pay rates were nominally quoted per hour, under Leapforce, it was really task-based because they wouldn't pay more than AET. I know raters had their invoices rejected if their productivity was materially under 100%.

I assume they pay for all time worked under RaterLabs, due to employee wage and hour laws. Of course, they can still fire a rater for not meeting productivity requirements.

Re: 'Rater' is paid $10 hourly to teach Google's algorithm

#84

Post by geoduck14 »

If there is a web site with a LOT of content, does Google get SQR for each page or do they get SQR for "a couple of pages" and apply that rating for the entire site?

When you rate "accruacy" of information or "authorativeness" of a site, how do you know the site is "an authority"? are you guessing?

Re: 'Rater' is paid $10 hourly to teach Google's algorithm

#85

Post by fshbbdssbbgdd »

bhartzer wrote: post #71
fshbbdssbbgdd wrote: post #33

Earlier quoted context omitted.

Google already has similar signals: 1. Do you click on the link in the results page? 2. After clicking on that link, do you go back to the search or not? 3. How long do you stay there before coming back? Adding an explicit up/downvote would capture some additional information, but it would overlap with the above, and most users would ignore the button. I’d guess Google has at some point experimented with this and fou…

We know they probably use CTR as a ranking factor. However, a site could just literally add JS code on the page and disable the back button. So I don’t think that would work out that well for Google.

That sounds like a trick that would only work until Google noticed.

Re: 'Rater' is paid $10 hourly to teach Google's algorithm

#86

Post by hn_throwaway_99 »

demadog wrote: post #51

When those reports come out that “The average Google employee makes $220k a year” or whatever, it’s totally misleading since it doesn’t account for all the outsourced labor at $10 an hour that is critical to tuning their algorithms.

I don't think that's misleading at all. Why would I assume that non-Google employee compensation would go into the calculation for what the average Google employee makes?

As someone else pointed out, this pays substantially more than AWS Mechanical Turk. To be honest, it's difficult for me to imagine an easier job than this one. I mean, most people who would do this job could easily now make more at McDonald's or Walmart or whatever. But I think jobs at McDonald's or Walmart are considerably harder, so I don't see why this job should pay the same.

Re: 'Rater' is paid $10 hourly to teach Google's algorithm

#87

Post by [deleted] »

q-big wrote: post #61
remus wrote: post #39

Earlier quoted context omitted.

> Adding an explicit up/downvote would capture some additional information, but it would overlap with the above, and most users would ignore the button. I’d guess Google has at some point experimented with this and found it wasn’t worth keeping. Sounds pretty ripe for abuse too.

> Sounds pretty ripe for abuse too. civilized wrote (emphasis by me): "and Google enhances its ranking for me based on that".

[deleted]

Re: 'Rater' is paid $10 hourly to teach Google's algorithm

#88

Post by vultour »

demadog wrote: post #51

When those reports come out that “The average Google employee makes $220k a year” or whatever, it’s totally misleading since it doesn’t account for all the outsourced labor at $10 an hour that is critical to tuning their algorithms.

Google salaries outside of the US are a fraction of that.

Re: 'Rater' is paid $10 hourly to teach Google's algorithm

#89

Post by kappuchino »

amelius wrote: post #22
Gravyness wrote: post #3

Definitely explains the quality of search as of late.

Those raters must really love Pinterest ...

Acually ... no, but pinterest provides results that are relevant and satisfying to the general search (and the SQR - the Search Quality Rating, see link in discussion). Hence Pinterest stays in good standing even if most people like me (you, too?) hate it for its login/content walling tactics.

Re: 'Rater' is paid $10 hourly to teach Google's algorithm

#90

Post by lupire »

cpeterso wrote: post #74

The search quality rater guide says: “Your ratings will not directly affect how a particular webpage, website, or result appears in Google Search, nor will they cause specific webpages, websites, or results to move up or down on the search results page. Instead, your ratings will be used to measure how well search engine algorithms are performing for a broad range of searches.” Is this accurate? Or is the guide bendi…

Google hires professionals, and surveys the Internet username (Search users and Analytics-integrated website users) to teach the algorithm.
Post reply on HN