'Rater' is paid $10 hourly to teach Google's algorithm • Hacker News Board

Re: 'Rater' is paid $10 hourly to teach Google's algorithm

#111

Post by Avamander » Sun, May 08, 2022, 9:02 AM UTC

kappuchino wrote: post #89

amelius wrote: post #22

Earlier quoted context omitted.

Those raters must really love Pinterest ...

Acually ... no, but pinterest provides results that are relevant and satisfying to the general search (and the SQR - the Search Quality Rating, see link in discussion). Hence Pinterest stays in good standing even if most people like me (you, too?) hate it for its login/content walling tactics.

> but pinterest provides results that are relevant and satisfying to the general search

More like it matches some keywords like run-of-the-mill WordPress malware generates. I've seen plenty of both. Not to mention the similarity to the masses of biz/buzz/xyz spam websites.

Re: 'Rater' is paid $10 hourly to teach Google's algorithm

#112

Post by kappuchino » Sun, May 08, 2022, 3:23 PM UTC

sfmike wrote: post #107

kappuchino wrote: post #56

Ex Google Search quality Rater here, based in Germany. From 2004 to 2006 I worked on SQR and "special projects", with a much better pay (maybe because of german circumstances). I've been tracking the SQR-guide since then (public version here https://static.googleusercontent.com/media/guidelines.raterh... ), so I have still some current understanding. If you have questions, I'd be happy to try to answer them since my…

What do you think about gpt3 and ai content on serps and how will it be rated and ranked?

Good question, disapointing answer: Only time will tell. I have a hunch however, from stylometry work. Its that for short texts "generated" text will be virtually indestinguishable and this will grow as ml/ai systems advance to longer and longer texts. This will probably lead to different techniques to rate content (like more emphasis on the Your life/Your money factor).

Re: 'Rater' is paid $10 hourly to teach Google's algorithm

#113

Post by kappuchino » Sun, May 08, 2022, 3:28 PM UTC

dmarchand90 wrote: post #106

kappuchino wrote: post #56

Ex Google Search quality Rater here, based in Germany. From 2004 to 2006 I worked on SQR and "special projects", with a much better pay (maybe because of german circumstances). I've been tracking the SQR-guide since then (public version here https://static.googleusercontent.com/media/guidelines.raterh... ), so I have still some current understanding. If you have questions, I'd be happy to try to answer them since my…

I don't have any specific questions but would be really interested in hearing more!

Its an interesting boring job in its core. At least when I did it. You agree to see theoretically everything that people search for and try to rate it according to the linked manual. In my time - really long ago - britney spears was one of the most common searches (I remember it probably wrong, but the test I had to do was half rating questions on spears and tom hanks). But I also got to rate "donkey s*x" and things like "divorce help" which makes you think about the impact search results can have. The boring part was that it is kind of repetetive to rate and you seem to learn how simplistic people search most of the time (but then again, only common/frequent searches are tested, at least most of the time). And it was enlightning how search spammers adapted to each no measure to keep them out. I still know the names of the people who were infamous at that time for setting up fake, keyword stuffed "blogs" to get people to their shop.

Re: 'Rater' is paid $10 hourly to teach Google's algorithm

#114

Post by ma2rten » Sun, May 08, 2022, 5:16 PM UTC

kappuchino wrote: post #56

Ex Google Search quality Rater here, based in Germany. From 2004 to 2006 I worked on SQR and "special projects", with a much better pay (maybe because of german circumstances). I've been tracking the SQR-guide since then (public version here https://static.googleusercontent.com/media/guidelines.raterh... ), so I have still some current understanding. If you have questions, I'd be happy to try to answer them since my…

Do you feel the rating guidelines were well thought out? Is there anything you would change?

Did you work from an office or remotely? Did you interact with other raters?

Was there an interview process? How about performance reviews? How did Google ensure quality?

Re: 'Rater' is paid $10 hourly to teach Google's algorithm

#115

Post by washadjeffmad » Sun, May 08, 2022, 8:32 PM UTC

cyral wrote: post #64

washadjeffmad wrote: post #14

Earlier quoted context omitted.

This is probably one of the more embarrassing and shameful parts of Google's labor ecosystem. These "Google $title Experts" work as uncompensated customer support (a legal requirement that they treat as such, if the quality is any indicator) on various social media outlets like Reddit and Twitter. They don't just provide free labor, they also go to bat for Google's reputation, acting as marketing and PR liaisons to p…

90% of the time when I run into one of these "experts" on their support forum, the solution is just that the customer is wrong in some way and such feature isn't needed or something like that. Do these experts even have any access to help solve problems? I'm not sure what makes them so compelled to do this for free for one of the biggest companies ever.

Google pays for support at different tiers from various locations, and the Experts usually have access to second tier support which is usually hosted in the US. They're supposedly ITIL, so they claim to use CSI to solve recurring problems, which is even worse, because now they're not compensating people for skilled non-entry level work. It's mostly nativity and being able to tell their families that they're "working with Google".

First tier support is hosted primarily in India where they don't know who or how many are doing the work, and it's nearly impossible to get escalated as a customer to the correct support tier outside of their script. It's frustrating and mostly useless, but they can handle volume changes quickly at sub-human costs.

Re: 'Rater' is paid $10 hourly to teach Google's algorithm

#116

Post by demadog » Mon, May 09, 2022, 5:15 PM UTC

kappuchino wrote: post #66

demadog wrote: post #51

When those reports come out that “The average Google employee makes $220k a year” or whatever, it’s totally misleading since it doesn’t account for all the outsourced labor at $10 an hour that is critical to tuning their algorithms.

Its better paid than Amazon Mechanical Turk afaik (not saying it should not be paid better). The tuning is done literally by thousands world wide and since every rating task is done by multiple quality raters (afaik side by side is done by at least 5), it seems to be a cost vs. scale factor.

Amazon Turk is more of a platform for small business to hire other workers for micro-tasks. Raterlabs is used only by Google to improve their core product.

Re: 'Rater' is paid $10 hourly to teach Google's algorithm

#117

Post by yupper32 » Mon, May 09, 2022, 5:46 PM UTC

anm89 wrote: post #104

yupper32 wrote: post #94

Earlier quoted context omitted.

Your ranking is a signal that goes into the algorithm, not the actual end result.

I understand that. My post never implies that that was the case. > Quality Raters look at websites and answer questions about them. Wrong. I was never asked any questions beyond to rank to site quality for a query > Quality Raters don’t actually rank website Wrong again. This is what we did. We assessed website rankings for specific queries.

Why do you think you're the only type of rater?