Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Life of a Data Labeler

scale.ai

Post reply on HN
Print view View original

11–20 of 31 posts

Re: Life of a Data Labeler

#11

Post by joshvm »

b_tterc_p wrote: post #3

I know nothing about this particular company. I also see no reason to assume it’s particularly bad (it could very well be above average). But I do see a lot of data around company engagement. There’s basically no chance this level of positivity is ubiquitous for them, especially when the target workforce appears to be people otherwise below the poverty line. Although pointless tedium is not the most important factor…

There's a great paper about image labelling, written by an expert labeller: https://arxiv.org/abs/1210.3448

One interesting side to this is citizen science; websites like Zooniverse where you can get random members of the public to do your labelling. Technically there's little difference to what Scale does, just that it might be pictures of Pelicans or galaxies rather than cars on the road. In my experience what you're paying for with Scale is consistency. Zooniverse relies on the public being... Gaussian in quality - we needed something like twenty annotations per image I think.

People spend hours going through data and they don't get paid. On Zooniverse most of the data is boring, but the fun is the prospect of finding an unusual image. Some of the datasets are genuinely interesting (like the old ship logs). Many people say they enjoy the idea that they're contributing to science.

> Although pointless tedium is not the most important factor in rating a job, I think data labeling could well be along the most pointlessly tedious jobs ever created.

I don't want to pass judgement either way, but it's an interesting debate. These jobs have been around forever - data entry, for example, used to be (still is?) a fairly common unskilled job in first world countries. And tedium aside, you can at least work from home if you have a computer.

But where is the line? A lot of work I've seen on MTurk is transcription, for example. What about translation/subtitling services? These are labelling tasks in some way or another.

Re: Life of a Data Labeler

#12

Post by b_tterc_p »

killjoywashere wrote: post #9
b_tterc_p wrote: post #3

I know nothing about this particular company. I also see no reason to assume it’s particularly bad (it could very well be above average). But I do see a lot of data around company engagement. There’s basically no chance this level of positivity is ubiquitous for them, especially when the target workforce appears to be people otherwise below the poverty line. Although pointless tedium is not the most important factor…

> the most pointlessly tedious jobs ever created Are you saying that the tedium itself is engineered to be present? I mean, the work is definitely not pointless. It is as pointed as it gets, as significant as a miner in a mine, a farmer in the field. Monotonous, tedious, yes. Pointless? No. Now, the idea that the tedium is engineered in would be interesting. That would suggest it could be engineered out. Which is ver…

No I don’t it’s intentional. I suppose I should have written “pointless, tedious” instead.

I’m not sure how I would compare it to traditional blue collar low skilled labor jobs. Farm hand work, for instance, in addition to being physically difficult, is also surprisingly tedious contrary to what some people may imagine. But I think it is easier to understand why it is important work. If you don’t pick the berries, the berries die and nobody gets to eat them. If you’re a machine learning data labeler, you may believe that it’s creating a path to super intelligent computers. Or you may believe it’s a big waste of time. My acquaintances who did Alexa data labeling tended to lean towards the latter.

Re: Life of a Data Labeler

#14

Post by Simple_Guy »

king_magic wrote: post #5
9nGQluzmnq3M wrote: post #4

Earlier quoted context omitted.

Mental tedium is a relative luxury: it's still clean indoor work with a predictable income, and thus far preferable compared to many other options like planting rice (literally backbreaking), working in a factory repeating exactly the same physical motion over and over, scavenging through garbage, etc. The broader point, though, is while people in the West bemoan outsourcing to poor countries as exploitative etc (and…

RE: your first paragraph: Yeah, I mean I sort of get it. But if I really had to choose between the mental tedium of labeling data (which I end up doing quite a bit of myself) vs. backbreaking outdoor work, or something like repetitive food service prep... I’d pick the latter every single time. At least for me, data labeling is just brutally mind numbing and draining.

have you done backbreaking outdoor work?

Re: Life of a Data Labeler

#15

Post by gexla »

king_magic wrote: post #5
9nGQluzmnq3M wrote: post #4

Earlier quoted context omitted.

Mental tedium is a relative luxury: it's still clean indoor work with a predictable income, and thus far preferable compared to many other options like planting rice (literally backbreaking), working in a factory repeating exactly the same physical motion over and over, scavenging through garbage, etc. The broader point, though, is while people in the West bemoan outsourcing to poor countries as exploitative etc (and…

RE: your first paragraph: Yeah, I mean I sort of get it. But if I really had to choose between the mental tedium of labeling data (which I end up doing quite a bit of myself) vs. backbreaking outdoor work, or something like repetitive food service prep... I’d pick the latter every single time. At least for me, data labeling is just brutally mind numbing and draining.

> But if I really had to choose between the mental tedium of labeling data (which I end up doing quite a bit of myself) vs. backbreaking outdoor work, or something like repetitive food service prep... I’d pick the latter every single time.

In developing countries, status is an important element which people in developed nations may not think about. For example, how many derogative terms can you rattle off for people who labor in the heat of the sun all day? In the U.S. we have "redneck." There are worse terms for people of other races and nationalities in the region. In the U.S. these terms are a bad joke, in the Philippines, this is life. In the Philippines, skin whitening products are a big seller because dark skin is associated with the poor and laborers (the "Mestizo" has light skin which traces back to relatively well off Spanish heritage.)

You might choose outdoor work, but you have to understand the culture to understand why their choices might be different than yours.

Re: Life of a Data Labeler

#16

Post by tastroder »

xkcd-sucks wrote: post #13

It seems present-future "AI" will encode the implicit bias of the global underclass. Which is interesting

I'm not familiar with the process of this particular company but in general that's unlikely to have that much impact. While labelling itself might have an effect, lots of biases happen earlier, namely in the selection of samples to be labelled in the first place.

If you think about bias in ML models that we've seen in recent years for example: if you let people label 980 images of old white people and 20 diverse ones that's still a heavy bias, even if they spot and label the 2% outliers correctly.

Re: Life of a Data Labeler

#17

Post by cardigan »

b_tterc_p wrote: post #3

I know nothing about this particular company. I also see no reason to assume it’s particularly bad (it could very well be above average). But I do see a lot of data around company engagement. There’s basically no chance this level of positivity is ubiquitous for them, especially when the target workforce appears to be people otherwise below the poverty line. Although pointless tedium is not the most important factor…

I'm a software engineer at Scale and the author of this blog post.

Let's talk about Venezuela for a second. ~75% of the population lost >19lb in body weight in a year according to this survey: https://www.upi.com/Top_News/World-News/2017/02/19/Venezuela... It's unbelievable that we haven't figured out how to prevent people from living like that in the 21st century. I think it's totally deplorable and honestly an affront to humanity.

You know what I think the most effective way to combat large scale poverty is? Not by working for an aid agency (we've all heard the horror stories) - instead, how about making those people economically valuable? The internet is an amazing way to reach those people - and guess what - Scale is actually doing that (evidence: these stories). If we continue to grow, we'll be doing that even more.

To be clear, this isn't the main mission of the company - but I don't see how anyone could think it isn't a great side effect. Hence the title of the blog post - positive externalities.

Re: Life of a Data Labeler

#18

Post by lopmotr »

DevKoala wrote: post #7

I don’t feel that the pat in the back is necessary when highlighting these realities of the world economic inequality. The whole article could have been published without the intense PR focus. To clarify, there is no hard numbers around the quantity of people performing this task, for how long they are employed, their average working hours per day, the median satisfaction rating, the median pay per employee, etc. It…

If they don't like it, why are they doing it? For every rational labeller, it must be better than every other option available to them. Perhaps you're saying the workers are irrational and need some outside for to stop them doing this job? Like it's an unhealthy addiction or is limiting their future opportunities?

Re: Life of a Data Labeler

#19

Post by irjustin »

cardigan wrote: post #17
b_tterc_p wrote: post #3

I know nothing about this particular company. I also see no reason to assume it’s particularly bad (it could very well be above average). But I do see a lot of data around company engagement. There’s basically no chance this level of positivity is ubiquitous for them, especially when the target workforce appears to be people otherwise below the poverty line. Although pointless tedium is not the most important factor…

I'm a software engineer at Scale and the author of this blog post. Let's talk about Venezuela for a second. ~75% of the population lost >19lb in body weight in a year according to this survey: https://www.upi.com/Top_News/World-News/2017/02/19/Venezuela... It's unbelievable that we haven't figured out how to prevent people from living like that in the 21st century. I think it's totally deplorable and honestly an affr…

I believe the OP assumed, as a company, Scale means well and that the team is potentially bringing better wages to people who don't traditionally have it.

The outright dismissal of OP's criticisms/comments doesn't allow for a discussion in that data entry is a very tedious job along with maybe there are people who are unhappy with the work but very happy with the pay (which I think is perfectly okay).

The comment reads as "I'm going to tell you off" with the `- and guess what -`.

Re: Life of a Data Labeler

#20

Post by itronitron »

cardigan wrote: post #17
b_tterc_p wrote: post #3

I know nothing about this particular company. I also see no reason to assume it’s particularly bad (it could very well be above average). But I do see a lot of data around company engagement. There’s basically no chance this level of positivity is ubiquitous for them, especially when the target workforce appears to be people otherwise below the poverty line. Although pointless tedium is not the most important factor…

I'm a software engineer at Scale and the author of this blog post. Let's talk about Venezuela for a second. ~75% of the population lost >19lb in body weight in a year according to this survey: https://www.upi.com/Top_News/World-News/2017/02/19/Venezuela... It's unbelievable that we haven't figured out how to prevent people from living like that in the 21st century. I think it's totally deplorable and honestly an affr…

I heard a talk by Edward James Olmos in which he made the point that if US companies think of themselves as socially responsible then they should pay non-US workers the same wage as they pay their US workers.

Given that all of Scale's open positions, and current FT employees presumably, are in San Francisco (and not remote), a pessimistic view would be that the data labelers are subsidizing Scale's lifestyle choices.

Post reply on HN