21–30 of 51 posts
Earlier quoted context omitted.
Good data and good evals are two legs of the 3-legged stool that a lot of AI teams are missing.
What would a product look like in this space?
I’ve resorted to building my own annotation apps.
I think this didn't age well, for HN, and it prompts some serious questions about our techbro startup culture. > Obvious but necessary: to incentivize productive work, we tie compensation to the number of characters transcribed, and assess financial penalties for failed tests (more on tests below). Penalties are priced such that subpar performance will result in little to no earnings for the labeller. So, these aren'…
An independent contractor is more likely to not be paid for meeting mutually agreed terms, not less likely.
I think this didn't age well, for HN, and it prompts some serious questions about our techbro startup culture. > Obvious but necessary: to incentivize productive work, we tie compensation to the number of characters transcribed, and assess financial penalties for failed tests (more on tests below). Penalties are priced such that subpar performance will result in little to no earnings for the labeller. So, these aren'…
> But rather, under a punishing set of Kafkaesque rules, like someone was thinking only of computer programs, oops. "Gamified", with huge negative points penalties and everything. To be under threat of not getting paid at all. I'm not defending these practices, but to share some context: One of the problems with getting workers to review ML output is it's incredibly, unbelievably boring. When the task is to review mo…
I think this didn't age well, for HN, and it prompts some serious questions about our techbro startup culture. > Obvious but necessary: to incentivize productive work, we tie compensation to the number of characters transcribed, and assess financial penalties for failed tests (more on tests below). Penalties are priced such that subpar performance will result in little to no earnings for the labeller. So, these aren'…
> But rather, under a punishing set of Kafkaesque rules, like someone was thinking only of computer programs, oops. "Gamified", with huge negative points penalties and everything. To be under threat of not getting paid at all. I'm not defending these practices, but to share some context: One of the problems with getting workers to review ML output is it's incredibly, unbelievably boring. When the task is to review mo…
Earlier quoted context omitted.
What would a product look like in this space?
There are several data labeling products on the market such as Label Studio. I’ve resorted to building my own annotation apps.
> Still, expert reviewers will occasionally disagree in their labelling. To ensure quality, an audio clip [box characters], at which point [...] Have they censored their own article?
> All labellers are either licensed pilots or controllers (or VATSIM pilots/controllers). I would think such people can make better money by actually working as a pilot or controller?
Early career professional pilots make surprisingly little money flying.
And professional pilots of all sorts often find themselves in a hotel in a city away from home with time to kill.
> All labellers are either licensed pilots or controllers (or VATSIM pilots/controllers). I would think such people can make better money by actually working as a pilot or controller?
EDIT: and that assumes perfect accuracy, the actual pay will be lower if you miss anything
That's an immediate nope for me. I don't care if I can file a dispute, unless I can resolve it then and there, I'm not going to be at the whim of some faceless escalation system, or an uninformed CS agent.
But as other commenters have warned: beware of labor laws, especially in CA/NY/MA.
I've had a front-row seat to this...our company hires + employs contract W2 and 1099 workers for the tech industry. Two years ago we started to get a ton of demand from data labeling companies and more recently foundation model cos who are doing DIY data labeling. Companies are converting 1099 workforces to W2 to avoid misclassification. Or they're trying to button up their use of 1099 to avoid being offside.