Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Evidence of inconsistencies in evaluation process and selection of winners

kaggle.com

Post reply on HN
Print view View original

201–210 of 338 posts

Re: Evidence of inconsistencies in evaluation process and selection of winners

#201

Post by fc417fc802 »

jliptzin wrote: post #156
nz wrote: post #63

Earlier quoted context omitted.

With the exception of _one_ company that I worked at, pretty much every[0] company was a struggle between engineering and management. Engineering wants to get the software correct, and management wants to fire-hose features into the market. Most of the time (so more than half, at least), management tends to have a compulsion to mindlessly imitate what other companies/competitors are doing, usually without prioritizat…

Management is just responding to idiot end-users. I have been on plenty of sales calls where customers ask if features X,Y,Z are available, knowing that there’s a 99% chance they’ll never need them, but they ask anyway just because they’ve heard that someone else used a feature like that at some point in the past. If it’s not, they just assume the software is inferior.

Absolutely, but I think that just circles back around to the core point made upthread that we need to address the underlying causes of that behavioral pattern. The customers are often subject to some manner of utterly moronic performance metric where ticking off a list of features without applying critical thought is strongly incentivized.

It seems to me to be a behavioral pattern with deep seated cultural roots. How many times have you found someone you were interacting with becoming frustrated or impatient when he couldn't immediately grasp a complex topic? How frequently have you witnessed that directly resulting in corners being cut in order to "just get on with things"? When some plurality of participants are either unwilling to spend the time they have, or are short on time, or both, the careless attitude proceeds to cascade through the network.

Re: Evidence of inconsistencies in evaluation process and selection of winners

#202

Post by crabmusket »

derektank wrote: post #67
quantummagic wrote: post #52

Earlier quoted context omitted.

Pragmatically speaking, a half-assed answer now, is often better than a perfect answer tomorrow.

Yeah, we live finite lives. Time is the one thing the vast majority of us aren’t getting more of. Of course speed is a priority. This isn’t a “capital” thing, it’s a fundamental part of the human experience.

The flip side of that is, why spend your finite life in a way that's not enjoyable? If the experience of constantly being under pressure to deliver, to go faster, to work more, to produce more, isn't enjoyable - then it's a waste of a life.

Speed itself isn't the priority. You've got to ask what the direction is. If you're working on a project that is meaningful to you personally (maybe because it is very meaningful to others you care about!) then you'll want it to be done.

Re: Evidence of inconsistencies in evaluation process and selection of winners

#203

Post by ryukoposting »

jagged-chisel wrote: post #37
ecshafer wrote: post #31

AI is useful. But the amount of people that are simply offloading all of their thinking to AI and blindly accepting the answer is absurd. Kaggle is most likely using ai to assess the submissions and are not using any common sense by blindly accepting the results.

I think we need to address the underlying causes of people outsourcing their thinking like that. And a big contribution is “move fast.” No one has time to read, process, and think, because The Powers That Be (capital) want their results now.

I hate the corporate assholes that rule us like anyone else, but this seems rather conspiratorial.

It's not that nobody has time to read, process, or think (that's a uniquely Silicon Valley phenomenon). It's that there's no punishment for failing to read, process, or think. Hell, it seems like your slop is just as likely to be rewarded as someone else's thoughtful work. And the punishment in this case would be... not winning?

I think the punishments for cheating the system with AI slop are far too light in a lot of domains right now. We need to change the risk-reward calculus one way or another, and failing to reward good work obviously isn't a solution.

I'm not sure what the solution looks like exactly, but I'm certain it's punitive.

Re: Evidence of inconsistencies in evaluation process and selection of winners

#204

Post by mr_toad »

People have been using brute force methods to win Kaggle competitions since the beginning (and other people have been complaining about it just as long).

At its core, ML is all about computer generated models (automated feature selection, hyper parameter tuning). Many (most?) of the models produced in Kaggle are already black boxes and have been for a long time. The model that won the Netflix prize was never used in production for that reason.

Using an LLM to generate code to generate a black box is pretty much par for the course.

Re: Evidence of inconsistencies in evaluation process and selection of winners

#206

Post by SwellJoe »

This is why I'm not wasting my time on these things (either in participating or in paying attention to the results). The noise is just too high. I'd be furious if I'd spent a bunch of time on doing actual work for something like this only to have slop win. Hopefully, the other contestants worked on stuff that is transferable to other purposes.

Re: Evidence of inconsistencies in evaluation process and selection of winners

#207

Post by quantummagic »

crabmusket wrote: post #202
derektank wrote: post #67

Earlier quoted context omitted.

Yeah, we live finite lives. Time is the one thing the vast majority of us aren’t getting more of. Of course speed is a priority. This isn’t a “capital” thing, it’s a fundamental part of the human experience.

The flip side of that is, why spend your finite life in a way that's not enjoyable? If the experience of constantly being under pressure to deliver, to go faster, to work more, to produce more, isn't enjoyable - then it's a waste of a life. Speed itself isn't the priority. You've got to ask what the direction is. If you're working on a project that is meaningful to you personally (maybe because it is very meaningful…

There is pleasure in completing things. In actually producing something, and moving on to the next thing. It isn't enjoyable to have paralysis by analysis... too afraid to take the next step, because you have less than perfect information. In the pathological case (since we never really have perfect information), there are people who never get anything done.

Re: Evidence of inconsistencies in evaluation process and selection of winners

#208

Post by Mtinie »

breezybottom wrote: post #78
Suzuran wrote: post #57

Earlier quoted context omitted.

There is also the "You aren't paid to think, you are paid to do exactly what I tell you, nothing more or less!" school of management. I'm not sure how prevalent this attitude is now but it was very common in the 90s and 2000s. The AI and the bosses that want you to use it all speak from positions of authority and confidence. That's their right, granted to them by their position. You don't speak that way because as a…

So your position is that people actually want to do more work, but their managers are forcing them to work less? I don't buy it.

I interpreted the comment you are responding to as “people want to do what they believe is the right work” rather than simply the mandated work (which they might fundamentally disagree with, directionally). When people have limited agency, apathy increases.

Re: Evidence of inconsistencies in evaluation process and selection of winners

#210

Post by bcrosby95 »

HPsquared wrote: post #74
quantummagic wrote: post #52

Earlier quoted context omitted.

Pragmatically speaking, a half-assed answer now, is often better than a perfect answer tomorrow.

Something like the time value of money. But on the other hand, a bad answer can have negative value. Although "wrong and early" is better than "wrong and late".

I don't know. There's a lot of instances where the first movers lose because they were a bit wrong and others learned from their mistakes.
Post reply on HN