Live data from Hacker News

Advent of Code 2023's new AI/LLM Policy

adventofcode.com

61–70 of 133 posts

Re: Advent of Code 2023's new AI/LLM Policy

#61
post #22

Maybe I'm a dinosaur but I'm utterly mystified by folks who are interested in making the leaderboard or even solving AoC puzzles with AI assistance. To me it feels like going to a 5k charity run, taking a motorcycle to the finish line, and parading your gold medal around. Like, who are you impressing? And are you even proud of yourself? I understand the appeal of using the best and fastest means when there are real-w…

Gamification is a powerful motivator. There is a leaderboard, climbing it is a reward in itself.

Re: Advent of Code 2023's new AI/LLM Policy

#62
post #6

I can't help but feel part of the meaning drops out of it when there are carve outs for tools. If an AI can do it instantly, it starts to feel like pointless busy work, isn't that why we become programmers in the first place? If your intent is to make it a pure human-keyboard-code challenge, then I would have thought this would be a great opportunity to make the underlying questions harder, or to find the edge cases…

Last year, when GPT-3 was already around, people managed to get it solve some of the earlier problems itself, but not the later problems. This makes sense, because the earlier problems are made deliberately easier, while more difficult problems are saved for later. To eliminate this "busy work" aspect would require knocking out those easier problems, which would be a loss for those who can't solve the later problems at all.

Even then, it seems like something being beatable by a computer doesn't make it pointless to do as an exercise. We still teach children to add and multiply, even though computers can do these things billions of times faster than a human can and have been able to do so for decades. We believe that being able to do these things in ones own head is valuable even if it isn't the fastest way of doing them. Besides, there are still many people who practice and enjoy chess even after computers beat humans at it, and in fact many people integrate computer chess programs into their practice and analysis of chess.

The advent of code problems are already "busy work", in the sense of not making much sense, and having numerous complexities thrown into them for the sake of it. People have still enjoyed them for the competitive aspect, or for the fun of solving a puzzle. Those aspects don't change when you introduce an AI that can solve them faster.

Re: Advent of Code 2023's new AI/LLM Policy

#63
post #22

Maybe I'm a dinosaur but I'm utterly mystified by folks who are interested in making the leaderboard or even solving AoC puzzles with AI assistance. To me it feels like going to a 5k charity run, taking a motorcycle to the finish line, and parading your gold medal around. Like, who are you impressing? And are you even proud of yourself? I understand the appeal of using the best and fastest means when there are real-w…

Why even have a leaderboard for this? At least make it anonymous.

Re: Advent of Code 2023's new AI/LLM Policy

#64
I get the sentiment from the pro LLM folks, but this is a fun thing we do once a year that is held by a few dedicated volunteers that want to keep it to be a programming competition. Using an LLM is not in the _same_ spirit as past years and if AoC wants to maintain that original feeling then I'm all for it.

Re: Advent of Code 2023's new AI/LLM Policy

#65
post #22

Maybe I'm a dinosaur but I'm utterly mystified by folks who are interested in making the leaderboard or even solving AoC puzzles with AI assistance. To me it feels like going to a 5k charity run, taking a motorcycle to the finish line, and parading your gold medal around. Like, who are you impressing? And are you even proud of yourself? I understand the appeal of using the best and fastest means when there are real-w…

> I'm utterly mystified by folks who are interested in making the leaderboard or even solving AoC puzzles with AI assistance

This reminds me of the Project Euler position on people posting their solutions publicly. The assumption is others would seek them out, get credit they don’t deserve and not learn as much.

I understand trying to prevent skewing the results, but personally it doesn’t affect me since I’m not doing it to compete in the website, I’m doing it for self edification.

I believe this actually helps me compete in the thing that really matters, getting a job, bringing high quality products to market and generally solving real problems people have.

Are LLMs starting to solve some of that? Sure. So then I want to compete for the jobs that still can’t be solved by them and involve more complicated and interesting solutions.

If people want to handicap their problem solving abilities or compete for jobs supervising boilerplate generation, all the better for me. I’ll take a hopefully more rewarding career over making the leaderboard on AoC and PE. Though I do feel sorry for their maintainers.

> Is it fun to hit over and over?

It’s probably a mixture of entertainment (I’ve seen countless apps where it looks like people just tap the screen as fast as possible without even really watching what’s going on) and imagining they’re learning something by watching answers appear (although IMO the lack of engagement leads to lack of retention; sort of like Socrates’ position on reading vs discussion).

Re: Advent of Code 2023's new AI/LLM Policy

#66
post #52

Earlier quoted context omitted.

>Seems like they need to adapt the level of difficulty "They" is one guy - Eric Wastl. He has a very specific style and hey may not want to change the entire style of the challenge just because some dorks want to cheat. I hope that he doesn't change it -- AoC is very approachable, especially inside the first ~15 or so days. It's a charming set of puzzles that people look forward to. I don't want it to become an elite…

> With that said, I don't think LLMs will be helpful with some of the tougher problems. It's a hard thing to test, because I assume CoPilot would do well with previous year challenges due to the massive amount of public code available on the same problems. I'm not 100% sure on this, but I think copilot isn't trained on anything past January 2022 (the gpt-3.5 and 4 cutoff dates), so I think it still shouldn't do well…

You mean September 2021, right?

Re: Advent of Code 2023's new AI/LLM Policy

#67
post #22

Maybe I'm a dinosaur but I'm utterly mystified by folks who are interested in making the leaderboard or even solving AoC puzzles with AI assistance. To me it feels like going to a 5k charity run, taking a motorcycle to the finish line, and parading your gold medal around. Like, who are you impressing? And are you even proud of yourself? I understand the appeal of using the best and fastest means when there are real-w…

Have you ever heard about cheaters in Counter Strike?

Re: Advent of Code 2023's new AI/LLM Policy

#68
post #22

Maybe I'm a dinosaur but I'm utterly mystified by folks who are interested in making the leaderboard or even solving AoC puzzles with AI assistance. To me it feels like going to a 5k charity run, taking a motorcycle to the finish line, and parading your gold medal around. Like, who are you impressing? And are you even proud of yourself? I understand the appeal of using the best and fastest means when there are real-w…

Because AoC has a leaderboard? Whenever you have a leaderboard of any kind, there will be people trying to get up there. It's also likely that a lot of AI users will be skilled developers too. They already know they can solve the problems, they also think they deserve the leaderboard rankings, so using AI is ok.

When everyone else is using steroids...

Re: Advent of Code 2023's new AI/LLM Policy

#69

Earlier quoted context omitted.

Lance Armstrong Foundation on the line for you...

The incentives to cheat in a professional competition are more obvious since the money you earn is related to your performance. In the case of AoC, the only objective incentive I can think about is impressing a recruiter. Maybe it's enough if you're motivated enough.

Why do people cheat in video games?

Re: Advent of Code 2023's new AI/LLM Policy

#70
post #20

"(If you want to use AI to help you solve puzzles, I can't really stop you, but I feel like it's harder to get better at programming if you ask an AI to do the programming for you.)" I'd say that whatever "better at programming" is quickly evolving. AI is a tool like high level languages, IDEs, autocomplete or all the different types of code analyzers. How is ignoring a powerful tool like AI going to get you better a…

"I feel like it's harder to get better at programming if you ask an AI to do the programming for you" I've been thinking about this for a full year now, and I'm ready to present my opinion: I think AI assistance makes it easier, not harder to get better at programming. Sure, if you're lazy you won't learn anything because you'll get the AI to do all the work for you. But if you're that lazy I imagine you wouldn't hav…

I’m sure you feel like you’re getting better by working with an AI, but make sure to test your skills without any AI help.

The ”illusion of competence” effect can be really strong, and it sneaks up on you without you noticing it.

Post reply on HN