Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Advent of Code 2023's new AI/LLM Policy

adventofcode.com

Post reply on HN
Print view View original

71–80 of 133 posts

Re: Advent of Code 2023's new AI/LLM Policy

#71

Post by zeroCalories »

lsy wrote: post #22

Maybe I'm a dinosaur but I'm utterly mystified by folks who are interested in making the leaderboard or even solving AoC puzzles with AI assistance. To me it feels like going to a 5k charity run, taking a motorcycle to the finish line, and parading your gold medal around. Like, who are you impressing? And are you even proud of yourself? I understand the appeal of using the best and fastest means when there are real-w…

I think this depends entirely on the competition. Athletes "cheat" by taking ped, but it's still interesting to see how far they're able to take themselves. Similarly, if an AI assisted programmer is able to solve aoc's relatively complex problems at lightning speed that's interesting too. But if the comonm understanding is that people should solve it without AI, obviously I agree ot feels shitty to go against that.

Re: Advent of Code 2023's new AI/LLM Policy

#72

Post by internetter »

adverbly wrote: post #59

I wonder how long until they drop the leaderboard entirely. Most people are not realistically shooting for a spot on the leaderboard. It adds novelty value to know that some geniuses are out there who can solve a problem in 20 minutes which takes me over 4 hours, but the main value is my private leaderboard, and the enjoyment of solving the puzzles.

Conversely, there is this whole other subsection of people who play the AOC to place on the leaderboard (I do not – too late at night). It's fascinating to see their strategies. Personally, I focus on reproducible code, whereas they just want the answer, so often they run it in the console of the AOC website. Also, they have all developed their own libraries for common problems. It's an art, and it's fascinating to watch the videos they release.

Re: Advent of Code 2023's new AI/LLM Policy

#73

Post by golergka »

lsy wrote: post #22

Maybe I'm a dinosaur but I'm utterly mystified by folks who are interested in making the leaderboard or even solving AoC puzzles with AI assistance. To me it feels like going to a 5k charity run, taking a motorcycle to the finish line, and parading your gold medal around. Like, who are you impressing? And are you even proud of yourself? I understand the appeal of using the best and fastest means when there are real-w…

> To me it feels like going to a 5k charity run, taking a motorcycle to the finish line, and parading your gold medal around.

Almost. But the crucial difference is — imagine an alternative universe, where humanity didn't have horses or any other animals or devices, even rickshas or wheels, to help moving around the world, and suddenly the first primitive cars were being invented just a couple of years ago.

Wouldn't you feel enthusiastic to use those new devices and test them everywhere?

Re: Advent of Code 2023's new AI/LLM Policy

#74

Post by koromak »

lsy wrote: post #22

Maybe I'm a dinosaur but I'm utterly mystified by folks who are interested in making the leaderboard or even solving AoC puzzles with AI assistance. To me it feels like going to a 5k charity run, taking a motorcycle to the finish line, and parading your gold medal around. Like, who are you impressing? And are you even proud of yourself? I understand the appeal of using the best and fastest means when there are real-w…

Look at how many bots infest online multiplayer games. Some portion of humans get a kick out of cheating.

Re: Advent of Code 2023's new AI/LLM Policy

#75

Post by daniel-s »

lsy wrote: post #22

Maybe I'm a dinosaur but I'm utterly mystified by folks who are interested in making the leaderboard or even solving AoC puzzles with AI assistance. To me it feels like going to a 5k charity run, taking a motorcycle to the finish line, and parading your gold medal around. Like, who are you impressing? And are you even proud of yourself? I understand the appeal of using the best and fastest means when there are real-w…

Probably the same people that cheat at online videogames.

Re: Advent of Code 2023's new AI/LLM Policy

#76

Post by foooorsyth »

simonw wrote: post #8

This looks like a good policy to me. I used ChatGPT (3.5 - 4 wasn't out yet) to help with Advent of Code last year and found it extremely useful - but I was careful not to attempt to make the leaderboard because it felt unfair to me: https://simonwillison.net/2022/Dec/5/rust-chatgpt-copilot/

Extremely useful for getting over fighting the borrow checker while learning Rust, or extremely useful for a proficient programmer in the language being used? Did ChatGPT actually solve the fundamental problems beyond ~day 15? Or just help you with the BC and parsing the data and other little boilerplate like that?

Re: Advent of Code 2023's new AI/LLM Policy

#77

Post by mountaineer »

postalrat wrote: post #4

"(If you want to use AI to help you solve puzzles, I can't really stop you, but I feel like it's harder to get better at programming if you ask an AI to do the programming for you.)" I'd say that whatever "better at programming" is quickly evolving. AI is a tool like high level languages, IDEs, autocomplete or all the different types of code analyzers. How is ignoring a powerful tool like AI going to get you better a…

Right, the policy seemed fine to me until it got to this part. AI can be a way to get better at programming too. When I was learning Elixir, I watched José Valim work through the AoC puzzles[0] and followed along, typing in what José had and then working to understand it. José was doing the programming but it helped me get better at programming I think.

[0] https://www.twitch.tv/collections/YDM6eKu6bhV1Nw

Re: Advent of Code 2023's new AI/LLM Policy

#78

Post by dghlsakjg »

HPsquared wrote: post #37
lsy wrote: post #22

Maybe I'm a dinosaur but I'm utterly mystified by folks who are interested in making the leaderboard or even solving AoC puzzles with AI assistance. To me it feels like going to a 5k charity run, taking a motorcycle to the finish line, and parading your gold medal around. Like, who are you impressing? And are you even proud of yourself? I understand the appeal of using the best and fastest means when there are real-w…

People can tell if you use a motorbike though. More like, performance enhancing drugs... or designing an anthropomorphic running robot that completes the race for you. Designing such a robot would be an achievement in its own right, but not really in the spirit of the competition.

If you designed and built the robot, its one thing (still not right, but interesting).

If you just rented some time on someone else's robot, its just plain cheating.

I would love to see teams compete, in a separate venue/ranking, based on their own model's ability to write code with minimal prompting.

It would be kinda cool to see what can be done in different 'classes' of model. How far can you get with a local model running on an 8gb card, or just using an M2 mac. How far can you get with a custom tuned chatgpt vs a programming specialized model. Etc...

Re: Advent of Code 2023's new AI/LLM Policy

#79

Post by mckn1ght »

setgree wrote: post #54
lsy wrote: post #22

Maybe I'm a dinosaur but I'm utterly mystified by folks who are interested in making the leaderboard or even solving AoC puzzles with AI assistance. To me it feels like going to a 5k charity run, taking a motorcycle to the finish line, and parading your gold medal around. Like, who are you impressing? And are you even proud of yourself? I understand the appeal of using the best and fastest means when there are real-w…

> To me it feels like going to a 5k charity run, taking a motorcycle to the finish line, and parading your gold medal around. Like, who are you impressing? And are you even proud of yourself? Weirdly enough, basically this happens all the time at races [0]. Not a motorcycle, but a runner in my local competitive scene used a bicycle to fake her strava data for a HM in 2017 [1]. I don't get it either. [0] https://www.w…

Geez, the number of support requests I’ve sent to Strava over the years asking them to clean up their data. I don’t know about Android, but iOS can fairly reliably tell the difference between walking, running, bicycling, driving… They should be able to couple this with some better data science to filter things out like deliberate cheating, forgetting to set the right activity type or stop tracking before you drive away, or glitches in eg the GPS radio that result in huge changes in location or altitude, pushing you to the top of leaderboards with impossible results.

Re: Advent of Code 2023's new AI/LLM Policy

#80

Post by zelon88 »

lsy wrote: post #22

Maybe I'm a dinosaur but I'm utterly mystified by folks who are interested in making the leaderboard or even solving AoC puzzles with AI assistance. To me it feels like going to a 5k charity run, taking a motorcycle to the finish line, and parading your gold medal around. Like, who are you impressing? And are you even proud of yourself? I understand the appeal of using the best and fastest means when there are real-w…

I agree with the sentiment, but I believe your motorcycle analogy is a bit flawed.

Bringing a motorcycle to a 5K is obviously cheating. The motorcycle is a factor of 10 faster than its closest competitor. That is not where LLM/AI is currently at. The code generated by AI is not 10 times better than human code. It is just human code cleverly regurgitated.

Comparing AI/LLM to the motorcycle is giving it too much credit. AI is more like a 500CC motorcycle vs a 300CC motorcycle.

Sure they are pretty close, and the human might win once in a while, but the AI has just enough of an advantage that is guaranteed to burn the human out and win 95+% of the time.

Post reply on HN