Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Advent of Code 2023's new AI/LLM Policy

adventofcode.com

Post reply on HN
Print view View original

31–40 of 133 posts

Re: Advent of Code 2023's new AI/LLM Policy

#31

Post by TrianguloY »

postalrat wrote: post #4

"(If you want to use AI to help you solve puzzles, I can't really stop you, but I feel like it's harder to get better at programming if you ask an AI to do the programming for you.)" I'd say that whatever "better at programming" is quickly evolving. AI is a tool like high level languages, IDEs, autocomplete or all the different types of code analyzers. How is ignoring a powerful tool like AI going to get you better a…

Using AI is basically asking your college for the solution. It helps if it's used on the right occasions, but will make you very dependent if abused. It's the same for tools, not gonna lie, but usually you need to understand how to use a tool before using it (and it's better if you know what the tool is doing).

Is the difference between telling someone to do something versus asking how to do it.

Re: Advent of Code 2023's new AI/LLM Policy

#32

Post by ps256 »

reedf1 wrote: post #6

I can't help but feel part of the meaning drops out of it when there are carve outs for tools. If an AI can do it instantly, it starts to feel like pointless busy work, isn't that why we become programmers in the first place? If your intent is to make it a pure human-keyboard-code challenge, then I would have thought this would be a great opportunity to make the underlying questions harder, or to find the edge cases…

> it starts to feel like pointless busy work

The point of AoC was always to practice something new, learn a couple of things, challenge yourself a bit, share solutions and ideas with people etc. I don't see how that's changed by the existence of LLMs or by this policy.

Re: Advent of Code 2023's new AI/LLM Policy

#33

Post by sucralose »

kstrauser wrote: post #30

That's reasonable. I might try a wholly separate challenge this year: seeing if I can talk an AI into solving each of the puzzles. I wonder how far I could get using only ChatGPT's output without any hand-coding.

Couldn't you tell it exactly what to write without technically hand-coding it? "Make an if statement checking if x is greater than 5, and if so, print y."

Re: Advent of Code 2023's new AI/LLM Policy

#34

Post by ps256 »

Art9681 wrote: post #21

Seems like they need to adapt the level of difficulty to the new world of developer tooling. An AI may be able to write a starter template and used wisely can move the project forward, but it still requires knowledge and expertize to build a system of systems even with the AI. This is an opportunity to change our expectations of developer potential. Up the stakes and innovation will soon follow. Edit: This is also an…

This entirely misses the point of AoC. We already have system-of-systems challenges - they're called work and they're not as fun as the self-contained whimsical puzzles that are AoC.

Re: Advent of Code 2023's new AI/LLM Policy

#35

Post by foooorsyth »

Art9681 wrote: post #21

Seems like they need to adapt the level of difficulty to the new world of developer tooling. An AI may be able to write a starter template and used wisely can move the project forward, but it still requires knowledge and expertize to build a system of systems even with the AI. This is an opportunity to change our expectations of developer potential. Up the stakes and innovation will soon follow. Edit: This is also an…

>Seems like they need to adapt the level of difficulty

"They" is one guy - Eric Wastl. He has a very specific style and hey may not want to change the entire style of the challenge just because some dorks want to cheat. I hope that he doesn't change it -- AoC is very approachable, especially inside the first ~15 or so days. It's a charming set of puzzles that people look forward to. I don't want it to become an elite competitive programming competition just to stay ahead of LLMs. I want to be able to partake with 1-2 hours a night after my kids go to bed.

With that said, I don't think LLMs will be helpful with some of the tougher problems. It's a hard thing to test, because I assume CoPilot would do well with previous year challenges due to the massive amount of public code available on the same problems. I don't think it'd do well on designing the IntCode VM or the modular arithmetic problem on first time seeing them without examples on GitHub. But perhaps Eric has been feeding some LLMs the unseen problems for this year and the LLMs are cracking them? I'm skeptical but perhaps they've crossed that rubicon.

Re: Advent of Code 2023's new AI/LLM Policy

#36

Post by grenoire »

postalrat wrote: post #4

"(If you want to use AI to help you solve puzzles, I can't really stop you, but I feel like it's harder to get better at programming if you ask an AI to do the programming for you.)" I'd say that whatever "better at programming" is quickly evolving. AI is a tool like high level languages, IDEs, autocomplete or all the different types of code analyzers. How is ignoring a powerful tool like AI going to get you better a…

There's a distinction between the short-term economic incentives of using AI (work gets done quicker, more code gets written, output is created faster) versus the long-term personal incentives of not using AI early on. I believe mathematics and various natural sciences benefit from having a complete understanding of the tools, their inner workings, and their potential use-cases within your head. I think programming is similar too. If you are already a skilled programmer who is able to construct complex-and-reliable (engineering incentives) systems with great accuracy and reliability, then AI will add speed to your conquests. However, if you are a beginner who is not clear on what fits where, and how a canoe is built before a cargo ship, then AI will hinder the quality of your work, and eventually put you out of a job because you're just a mere puppet to the tool.

Re: Advent of Code 2023's new AI/LLM Policy

#37

Post by HPsquared »

lsy wrote: post #22

Maybe I'm a dinosaur but I'm utterly mystified by folks who are interested in making the leaderboard or even solving AoC puzzles with AI assistance. To me it feels like going to a 5k charity run, taking a motorcycle to the finish line, and parading your gold medal around. Like, who are you impressing? And are you even proud of yourself? I understand the appeal of using the best and fastest means when there are real-w…

People can tell if you use a motorbike though. More like, performance enhancing drugs... or designing an anthropomorphic running robot that completes the race for you.

Designing such a robot would be an achievement in its own right, but not really in the spirit of the competition.

Re: Advent of Code 2023's new AI/LLM Policy

#38

Post by gedy »

lsy wrote: post #22

Maybe I'm a dinosaur but I'm utterly mystified by folks who are interested in making the leaderboard or even solving AoC puzzles with AI assistance. To me it feels like going to a 5k charity run, taking a motorcycle to the finish line, and parading your gold medal around. Like, who are you impressing? And are you even proud of yourself? I understand the appeal of using the best and fastest means when there are real-w…

There is a lot of people (and cultures) who crave status and clout, no matter the route. Cheaters in Ivy League colleges, etc.

Re: Advent of Code 2023's new AI/LLM Policy

#39

Post by Buttons840 »

lsy wrote: post #22

Maybe I'm a dinosaur but I'm utterly mystified by folks who are interested in making the leaderboard or even solving AoC puzzles with AI assistance. To me it feels like going to a 5k charity run, taking a motorcycle to the finish line, and parading your gold medal around. Like, who are you impressing? And are you even proud of yourself? I understand the appeal of using the best and fastest means when there are real-w…

Or is it like a drag race where the first vehicle across the line is celebrated?

There's two views: (1) programming as a craft and an art, a human creation, and (2) programming as a hurdle to overcome to accomplish more important things. People with both views might wish to practice.

Re: Advent of Code 2023's new AI/LLM Policy

#40

Post by SimplyUnknown »

postalrat wrote: post #11
SimplyUnknown wrote: post #5

Earlier quoted context omitted.

Maybe better programmer not so much, but I'd say the spirit of AoC is to crack a puzzle and translate the solution into code. Solving this using LLM will not help you getting better at solving puzzles, solving puzzles, or grasp the main concepts of a programming language. In that sense, ChatGPT is not a pedagogical asset in this particular instance.

Obviously they get to defined whatever better means in the context of AoC. But better at AoC does not mean a better programmer in any way.

Surely not, but if you practicing to get better at programming using AoC, LLMs are unlikely to help you.
Post reply on HN