Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Advent of Code 2023's new AI/LLM Policy

adventofcode.com

Post reply on HN
Print view View original

121–130 of 133 posts

Re: Advent of Code 2023's new AI/LLM Policy

#121

Post by meiraleal »

beepbooptheory wrote: post #17
postalrat wrote: post #4

"(If you want to use AI to help you solve puzzles, I can't really stop you, but I feel like it's harder to get better at programming if you ask an AI to do the programming for you.)" I'd say that whatever "better at programming" is quickly evolving. AI is a tool like high level languages, IDEs, autocomplete or all the different types of code analyzers. How is ignoring a powerful tool like AI going to get you better a…

I know we have seen some recent advancements with the Large Language Models, but I'm pretty sure they still can't, say, generate a working python Flask server for my specific REST API through only some prompts, can they? Because that would be incredible.

If this isn't /s, yes you can easily do that. With tests and doc.

Re: Advent of Code 2023's new AI/LLM Policy

#122

Post by golergka »

danjoredd wrote: post #86
postalrat wrote: post #4

"(If you want to use AI to help you solve puzzles, I can't really stop you, but I feel like it's harder to get better at programming if you ask an AI to do the programming for you.)" I'd say that whatever "better at programming" is quickly evolving. AI is a tool like high level languages, IDEs, autocomplete or all the different types of code analyzers. How is ignoring a powerful tool like AI going to get you better a…

For the same reason giving a 5 year old a calculator to "learn" addition doesn't teach them math. You still need to know why shit works in order to use it in a fully educated way. Otherwise you are just "putting magic words into the text editor" and your code will be held together with bailing wire and chewing gum. For day-to-day use, as long as you know what the code is doing when you run it, its fine. But to get to…

I have been coding for almost 30 years now, and when I'm going to compete in Advent of Code I'm going to explicitly try and use AI tools because that's the new skills I need to train now.

Re: Advent of Code 2023's new AI/LLM Policy

#123

Post by HPsquared »

michaelteter wrote: post #118
dghlsakjg wrote: post #78

Earlier quoted context omitted.

If you designed and built the robot, its one thing (still not right, but interesting). If you just rented some time on someone else's robot, its just plain cheating. I would love to see teams compete, in a separate venue/ranking, based on their own model's ability to write code with minimal prompting. It would be kinda cool to see what can be done in different 'classes' of model. How far can you get with a local mode…

> I would love to see teams compete, in a separate venue/ranking, based on their own model's ability to write code with minimal prompting. Yes, an AIdvent of Code would be interesting...

I wonder if, in future, there will be specific days and seasons to commemorate AI achievements. Like, if Wall-E was a real story would we have a holiday to honor him? Maybe even just observed by the machines / operators?

Re: Advent of Code 2023's new AI/LLM Policy

#124

Post by mountaineer »

talideon wrote: post #98
mountaineer wrote: post #77

Earlier quoted context omitted.

Right, the policy seemed fine to me until it got to this part. AI can be a way to get better at programming too. When I was learning Elixir, I watched José Valim work through the AoC puzzles[0] and followed along, typing in what José had and then working to understand it. José was doing the programming but it helped me get better at programming I think. [0] https://www.twitch.tv/collections/YDM6eKu6bhV1Nw

In that case, you're not actually competing in the competition, and that kind of thing _is_ useful. However, it'd be very much against the spirit of the competition to take those answers and use them as your own, and that's really what the policy is getting at.

Agreed, that was the sentiment I was aiming for, separating the competition aspect from the getting better at programming aspect. The policy with respect to competition seems fine. The follow-on in parens to the policy was the part I was referring to:

> (If you want to use AI to help you solve puzzles, I can't really stop you, but I feel like it's harder to get better at programming if you ask an AI to do the programming for you.)

Re: Advent of Code 2023's new AI/LLM Policy

#126

Post by precompute »

lsy wrote: post #22

Maybe I'm a dinosaur but I'm utterly mystified by folks who are interested in making the leaderboard or even solving AoC puzzles with AI assistance. To me it feels like going to a 5k charity run, taking a motorcycle to the finish line, and parading your gold medal around. Like, who are you impressing? And are you even proud of yourself? I understand the appeal of using the best and fastest means when there are real-w…

https://en.wikipedia.org/wiki/Goodhart%27s_law

Re: Advent of Code 2023's new AI/LLM Policy

#128

Post by danielbln »

ps256 wrote: post #34
Art9681 wrote: post #21

Seems like they need to adapt the level of difficulty to the new world of developer tooling. An AI may be able to write a starter template and used wisely can move the project forward, but it still requires knowledge and expertize to build a system of systems even with the AI. This is an opportunity to change our expectations of developer potential. Up the stakes and innovation will soon follow. Edit: This is also an…

This entirely misses the point of AoC. We already have system-of-systems challenges - they're called work and they're not as fun as the self-contained whimsical puzzles that are AoC.

There is no reason they can't be. I don't know about you, but the system of systems challenges I deal with at work are certainly not whimsical.

Re: Advent of Code 2023's new AI/LLM Policy

#129

Post by danielbln »

avarun wrote: post #66
yunwal wrote: post #52

Earlier quoted context omitted.

> With that said, I don't think LLMs will be helpful with some of the tougher problems. It's a hard thing to test, because I assume CoPilot would do well with previous year challenges due to the massive amount of public code available on the same problems. I'm not 100% sure on this, but I think copilot isn't trained on anything past January 2022 (the gpt-3.5 and 4 cutoff dates), so I think it still shouldn't do well…

You mean September 2021, right?

It seems the cut-off has shifted from September 21 to January 22 recently.
Post reply on HN