Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI solves Advent of Code 2022

note89.github.io

Post reply on HN
Print view View original

71–80 of 82 posts

Re: AI solves Advent of Code 2022

#71

Post by jffhn »

lgrapenthin wrote: post #23

This day I asked it not too fundamental questions about Clojure and it was able to provide impressive, accurate answers and provide correct code examples. However if you continue the dialogue and ask it to do more advanced stuff, it will just make up stuff out of thin air. For instance it will use functions that don't exist and claim that they can be imported from packages that don't exist or don't have them. Once yo…

>The whole thing is really weird, because it's somewhat very smart and capable and incredibly stupid and dishonest at the same time.

Not so weird: this judgment could apply to a lot of humans and to whole fields of human activities, if not to the essence of life itself.

In particular it reminds me of a con man every expert could see through, but that mesmerized management with his buzzwords talk, causing an exodus of competent people and high turnover for a few years, and most likely many millions in damage.

With advanced enough AIs handling full remote jobs, this could be done on steroids, getting you a lot of income while wreaking havoc in the companies.

Re: AI solves Advent of Code 2022

#72

Post by pps »

swid wrote: post #60
pps wrote: post #48

Earlier quoted context omitted.

Does it have access to the web? For one of my questions it answered: "Unfortunately, I am unable to provide a detailed description of the education system in Poland and its changes over the last 30 years because I have limited access to information and cannot browse the internet." but I have no idea if it's not lying :)

Some posts yesterday showed that network is usually disabled, but with the right prompt, you can enable network and someone got it to like a Twitter post.

I would like to see how to achieve that. I tried asking it to translate the first paragraph of a site, but it provided more of an interpretation or summary of the article, rather than the actual text. When asked to copy the text from the website, it said that it couldn't. Additionally, when asked to provide a summary of a website that didn't exist before (created it myself yesterday), it gave a summary that was completely fictional, based on its interpretation of the URL.

Re: AI solves Advent of Code 2022

#74

Post by PartiallyTyped »

jstanley wrote: post #29
monsieurbanana wrote: post #27

Earlier quoted context omitted.

I think it's pretty different from using high-level languages. I'm not interested in a competition that would be decided before even starting by who has the best AI program.

That's why it needs to be free software. I'm not interested in a competition that is decided by who has the best Python interpreter, but since we all have the same Python interpreter that isn't a problem.

But that is missing the point.

It stops being about the most ingenious solution. It becomes a pay2win game. There is no creativity, there is no actual competition.

Re: AI solves Advent of Code 2022

#75

Post by melagonster »

jffhn wrote: post #71
lgrapenthin wrote: post #23

This day I asked it not too fundamental questions about Clojure and it was able to provide impressive, accurate answers and provide correct code examples. However if you continue the dialogue and ask it to do more advanced stuff, it will just make up stuff out of thin air. For instance it will use functions that don't exist and claim that they can be imported from packages that don't exist or don't have them. Once yo…

>The whole thing is really weird, because it's somewhat very smart and capable and incredibly stupid and dishonest at the same time. Not so weird: this judgment could apply to a lot of humans and to whole fields of human activities, if not to the essence of life itself. In particular it reminds me of a con man every expert could see through, but that mesmerized management with his buzzwords talk, causing an exodus of…

guess it actually found how human work.

Re: AI solves Advent of Code 2022

#76

Post by cl0ckt0wer »

lgrapenthin wrote: post #23

This day I asked it not too fundamental questions about Clojure and it was able to provide impressive, accurate answers and provide correct code examples. However if you continue the dialogue and ask it to do more advanced stuff, it will just make up stuff out of thin air. For instance it will use functions that don't exist and claim that they can be imported from packages that don't exist or don't have them. Once yo…

How much could you charge for packages that the AI says should exist?

Re: AI solves Advent of Code 2022

#77

Post by jstanley »

PartiallyTyped wrote: post #74
jstanley wrote: post #29

Earlier quoted context omitted.

That's why it needs to be free software. I'm not interested in a competition that is decided by who has the best Python interpreter, but since we all have the same Python interpreter that isn't a problem.

But that is missing the point. It stops being about the most ingenious solution. It becomes a pay2win game. There is no creativity, there is no actual competition.

The problems can just become more difficult to the point that creative prompt engineering is required.

This is actually a really really good thing, because it means the level of abstraction at which programmers work has just taken a big step up.

Re: AI solves Advent of Code 2022

#78

Post by iudqnolq »

OctopusLupid wrote: post #65
swid wrote: post #60

Earlier quoted context omitted.

Some posts yesterday showed that network is usually disabled, but with the right prompt, you can enable network and someone got it to like a Twitter post.

I think the “liking Twitter post” part was just a coincidental joke.

Correct. He also claimed the chatbot signed in as Grime's Twitter account to do the action, which is obviously implausible. If you look in the replies you'll see him clarify it was a joke.

Re: AI solves Advent of Code 2022

#79

Post by gnyman »

lgrapenthin wrote: post #23

This day I asked it not too fundamental questions about Clojure and it was able to provide impressive, accurate answers and provide correct code examples. However if you continue the dialogue and ask it to do more advanced stuff, it will just make up stuff out of thin air. For instance it will use functions that don't exist and claim that they can be imported from packages that don't exist or don't have them. Once yo…

I think it's just not great at Clojure, it's less popular and so there is less of it in its training data. Also Clojure seems to be kind of hard to get right.

I started trying to learn Clojure with this years advent of code, but got stuck and first tried to use ChatGPT to solve it. My impression matches yours in that it consistently produced non working code and even when told about the error it was unable to fix it.

Then I instead decided to let it use any tool or language it knows, and I'm now documenting how it's doing solving the things.

If you're interested, here is Day1 where I first tried to use it to help me solve it with clojure, but then I gave up and asked for any concise solution. So I got a working solution with `awk` https://blog.nyman.re/2022/12/02/chatgpt-does-advent.html

The second day I just let it pick anything, and it successfully solved the day2 puzzle using python which seems to be it's go-to language. https://blog.nyman.re/2022/12/03/chatgpt-does-advent.html

Re: AI solves Advent of Code 2022

#80

Post by joaogui1 »

FiberBundle wrote: post #7

I think it must have seen the solution somewhere already on the web. I find it extremely hard to believe that such a general purpose chatbot would just be able to solve programming problems. Deepmind had a paper [1] on solving programming problems a couple of months ago and they had to apply quite specialized heuristics in order to solve these problems. Obviously ChatGPT does nothing of the sorts and it just seems ex…

AlphaCode is solving much harder problems than these first few days of AoC
Post reply on HN