Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI has a deep understanding of how this code works

github.com

Post reply on HN
Print view View original

231–240 of 329 posts

Re: AI has a deep understanding of how this code works

#231

Post by seanmcdirmid »

thatguysaguy wrote: post #230
joelreymont wrote: post #211

Earlier quoted context omitted.

I think it's for me to redo the PR and break it into smaller pieces. There's value in the PR in that it does not require you to install the separate OxCaml fork from Jane St which doesn't work with all the OCaml packages. Or wasn't when I tried it back in August.

A big part of software engineering is maintenance not just adding features. When you drop a 22,000 line PR without any discussion or previous work on the project, people will (probably correctly) assume that you aren't there for the long haul to take care of it. On top of that, there's a huge asymmetry when people use AI to spit out huge PRs and expect thorough review from project maintainers. Of course they're not g…

AI actually has the advantage here in my experience. Yes, you can do AI wrong and tell it to just change code, write no documentation, provide no notes on the changes, and not write any tests. But you would be dumb to do it that way.

As it stands now you can set AI to do actual software development with documentation, notes, reasoning for changes, tests, and so on. It isn’t exactly easy to do this, a novice to AI and software development definitely wouldn’t set it up this way, but it isn’t what the tech can really do. There is a lot to be done in using different AI to write tests and code (well, don’t let an AI who can see the code to write the tests, or you could just get a bunch of change detector crap), but in general it mostly turns out that all the things SWEs can do to improve their work works on AI also.

Re: AI has a deep understanding of how this code works

#232

Post by joelreymont »

robertheadley wrote: post #180
yodsanklai wrote: post #170

Earlier quoted context omitted.

In my big tech company, you don't want to be dismissive of AI if you don't want to sound like a paria. It's hard to believe how much faith leadership has in AI. They really want every engineer to use AI as much as possible. Reviewing is increasingly done by AI as well. That being said, I don't think that's why reviewers here were so cordial, but this is the tone you'd expect in the corporate world.

I wouldn't say they were dismissive of AI, just that they are unwilling to merge code that they don't have the time or motivation to review. If you want AI code merged, make it small so it it's an easy review. That being said, I completely understand being unwilling to merge AI code at all.

Why would you be unwilling to merge AI code at all?

Consider my other PR against the Zig compiler [1]... I was careful to make it small and properly document it but there's a strict anti-AI policy for Zig and they closed the PR.

Why?

Is it not small? Not carefully documented? Is there no value it int?

I'm not complaining or arguing for justice. I'm genuinely interested in how people think in this instance. If the sausage looks good and tastes great, and was made observing the proper health standards, do you still care how the sausage was made?!

[1] https://github.com/joelreymont/zig/pull/1 [2] https://ziggit.dev/t/bug-wrong-segment-ordering-for-macos-us...

Re: AI has a deep understanding of how this code works

#233

Post by joelreymont »

seanmcdirmid wrote: post #231
thatguysaguy wrote: post #230

Earlier quoted context omitted.

A big part of software engineering is maintenance not just adding features. When you drop a 22,000 line PR without any discussion or previous work on the project, people will (probably correctly) assume that you aren't there for the long haul to take care of it. On top of that, there's a huge asymmetry when people use AI to spit out huge PRs and expect thorough review from project maintainers. Of course they're not g…

AI actually has the advantage here in my experience. Yes, you can do AI wrong and tell it to just change code, write no documentation, provide no notes on the changes, and not write any tests. But you would be dumb to do it that way. As it stands now you can set AI to do actual software development with documentation, notes, reasoning for changes, tests, and so on. It isn’t exactly easy to do this, a novice to AI and…

Note that this PR works, was tested, etc.

I was careful to have AI run through the examples in the PR, run lldb on the sample code and make sure the output matches.

Some of the changes didn't make it in before the PR was closed but I don't think anyone bothered to actually check the work. All the discussion focused on the inappropriateness of the huge PR itself (yes, I agree), on it being written by AI... and on the AI somehow "stealing" work code.

Re: AI has a deep understanding of how this code works

#234

Post by joelreymont »

torben-friis wrote: post #139
jodrellblank wrote: post #131

Earlier quoted context omitted.

> " It's clear some people have had their brain broken by the existence of AI. " The AI wrote code which worked, for a problem the submitter had, which had not been solved by any human for a long time, and there is limited human developer time/interest/funding available for solving it. Dumping a mass of code (and work) onto maintainers without prior discussion is the problem[1]. If they had forked the repo, patched i…

>They claim to have read the code, tested the code, they know that other people want the functionality; is wanting to share working code a "broken brain"? There is clearly a deviation between the amount of oversight the author thinks they provided and the actual level of oversight. This is clear by the fact that they couldn’t even explain the misattribution. They also mention that this is not their area of expertise.…

By testing the code I mean that I actually focused on tests passing and the output in the examples being produced by AI running lldb using this modified compiler.

Re: AI has a deep understanding of how this code works

#235

Post by joelreymont »

nsagent wrote: post #138
jodrellblank wrote: post #131

Earlier quoted context omitted.

> " It's clear some people have had their brain broken by the existence of AI. " The AI wrote code which worked, for a problem the submitter had, which had not been solved by any human for a long time, and there is limited human developer time/interest/funding available for solving it. Dumping a mass of code (and work) onto maintainers without prior discussion is the problem[1]. If they had forked the repo, patched i…

It's clear Claude adapted code directly from the OxCaml implementation (the PR author said he pointed Claude at that code [1] and then provides a ChatGPT analysis [2] that really highlights the plagiarism, but ultimately comes to the conclusion that it isn't plagiarized). Either that highlights someone who is incompetent or they are willfully being blasé. Neither bodes well for contributing code while respecting copy…

The key is that AI adapted, not stole.

It's actually capable of reasoning and generating derivative code and not just copying stuff wholesale.

See examples at the bottom of my post:

https://joel.id/ai-will-write-your-next-compiler/

Re: AI has a deep understanding of how this code works

#236

Post by menaerus »

kqr wrote: post #226
menaerus wrote: post #216

Earlier quoted context omitted.

> Do you mind showing me some examples of that? That seems so implausible to me What's so special about it that I need to show you the example?

You are claiming humans copy-and-paste copyright headers without copying the corresponding code. To prove you're correct, you only need to show one (or a few) examples of it happening. To prove you incorrect, someone would have to go through all code in existence to show the absence of the phenomenon. Hence the burden of proof is on you.

No code besides the header was copied so I am asking what is so problematic about it?

Re: AI has a deep understanding of how this code works

#237

Post by seanmcdirmid »

joelreymont wrote: post #233
seanmcdirmid wrote: post #231

Earlier quoted context omitted.

AI actually has the advantage here in my experience. Yes, you can do AI wrong and tell it to just change code, write no documentation, provide no notes on the changes, and not write any tests. But you would be dumb to do it that way. As it stands now you can set AI to do actual software development with documentation, notes, reasoning for changes, tests, and so on. It isn’t exactly easy to do this, a novice to AI and…

Note that this PR works, was tested, etc. I was careful to have AI run through the examples in the PR, run lldb on the sample code and make sure the output matches. Some of the changes didn't make it in before the PR was closed but I don't think anyone bothered to actually check the work. All the discussion focused on the inappropriateness of the huge PR itself (yes, I agree), on it being written by AI... and on the…

I’ve been finding that the documentation the AI writes isn’t so much for humans, but for the AI when it later goes to work on the code again…well, to say AI benefits from good PRs as much as people do. You could ask the AI to break up the PR next time if possible, it will probably do so much more easily than you could do it manually.

Re: AI has a deep understanding of how this code works

#238

Post by pjmlp »

menaerus wrote: post #223
pjmlp wrote: post #192

Earlier quoted context omitted.

Yes? That is why some people are forbidden to contribute to projects if their eyes have read projects with incompatible licenses, in case people go to copyright court.

Yes what? Both oxcaml and ocaml have compatible LGPL licenses so I didn't get your argument. But even if that hadn't been the case, what exactly would be the problem? Are you saying that I cannot learn from a copyrighted book written by some respected and known author, and then apply that knowledge elsewhere because I would be risking to be sued for copyright infringement?

Depends on the license of the original material, which is why they tend to have a list of allowed use cases for copying content.

Naturally there are very flexible ones, very draconian ones, and those in the middle.

Most people get away with them, because it isn't like everyone is taking others to copyright court sessions every single day, unless there are millions at play.

Re: AI has a deep understanding of how this code works

#239

Post by joelreymont »

thorn wrote: post #220

I want to contribute to Ocaml now. Code owners are so polite. They spend their time to respond with clarity and humility. And yet this guy is trying so hard to troll and abuse their time and attention.

They are super-polite! There's an issue with process, IMO, and changes taking too long to go through the pipeline. This is why Jane St forked OCaml and are maintaining their fork. They have way more money than the OCaml team at INRIA and can afford to move as fast as they want to while waiting for their changes to make it upstream (sometime or never).

Re: AI has a deep understanding of how this code works

#240

Post by joelreymont »

KurSix wrote: post #166

This is a perfect real-world illustration of Brandolini's law: the amount of energy needed to refute bullshit is an order of magnitude bigger than to produce it. The guy spent 5 minutes prompting, while Oсaml maintainers spent hours of their time politely dissecting the mess. Open Source will lose this war unless it changes the rules of engagement for contributions

Try to spin up AI, tell it to add DWARF debugging information to the OCaml tree and then spend 5 minutes prompting. Come back and let us know the results.
Post reply on HN