Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We tasked Opus 4.6 using agent teams to build a C Compiler

anthropic.com

Post reply on HN
Print view View original

91–100 of 787 posts

Re: We tasked Opus 4.6 using agent teams to build a C Compiler

#91

Post by jcalvinowens »

modeless wrote: post #86
jcalvinowens wrote: post #81

Earlier quoted context omitted.

Being written in rust is meaningless IMHO. There is absolutely zero inherent value to something being written in rust. Sometimes it's the right tool for the job, sometimes it isn't.

It means that it's not directly copying existing C compiler code which is overwhelmingly not written in Rust. Even if your argument is that it is plagiarizing C code and doing a direct translation to Rust, that's a pretty interesting capability for it to have.

Surely you agree that directly copying existing code into a different language is still plagiarism?

I completely agree that "reweite this existing codebase into a new language" could be a very powerful tool. But the article is making much bolder claims. And the result was more limited in capability, so you can't even really claim they've achieved the rewrite skill yet.

Re: We tasked Opus 4.6 using agent teams to build a C Compiler

#92

Post by StilesCrisis »

lunar_mycroft wrote: post #64
NitpickLawyer wrote: post #25

Earlier quoted context omitted.

It's a bit disappointing that people are still re-hashing the same "it's in the training data" old thing from 3 years ago. It's not like any LLM could 1for1 regurgitate millions of LoC from any training set... This is not how it works. A pertinent quote from the article (which is a really nice read, I'd recommend reading it fully at least once): > Previous Opus 4 models were barely capable of producing a functional c…

LLMs can regurgitate almost all of the Harry Potter books, among others [0]. Clearly, these models can actually regurgitate large amounts of their training data, and reconstructing any gaps would be a lot less impressive than implementing the project truly from scratch. (I'm not claiming this is what actually happened here, just pointing out that memorization is a lot more plausible/significant than you say) [0] http…

The training data doesn't contain a Rust based C compiler that can build Linux, though.

Re: We tasked Opus 4.6 using agent teams to build a C Compiler

#93

Post by [deleted] »

jcalvinowens wrote: post #81
Philpax wrote: post #61

Earlier quoted context omitted.

What Rust-based compiler is it plagiarising from?

Being written in rust is meaningless IMHO. There is absolutely zero inherent value to something being written in rust. Sometimes it's the right tool for the job, sometimes it isn't.

[deleted]

Re: We tasked Opus 4.6 using agent teams to build a C Compiler

#94

Post by Marha01 »

gmueckl wrote: post #44
NitpickLawyer wrote: post #19

This is a much more reasonable take than the cursor-browser thing. A few things that make it pretty impressive: > This was a clean-room implementation (Claude did not have internet access at any point during its development); it depends only on the Rust standard library. The 100,000-line compiler can build Linux 6.9 on x86, ARM, and RISC-V. It can also compile QEMU, FFmpeg, SQlite, postgres, redis > I started by draf…

The result is hardly a clean room implementation. It was rather a brute force attempt to decompress fuzzily stored knowledge contained within the network and it required close steering (using a big suite of tests) to get a reasonable approximation to the desired output. The compression and storage happened during the LLM training. Prove this statement wrong.

Even with 1 TB of weights (probable size of the largest state of the art models), the network is far too small to contain any significant part of the internet as compressed data, unless you really stretch the definition of data compression.

Re: We tasked Opus 4.6 using agent teams to build a C Compiler

#95

Post by rubymamis »

Philpax wrote: post #61
jcalvinowens wrote: post #47

How much of this result is effectively plagiarized open source compiler code? I don't understand how this is compelling at all: obviously it can regurgitate things that are nearly identical in capability to already existing code it was explicitly trained on... It's very telling how all these examples are all "look, we made it recreate a shitter version of a thing that already exists in the training set".

What Rust-based compiler is it plagiarising from?

There are many, here's a simple Google search:

https://github.com/jyn514/saltwater

https://github.com/ClementTsang/rustcc

https://github.com/maekawatoshiki/rucc

Re: We tasked Opus 4.6 using agent teams to build a C Compiler

#96

Post by falloutx »

Philpax wrote: post #85
fxtentacle wrote: post #76

You could hire a reasonably skilled dev in India for a week for $1k —- or you could pay $20k in LLM tokens, spend 2 hours writing essays to explain what you want, and then get a buggy mess.

No human developer, not even Fabrice Bellard, could reproduce this specific result in a week. A subset of it, sure, but not everything this does.

just forked https://github.com/Vexu/arocc and it took me 5 seconds to complete it.

Re: We tasked Opus 4.6 using agent teams to build a C Compiler

#97

Post by dmitrygr »

sjsjsbsh wrote: post #27
dmitrygr wrote: post #4

> The generated code is not very efficient. Even with all optimizations enabled, it outputs less efficient code than GCC with all optimizations disabled. Worse than "-O0" takes skill... So then, it produced something much worse than tcc (which is better than gcc -O0), an equivalent of which one man can produce in under two weeks. So even all those tokens and dollars did not equal one man's week of work. Except the on…

> I can already feel the contracts coming to fix LLM slop like this when any company who takes this seriously needs it maintained and cannot Honest question, do you think it’d be easier to fix or rewrite from scratch? With domains I’m intimately familiar with, I’ve come very close to simply throwing the LLM code out after using it to establish some key test cases.

Rewrite is what I’ve been doing so far in such cases. Takes fewer hours

Re: We tasked Opus 4.6 using agent teams to build a C Compiler

#98

Post by jcalvinowens »

jeroenhd wrote: post #89
jcalvinowens wrote: post #47

How much of this result is effectively plagiarized open source compiler code? I don't understand how this is compelling at all: obviously it can regurgitate things that are nearly identical in capability to already existing code it was explicitly trained on... It's very telling how all these examples are all "look, we made it recreate a shitter version of a thing that already exists in the training set".

The fact it couldn't actually stick to the 16 bit ABI so it had to cheat and call out to GCC to get the system to boot says a lot. Without enough examples to copy from (despite CPU manuals being available in the training set) the approach failed. I wonder how well it'll do when you throw it a new/imaginary instruction set/CPU architecture; I bet it'll fail in similar ways.

IMHO a new architecture doesn't really make it any more interesting: there's too many examples of adding new architectures in the existing codebases. Maybe if the new machine had some bizarre novel property, I suppose, but I can't come up with a good example.

If the model were retrained without any of the existing compilers/toolchains in its training set, and it could still do something like this, that would be very compelling to me.

Re: We tasked Opus 4.6 using agent teams to build a C Compiler

#99

Post by halxc »

antirez wrote: post #88
geraneum wrote: post #36

Earlier quoted context omitted.

> This was a clean-room implementation This is really pushing it, considering it’s trained on… internet, with all available c compilers. The work is already impressive enough, no need for such misleading statements.

The LLM does not contain a verbatim copy of whatever it saw during the pre-training stage, it may remember certain over-represented parts, otherwise it has a knowledge about a lot of things but such knowledge, while about a huge amount of topics, is similar to the way you could remember things you know very well. And, indeed, if you give it access to internet or the source code of GCC and other compilers, it will imp…

We all saw verbatim copies in the early LLMs. They "fixed" it by implementing filters that trigger rewrites on blatant copyright infringement.

It is a research topic for heaven's sake:

https://arxiv.org/abs/2504.16046

Re: We tasked Opus 4.6 using agent teams to build a C Compiler

#100

Post by falloutx »

NitpickLawyer wrote: post #25
7734128 wrote: post #17

I'm sure this is impressive, but it's probably not the best test case given how many C compilers there are out there and how they presumably have been featured in the training data. This is almost like asking me to invent a path finding algorithm when I've been thought Dijkstra's and A*.

It's a bit disappointing that people are still re-hashing the same "it's in the training data" old thing from 3 years ago. It's not like any LLM could 1for1 regurgitate millions of LoC from any training set... This is not how it works. A pertinent quote from the article (which is a really nice read, I'd recommend reading it fully at least once): > Previous Opus 4 models were barely capable of producing a functional c…

They can literally print out entire books line by line.
Post reply on HN