Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We tasked Opus 4.6 using agent teams to build a C Compiler

anthropic.com

Post reply on HN
Print view View original

111–120 of 787 posts

Re: We tasked Opus 4.6 using agent teams to build a C Compiler

#111

Post by lossolo »

Philpax wrote: post #61
jcalvinowens wrote: post #47

How much of this result is effectively plagiarized open source compiler code? I don't understand how this is compelling at all: obviously it can regurgitate things that are nearly identical in capability to already existing code it was explicitly trained on... It's very telling how all these examples are all "look, we made it recreate a shitter version of a thing that already exists in the training set".

What Rust-based compiler is it plagiarising from?

Language doesn't really matter, it's not how things are mapped in the latent space. It only needs to know how to do it in one language.

Re: We tasked Opus 4.6 using agent teams to build a C Compiler

#112

Post by epolanski »

However it was achieved, building a such a complex project like a C compiler on a 20k $ budget in full autonomy is quite impressive.

Imho some commenters focus way too much on the (many, and honestly also shared by the blog post too) cons, that they forget to be genuinely impressed by the steps forward.

Re: We tasked Opus 4.6 using agent teams to build a C Compiler

#114

Post by RobMurray »

wmf wrote: post #82
NitpickLawyer wrote: post #25

Earlier quoted context omitted.

It's a bit disappointing that people are still re-hashing the same "it's in the training data" old thing from 3 years ago. It's not like any LLM could 1for1 regurgitate millions of LoC from any training set... This is not how it works. A pertinent quote from the article (which is a really nice read, I'd recommend reading it fully at least once): > Previous Opus 4 models were barely capable of producing a functional c…

In this case it's not reproducing training data verbatim but it probably is using algorithms and data structures that were learned from existing C compilers. On one hand it's good to reuse existing knowledge but such knowledge won't be available if you ask Claude to develop novel software.

How often do you need to invent novel algorithms or data structures? Most human written code is just rehashing existing ideas as well.

Re: We tasked Opus 4.6 using agent teams to build a C Compiler

#115

Post by antirez »

ben_w wrote: post #106
halxc wrote: post #99

Earlier quoted context omitted.

We all saw verbatim copies in the early LLMs. They "fixed" it by implementing filters that trigger rewrites on blatant copyright infringement. It is a research topic for heaven's sake: https://arxiv.org/abs/2504.16046

We saw partial copies of large or rare documents, and full copies of smaller widely-reproduced documents, not full copies of everything. An e.g. 1 trillion parameter model is not a lossless copy of a ten-petabyte slice of plain text from the internet. The distinction may not have mattered for copyright laws if things had gone down differently, but the gap between "blurry JPEG of the internet" and "learned stuff" is m…

Besides, the fact an LLM may recall parts of certain documents, like I can recall incipits of certain novels, does not mean that when you ask LLM of doing other kind of work, that is not recalling stuff, the LLM will mix such things verbatim. The LLM knows what it is doing in a variety of contexts, and uses the knowledge to produce stuff. The fact that for many people LLMs being able to do things that replace humans is bitter does not mean (and is not true) that this happens mainly using memorization. What coding agents can do today have zero explanation with memorization of verbatim stuff. So it's not a matter of copyright. Certain folks are fighting the wrong battle.

Re: We tasked Opus 4.6 using agent teams to build a C Compiler

#116

Post by ks2048 »

It's cool that you can look at the git history to see what it did. Unfortunately, I do not see any of the human written prompts (?).

First 10 commits, "git log --all --pretty=format:%s --reverse | head",

  Initial commit: empty repo structure
  Lock: initial compiler scaffold task
  Initial compiler scaffold: full pipeline for x86-64, AArch64, RISC-V
  Lock: implement array subscript and lvalue assignments
  Implement array subscript, lvalue assignments, and short-circuit evaluation
  Add idea: type-aware codegen for correct sized operations
  Lock: type-aware codegen for correct sized operations
  Implement type-aware codegen for correct sized operations
  Lock: implement global variable support
  Implement global variable support across all three backends

Re: We tasked Opus 4.6 using agent teams to build a C Compiler

#117

Post by boroboro4 »

ben_w wrote: post #106
halxc wrote: post #99

Earlier quoted context omitted.

We all saw verbatim copies in the early LLMs. They "fixed" it by implementing filters that trigger rewrites on blatant copyright infringement. It is a research topic for heaven's sake: https://arxiv.org/abs/2504.16046

We saw partial copies of large or rare documents, and full copies of smaller widely-reproduced documents, not full copies of everything. An e.g. 1 trillion parameter model is not a lossless copy of a ten-petabyte slice of plain text from the internet. The distinction may not have mattered for copyright laws if things had gone down differently, but the gap between "blurry JPEG of the internet" and "learned stuff" is m…

While I mostly agree with you, it worth noting modern llms are trained on 10-20-30T of tokens which is quite comparable to their size (especially given how compressible the data is)

Re: We tasked Opus 4.6 using agent teams to build a C Compiler

#118

Post by lossolo »

RobMurray wrote: post #114
wmf wrote: post #82

Earlier quoted context omitted.

In this case it's not reproducing training data verbatim but it probably is using algorithms and data structures that were learned from existing C compilers. On one hand it's good to reuse existing knowledge but such knowledge won't be available if you ask Claude to develop novel software.

How often do you need to invent novel algorithms or data structures? Most human written code is just rehashing existing ideas as well.

They're very good at reiterating, that's true. The issue is that without the people outside of "most humans" there would be no code and no civilization. We'd still be sitting in trees. That is real intelligence.

Re: We tasked Opus 4.6 using agent teams to build a C Compiler

#119

Post by jesse__ »

Marha01 wrote: post #94
gmueckl wrote: post #44

Earlier quoted context omitted.

The result is hardly a clean room implementation. It was rather a brute force attempt to decompress fuzzily stored knowledge contained within the network and it required close steering (using a big suite of tests) to get a reasonable approximation to the desired output. The compression and storage happened during the LLM training. Prove this statement wrong.

Even with 1 TB of weights (probable size of the largest state of the art models), the network is far too small to contain any significant part of the internet as compressed data, unless you really stretch the definition of data compression.

This sounds very wrong to me.

Take the C4 training dataset for example. The uncompressed, uncleaned, size of the dataset is ~6TB, and contains an exhaustive English language scrape of the public internet from 2019. The cleaned (still uncompressed) dataset is significantly less than 1TB.

I could go on, but, I think it's already pretty obvious that 1TB is more than enough storage to represent a significant portion of the internet.

Re: We tasked Opus 4.6 using agent teams to build a C Compiler

#120

Post by lossolo »

NitpickLawyer wrote: post #25
7734128 wrote: post #17

I'm sure this is impressive, but it's probably not the best test case given how many C compilers there are out there and how they presumably have been featured in the training data. This is almost like asking me to invent a path finding algorithm when I've been thought Dijkstra's and A*.

It's a bit disappointing that people are still re-hashing the same "it's in the training data" old thing from 3 years ago. It's not like any LLM could 1for1 regurgitate millions of LoC from any training set... This is not how it works. A pertinent quote from the article (which is a really nice read, I'd recommend reading it fully at least once): > Previous Opus 4 models were barely capable of producing a functional c…

They couldn't do it because they weren't fine-tuned for multi-agent workflows, which basically means they were constrained by their context window.

How many agents did they use with previous Opus? 3?

You've chosen an argument that works against you, because they actually could do that if they were trained to.

Give them the same post-training (recipes/steering) and the same datasets, and voila, they'll be capable of the same thing. What do you think is happening there? Did Anthropic inject magic ponies?

Post reply on HN