Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We tasked Opus 4.6 using agent teams to build a C Compiler

anthropic.com

Post reply on HN
Print view View original

281–290 of 787 posts

Re: We tasked Opus 4.6 using agent teams to build a C Compiler

#281

Post by kelnos »

cheema33 wrote: post #210
Havoc wrote: post #121

Cool project, but they really could have skipped the mention of clean room. Something trained on every copyrighted thing known to mankind is the opposite of clean room

As others have pointed out, humans train on existing codebases as well. And then use that knowledge to build clean room implementations.

No they don't. One team meticulously documents and specs out what the original code does, and then a completely independent team, who has never seen the original source code, implements it.

Otherwise it's not clean-room, it's plagiarism.

Re: We tasked Opus 4.6 using agent teams to build a C Compiler

#282

Post by simonw »

kelnos wrote: post #275
NitpickLawyer wrote: post #19

This is a much more reasonable take than the cursor-browser thing. A few things that make it pretty impressive: > This was a clean-room implementation (Claude did not have internet access at any point during its development); it depends only on the Rust standard library. The 100,000-line compiler can build Linux 6.9 on x86, ARM, and RISC-V. It can also compile QEMU, FFmpeg, SQlite, postgres, redis > I started by draf…

Honestly I don't find it that impressive. I mean, it's objectively impressive that it can be done at all, but it's not impressive from the standpoint of doing stuff that nearly all real-world users will want it to do. The C specification and Linux kernel source code are undoubtedly in its training data, as are texts about compilers from a theoretical/educational perspective. Meanwhile, I'm certain most people will ne…

Are you a frequent user of coding agents?

I ask because, as someone who uses these things every day, the idea that this kind of thing only works because of similar projects in the training data doesn't fit my mental model of how they work at all.

I'm wondering if the "it's in the training data" theorists are coding agent practitioners, or if they're mainly people who don't use the tools.

Re: We tasked Opus 4.6 using agent teams to build a C Compiler

#283

Post by ndesaulniers »

chasd00 wrote: post #256
ndesaulniers wrote: post #243

Earlier quoted context omitted.

Some people care more about compile times than the performance of generated code. Perhaps even the correctness of generated code. Perhaps more so than determinism of the generated code. Different people in different contexts can have different priorities. Trying to make everyone happy can sometimes lead to making no one happy. Thus dichotomies like `-O2` vs `-Os`. EDIT (since HN is preventing me from responding): > S…

a compiler introducing bugs into code it compiles is a nightmare thankfully few have faced. The only thing worse would be a CPU bug like the legendary Pentium bug. Imagine you compile something like Postgres only to have it crash in some unpredictable way. How long do you stare at Postgres source before suspecting the compiler? What if this compiler was used to compile code in software running all over cloud stacks?…

Yeah, my current boss spent time weeding out such hardware bugs: https://arxiv.org/abs/2110.11519 (EDIT: maybe https://x.com/Tesla_AI/status/1930686196201714027 is a more relevant citation)

They found a bimodal distribution in failures over the lifetime of chips. Infant mortality was well understood. Silicon aging over time was much less well understood, and I still find surprising.

Re: We tasked Opus 4.6 using agent teams to build a C Compiler

#284

Post by fatherwavelet »

NitpickLawyer wrote: post #25
7734128 wrote: post #17

I'm sure this is impressive, but it's probably not the best test case given how many C compilers there are out there and how they presumably have been featured in the training data. This is almost like asking me to invent a path finding algorithm when I've been thought Dijkstra's and A*.

It's a bit disappointing that people are still re-hashing the same "it's in the training data" old thing from 3 years ago. It's not like any LLM could 1for1 regurgitate millions of LoC from any training set... This is not how it works. A pertinent quote from the article (which is a really nice read, I'd recommend reading it fully at least once): > Previous Opus 4 models were barely capable of producing a functional c…

At some point it becomes like someone playing a nice song on piano and then someone countering with "that is great but play a song you don't know!".

Then they start improvising and the same person counters with "what a bunch of slop, just making things up!"

Re: We tasked Opus 4.6 using agent teams to build a C Compiler

#285

Post by soulofmischief »

PunchyHamster wrote: post #134
soulofmischief wrote: post #126

Earlier quoted context omitted.

The point is that it's a probabilistic knowledge manifold, not a database.

we all know that.

Unfortunately, that doesn't seem to be the case. The person I replied to might not understand this, either.

Re: We tasked Opus 4.6 using agent teams to build a C Compiler

#286

Post by jbjbjbjb »

ndesaulniers wrote: post #165

I spent a good part of my career (nearly a decade) at Google working on getting Clang to build the linux kernel. https://clangbuiltlinux.github.io/ This LLM did it in (checks notes): > Over nearly 2,000 Claude Code sessions and $20,000 in API costs It may build, but does it boot (was also a significant and distinct next milestone)? (Also, will it blend?). Looks like yes! > The 100,000-line compiler can build a bootab…

It’s cool but there’s a good chance it’s just copying someone else’s homework albeit in an elaborate round about way.

Re: We tasked Opus 4.6 using agent teams to build a C Compiler

#287

Post by [deleted] »

ndesaulniers wrote: post #165

I spent a good part of my career (nearly a decade) at Google working on getting Clang to build the linux kernel. https://clangbuiltlinux.github.io/ This LLM did it in (checks notes): > Over nearly 2,000 Claude Code sessions and $20,000 in API costs It may build, but does it boot (was also a significant and distinct next milestone)? (Also, will it blend?). Looks like yes! > The 100,000-line compiler can build a bootab…

[deleted]

Re: We tasked Opus 4.6 using agent teams to build a C Compiler

#289

Post by [deleted] »

benjiro wrote: post #180
Havoc wrote: post #121

Cool project, but they really could have skipped the mention of clean room. Something trained on every copyrighted thing known to mankind is the opposite of clean room

Hot take: If you try to reimplement something in a clean room, its a step by step process, using your own accumulated knowledge as the basis. That knowledge that you hold in your brain, all too often is code that may have copyrights on it, from the companies you worked on. Is it any different for a LLM? The fact that the LLM is trained on more data, does not change that when you work for a company, leave it, take tha…

[deleted]

Re: We tasked Opus 4.6 using agent teams to build a C Compiler

#290

Post by bdangubic »

simonw wrote: post #282
kelnos wrote: post #275

Earlier quoted context omitted.

Honestly I don't find it that impressive. I mean, it's objectively impressive that it can be done at all, but it's not impressive from the standpoint of doing stuff that nearly all real-world users will want it to do. The C specification and Linux kernel source code are undoubtedly in its training data, as are texts about compilers from a theoretical/educational perspective. Meanwhile, I'm certain most people will ne…

Are you a frequent user of coding agents? I ask because, as someone who uses these things every day, the idea that this kind of thing only works because of similar projects in the training data doesn't fit my mental model of how they work at all. I'm wondering if the "it's in the training data" theorists are coding agent practitioners, or if they're mainly people who don't use the tools.

I am all-daily user (multiple claude max accounts). this fits my mental model mostly but not model I had before but developed with daily use. my job revolves around two core things:

1. data analysis / visualization / …

2. “is this possible? can this even be done?”

for #1 - I don’t do much anymore, for #2 I mostly do it still all “by hand” not for the lack of serious trying. so “it can do #1 1000x better than me cause it is generally solved problem(s) it is trained on while it can’t effectively do #2 otherwise” fits perfectly

Post reply on HN