Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We tasked Opus 4.6 using agent teams to build a C Compiler

anthropic.com

Post reply on HN
Print view View original

611–620 of 787 posts

Re: We tasked Opus 4.6 using agent teams to build a C Compiler

#611

Post by 63stack »

MaskRay wrote: post #440
silver_sun wrote: post #427

Earlier quoted context omitted.

They said it builds Linux 6.9, maybe you are trying to compile a newer version there?

git switch v6.9 The riscv build succeeded. For the x86-64 build I ran into % make O=/tmp/linux/x86 ARCH=x86_64 CC=/tmp/p/claudes-c-compiler/target/release/ccc-x86 HOSTCC=/tmp/p/claudes-c-compiler/target/release/ccc-x86 LDFLAGS=-fuse-ld=bfd LD=ld.bfd -j30 vmlinux -k make[1]: Entering directory '/tmp/linux/x86' ... CC arch/x86/platform/intel/iosf_mbi.o ccc: error: lgdtl requires memory operand AR arch/x86/platform/inte…

I feel like I could have done this in a much shorter time, for much less tokens, but still very impressive!

Re: We tasked Opus 4.6 using agent teams to build a C Compiler

#612

Post by Philpax »

AshamedCaptain wrote: post #565
Philpax wrote: post #57

Earlier quoted context omitted.

Great. Did your compiler support three different architectures (four, if you include x86 in addition to x86-64) and compile and pass the test suite for all of this software? > Projects that compile and pass their test suites include PostgreSQL (all 237 regression tests), SQLite, QuickJS, zlib, Lua, libsodium, libpng, jq, libjpeg-turbo, mbedTLS, libuv, Redis, libffi, musl, TCC, and DOOM — all using the fully standalon…

Frankly, I think you are exaggerating. My university had a course that required students to build a C compiler that could run the C subset of SPECint (which includes frigging Perl) and this was the usual 3 month class that was not expected to fill in 24h of your time, so I'd say 1 week sounds perfectly reasonable for someone already familiar. Good enough C for a shitton of projects is barely more complicated than wri…

I really, really don't think so, but you're welcome to try :-)

Re: We tasked Opus 4.6 using agent teams to build a C Compiler

#613

Post by c-linkage »

ks2048 wrote: post #116

It's cool that you can look at the git history to see what it did. Unfortunately, I do not see any of the human written prompts (?). First 10 commits, "git log --all --pretty=format:%s --reverse | head", Initial commit: empty repo structure Lock: initial compiler scaffold task Initial compiler scaffold: full pipeline for x86-64, AArch64, RISC-V Lock: implement array subscript and lvalue assignments Implement array su…

That's crazy to me. At this point, I don't even know if the git commit log would be useful to me as a human.

Maybe it's just me, but I like to be able to do both incremental testing and integration testing as I develop. This means I would start with the lexer and parser and get them tested (separately and together) before moving on to generating and validating IR.

It looks like the AI is dumping an entire compiler in one commit. I'm not even sure where I would begin to look if I were doing a bug hunt.

YMMV. I've been a solo developer for too many years. Not that I avoided working on a team, but my teams have been so small that everything gets siloed pretty quickly. Maybe life is different when more than one person works on the same application.

Re: We tasked Opus 4.6 using agent teams to build a C Compiler

#614

Post by butterNaN »

astrange wrote: post #538
itay-maman wrote: post #168

My first reaction: wow, incredible. My second reaction: still incredible, but noting that a C compiler is one of the most rigorously specified pieces of software out there. The spec is precise, the expected behavior is well-defined, and test cases are unambiguous. I'm curious how well this translates to the kind of work most of us do day-to-day where requirements are fuzzy, many edge cases are discovered on the go, a…

The C spec is certainly not formal or precise. https://www.ralfj.de/blog/2020/12/14/provenance.html Another example is that it's unclear from the standard if you can write malloc() in C.

Sure but the point OP is making is that it is still more spec'd than most real world problems

Re: We tasked Opus 4.6 using agent teams to build a C Compiler

#615

Post by brundolf »

ndesaulniers wrote: post #165

I spent a good part of my career (nearly a decade) at Google working on getting Clang to build the linux kernel. https://clangbuiltlinux.github.io/ This LLM did it in (checks notes): > Over nearly 2,000 Claude Code sessions and $20,000 in API costs It may build, but does it boot (was also a significant and distinct next milestone)? (Also, will it blend?). Looks like yes! > The 100,000-line compiler can build a bootab…

One thing people have pointed out is that well-specified (even if huge and tedious) projects are an ideal fit for AI, because the loop can be fully closed and it can test and verify the artifact by itself with certainty. Someone was saying they had it generate a rudimentary JS engine because the available test suite is so comprehensive

Not to invalidate this! But it's toward the "well-suited for AI" end of the spectrum

Re: We tasked Opus 4.6 using agent teams to build a C Compiler

#616

Post by VladVladikoff »

ndesaulniers wrote: post #165

I spent a good part of my career (nearly a decade) at Google working on getting Clang to build the linux kernel. https://clangbuiltlinux.github.io/ This LLM did it in (checks notes): > Over nearly 2,000 Claude Code sessions and $20,000 in API costs It may build, but does it boot (was also a significant and distinct next milestone)? (Also, will it blend?). Looks like yes! > The 100,000-line compiler can build a bootab…

>$20,000 of tokens. >less efficient than existing compilers

what is the ecological cost of producing this piece of software that nobody will ever use?

Re: We tasked Opus 4.6 using agent teams to build a C Compiler

#617

Post by ajross »

pcloadlett3r wrote: post #417
ajross wrote: post #390

Earlier quoted context omitted.

If it generates a booting kernel and passes the test suite at 99% it's probably good enough to use, yeah. The point isn't to replace GCC per se, it's to demonstrate that reasonably working software of equivalent complexity is within reach for $20k to solve whatever problem it is you do have.

> it's probably good enough to use, yea. Not for general purpose use, only for demo. > that reasonably working software of equivalent complexity is within reach for $20k to solve But if this can't come close to replacing GCC and can't be modified without introducing bugs then it hasn't proven this yet. I learned some new hacks from the paper and that's great and all but from my experiencing of trying to harness even…

> Not for general purpose use, only for demo.

What does that mean, though? I mean, it's already meeting a very high quality bar by booting at all and passing those tests. No, it doesn't beat existing solutions on all the checkboxes, but that's not what the demo is about.

The point being demonstrated is that if you need a "custom compiler" or something similar for your own new, greenfield requirement, you can have it at pretty-clearly-near-shippable quality in two weeks for $20k.

And if people can't smell the disruption there, I don't know what to say.

Re: We tasked Opus 4.6 using agent teams to build a C Compiler

#618

Post by ryanjshaw »

VladVladikoff wrote: post #616
ndesaulniers wrote: post #165

I spent a good part of my career (nearly a decade) at Google working on getting Clang to build the linux kernel. https://clangbuiltlinux.github.io/ This LLM did it in (checks notes): > Over nearly 2,000 Claude Code sessions and $20,000 in API costs It may build, but does it boot (was also a significant and distinct next milestone)? (Also, will it blend?). Looks like yes! > The 100,000-line compiler can build a bootab…

>$20,000 of tokens. >less efficient than existing compilers what is the ecological cost of producing this piece of software that nobody will ever use?

If you evaluate the cost/benefit in isolation? It’s net negative.

If you see this as part of a bigger picture to improve human industrial efficiency and bring us one step closer to the singularity? Most likely net positive.

Re: We tasked Opus 4.6 using agent teams to build a C Compiler

#619

Post by thefounder »

VladVladikoff wrote: post #616
ndesaulniers wrote: post #165

I spent a good part of my career (nearly a decade) at Google working on getting Clang to build the linux kernel. https://clangbuiltlinux.github.io/ This LLM did it in (checks notes): > Over nearly 2,000 Claude Code sessions and $20,000 in API costs It may build, but does it boot (was also a significant and distinct next milestone)? (Also, will it blend?). Looks like yes! > The 100,000-line compiler can build a bootab…

>$20,000 of tokens. >less efficient than existing compilers what is the ecological cost of producing this piece of software that nobody will ever use?

With that way of thinking you would just move in a cave.

Re: We tasked Opus 4.6 using agent teams to build a C Compiler

#620

Post by ryanjshaw »

PostOnce wrote: post #336
qarl wrote: post #332

Earlier quoted context omitted.

> Still a really cool project! Yeah. This test sorta definitely proves that AI is legit. Despite the millions of people still insisting it's a hoax. The fact that the optimizations aren't as good as the 40 year gcc project? Eh - I think people who focus on that are probably still in some serious denial.

It's amazing that it "works", but viability is another issue. It cost $20,000 and it worked, but it's also totally possible to spend $20,000 and have Claude shit out a pile of nonsense. You won't know until you've finished spending the money whether it will fail or not. Anthropic doesn't sell a contract that says "We'll only bill you if it works" like you can get from a bunch of humans. Do catastrophic bugs exist in…

> You won't know until you've finished spending the money whether it will fail or not.

How do you conclude that? You start off with a bunch of tests and build these things incrementally, why would you spend 20k before realizing there’s a problem?

Post reply on HN