Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We tasked Opus 4.6 using agent teams to build a C Compiler

anthropic.com

Post reply on HN
Print view View original

571–580 of 787 posts

Re: We tasked Opus 4.6 using agent teams to build a C Compiler

#571

Post by m00x »

qarl wrote: post #369
PostOnce wrote: post #336

Earlier quoted context omitted.

It's amazing that it "works", but viability is another issue. It cost $20,000 and it worked, but it's also totally possible to spend $20,000 and have Claude shit out a pile of nonsense. You won't know until you've finished spending the money whether it will fail or not. Anthropic doesn't sell a contract that says "We'll only bill you if it works" like you can get from a bunch of humans. Do catastrophic bugs exist in…

> It cost $20,000 I'm curious - do you have ANY idea what it costs to have humans write 100,000 lines of code??? You should look it up. :)

It really depends on the human and the code it outputs.

I can get my 2y old child to output 100k LoC, but it won't be very good.

Re: We tasked Opus 4.6 using agent teams to build a C Compiler

#572

Post by andrekandre »

yoz-y wrote: post #297
underdeserver wrote: post #201

Earlier quoted context omitted.

Anthropic's version is in Rust though, so at least a little different.

One thing LLMs are really good at is translation. I haven’t tried porting projects from one language to another, but it wouldn’t surprise me if they were particularly good at that too.

as someone who has done that in a professional setting, it really does work well, at least for straightforward things like data classes/initializers and average biz logic with if else statements etc... things like code annotations and other more opaque stuff like that can get more unreliable though because there are less 1-1 representations... it would be interesting to train an llm for each encountered new pattern and slowly build up a reliable conversion workflow

Re: We tasked Opus 4.6 using agent teams to build a C Compiler

#573

Post by dirkc »

somebodythere wrote: post #526
LinXitoW wrote: post #411

Earlier quoted context omitted.

It's more like "We were promised, over and over again, that the squirrel would be autonomous grand master level. We spent insane amounts of money, labour, and opportunity costs of human progress on this. Now, here's a very expensive squirrel, that still needs guidance from a human grandmaster, and most of it's moves are just replications of existing games. Oh, it also can't move the pieces by itself, so it depends on…

even a squirrel that needs guidance from a human grandmaster, is heavily inspired by existing games, and who can use Piece Mover library is incredible. 5 years ago the squirrel was just a squirrel. then it was able to make legal moves. now it can play a whole game from start to finish, with help. that is incredible

I think the post you're responding to would agree, but is trying to make the argument that it isn't worth the cost:

> spent insane amounts of money, labour, and opportunity costs of human progress on this

That said, I would 100% approve of certain people pouring all their energy into AI to rather focus on teaching squirrels chess!

Re: We tasked Opus 4.6 using agent teams to build a C Compiler

#574

Post by 7thpower »

iberator wrote: post #470
ndesaulniers wrote: post #165

I spent a good part of my career (nearly a decade) at Google working on getting Clang to build the linux kernel. https://clangbuiltlinux.github.io/ This LLM did it in (checks notes): > Over nearly 2,000 Claude Code sessions and $20,000 in API costs It may build, but does it boot (was also a significant and distinct next milestone)? (Also, will it blend?). Looks like yes! > The 100,000-line compiler can build a bootab…

Claude did not wrote it. you wrote it with PREVIOUS EXPERIENCE with 20.000 long commandshyellihg him exactly what to do. Real usable AI would create it with simple: 'make c compilers c99 faster than GCC'. AI usage should be banned in general. It takes jobs faster than creating new ones ..

Businesses do not exist to create jobs; jobs are a byproduct.

Re: We tasked Opus 4.6 using agent teams to build a C Compiler

#575

Post by lelanthran »

qarl wrote: post #564
lelanthran wrote: post #559

Earlier quoted context omitted.

> Deal. I'll pay you IF you can achieve the same level of performance. Heck, I'll double it. > You must provide the entire git history with small commits. > I won't be holding my breath. Sure; I do this often (I operate as a company because I am a contractor) - money to be held in escrow, all the usual contracts, etc. It's a big risk for you , though - the level of performance isn't stated in the linked article so a…

> the level of performance isn't stated in the linked article so a parser in Python is probably sufficient. No, you'll have to match the performance of the actual code, regardless of what happens to be written in the article. It is a C compiler written in Rust. Obviously. Your games reveal your malign intent. EDIT: And good LORD. Who writes a C compiler in python. Do you know any other languages?!?

> No, you'll have to match the performance of the actual code, regardless of what is in the article. It is a C compiler written in Rust.

Look, it's clear that you don't hire s/ware developers very much - your specs are vague and open to interpretation, and it's also clear that I do get hired often, because I pointed out that your spec isn't clear.

As far as "playing games" goes, I'm not allowing you to change your single-sentence spec which, very importantly, has "must match performance", which I shall interpret to as "performance of emitted code" and not "performance of compiler".

> Your games reveal your intent.

It should be obvious to you by know that I've done this sort of thing before. The last C compiler I wrote was 95% compliant with the (at the time, new) C99 standard, and came to around 7000LoC - 8000LoC of C89.

> EDIT: And good LORD. Who writes a C compiler in python. Do you know any other languages?!?

Many. The last language I implemented (in C99) took about two weeks after hours (so, maybe 40 hours total?), was interpreted, and was a dialect of Lisp. It's probably somewhere on Github still, and that was (IIRC) only around 2000LoC.

What you appear to not know (maybe you're new to C) is that C was specifically designed for ease of implementation.

1. It was designed to be quick and easy to implement.

2. The extensions in GCC to allow building bootable Linux images are minimal, TBH.

3. The actual 16-bit emission necessary for booting was not done by CC, but by shelling out to GCC.

4. The 100kLoC does not include the tests; it used the GCC tests.

I mean, this isn't arcane and obscure knowledge, you know. You can search the net right now and find 100s of undergrad CS projects where they implement enough of C to compile many compliant existing programs.

I'm wondering; what languages did you write an implementation for? Any that you designed and then implemented?

Re: We tasked Opus 4.6 using agent teams to build a C Compiler

#576

Post by nnevatie »

chasd00 wrote: post #227
gignico wrote: post #3

> To stress test it, I tasked 16 agents with writing a Rust-based C compiler, from scratch, capable of compiling the Linux kernel. Over nearly 2,000 Claude Code sessions and $20,000 in API costs, the agent team produced a 100,000-line compiler that can build Linux 6.9 on x86, ARM, and RISC-V. If you don't care about code quality, maintainability, readability, conformance to the specification, and performance of the c…

> If you don't care about code quality, maintainability, readability, conformance to the specification, and performance of the compiler and of the compiled code, please, give me your $20,000, I'll give you your C compiler written from scratch :) i don't know if you could. Let's say you get a check for $20k, how long will it take you to make an equivalent performing and compliant compiler? Are you going to put your li…

> Who's going to pay your bills when the $20k is gone after 3 months?

And who's going to maintain this turd the LLM pushed out? It's a cool one-shot sort of thing, but let's not pretend this is useful as a real compiler or something anyone would like to maintain, as a human.

One could keep improving one the implementation by vibing more, but I think that's just taking you to the wrong direction of the rabbit hole.

Re: We tasked Opus 4.6 using agent teams to build a C Compiler

#577

Post by disgruntledphd2 »

thesz wrote: post #547
disgruntledphd2 wrote: post #540

Earlier quoted context omitted.

> This is an "in distribution" test. There are a lot of C compilers out there, including ones with git history, implemented from scratch. "In distribution" tests do not test generalization. It's still really, really impressive though. Like, economics aside this is amazing progress. I remember GPT3 not being able to hold context for more than a paragraph, we've come a long way since then. Hell, I remember bag of words…

> It's still really, really impressive though. Do we know how many attempts were done to create such compiler before during previous tests? Would Anthropic report on the failed attempt? Can this "really, really impressive" thing be a result of a luck? Much like quoting Quake code almost verbatim not so long ago.

> Do we know how many attempts were done to create such compiler before during previous tests? Would Anthropic report on the failed attempt? Can this "really, really impressive" thing be a result of a luck?

No we don't and yeah we would expect them to only report positive results (this is both marketing and investigation). That being said, they provide all the code et al for people to review.

I do agree that an out of distribution test would be super helpful, but given that it will almost certainly fail (given what we know about LLMs) I'm not too pushed about that given that it will definitely fail.

Look, I'm pretty sceptical about AI boosting, but this is a much better attempt than the windsurf browser thing from a few months back and it's interesting to know that one can get this work.

I do note that the article doesn't talk much about all the harnesses needed to make this work, which assuming that this approach is plausible, is the kind of thing that will be needed to make demonstrations like this more useful.

Re: We tasked Opus 4.6 using agent teams to build a C Compiler

#578

Post by jaccola »

7thpower wrote: post #574
iberator wrote: post #470

Earlier quoted context omitted.

Claude did not wrote it. you wrote it with PREVIOUS EXPERIENCE with 20.000 long commandshyellihg him exactly what to do. Real usable AI would create it with simple: 'make c compilers c99 faster than GCC'. AI usage should be banned in general. It takes jobs faster than creating new ones ..

Businesses do not exist to create jobs; jobs are a byproduct.

Even that is underselling it; jobs are a necessary evil that should be minimised. If we can have more stuff with fewer people needing to spend their lives providing it, why would we NOT want that?

Re: We tasked Opus 4.6 using agent teams to build a C Compiler

#579

Post by sieep »

forty wrote: post #293

We live a wonderful time where I can spend hours and $20000 to build a C compiler which is slow and inefficient and anyway requires an existing great compiler to even work, and then neither I nor the agent has any idea on how to make it useful :D

Heres $20b in VC funding. Congrats!
Post reply on HN