Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We tasked Opus 4.6 using agent teams to build a C Compiler

anthropic.com

Post reply on HN
Print view View original

541–550 of 787 posts

Re: We tasked Opus 4.6 using agent teams to build a C Compiler

#541

Post by astrange »

hmry wrote: post #224

If I, a human, read the source code of $THING and then later implement my own version, that's not a "clean-room" re-implementation. The whole point of "clean-room" is that no single person has access to both the original code and the new code. (That way, you can legally prove that no copyright infringement took place.) But when an AI does it, now it counts? Opus is trained on the source code of Clang, GCC, TCC, etc.…

Copyright doesn't protect ideas, it protects writing. Avoiding reading LLVM or GCC is to protect you from other kinds of IP issues, but it's not a copyright issue. The same people contribute to both projects despite their different licenses.

Re: We tasked Opus 4.6 using agent teams to build a C Compiler

#542

Post by astrange »

karmakaze wrote: post #236

I'm not particularly impressed that it can turn C into an SSA IR or assembly etc. The optimizations, however sophisticated is where anything impressive would be. Then again, we have lots of examples in the training set I would expect. C compilers are probably the most popular of all compilers. What would be more impressive is for it to have made a compiler for a well defined language that isn't very close to a popula…

> What would be more impressive is for it to have made a compiler for a well defined language that isn't very close to a popular language.

That doesn't seem difficult as long as you can translate it into a well-known IR. The Dragon Book for some reason spends all its time talking about frontend parsing, which does give you the impression it's impossible.

I agree writing compilers isn't especially difficult, but it is a lot of work and people are scared of it.

The hard part is UI - error handling and things like that.

Re: We tasked Opus 4.6 using agent teams to build a C Compiler

#543

Post by astrange »

hgs3 wrote: post #170
Rudybega wrote: post #139

Earlier quoted context omitted.

It wrote the compiler in Rust. As far as I know, there aren't any Rust based C compilers with the same capabilities. If you can find one that can compile the Linux kernel or get 99% on the GCC torture test suite, I would be quite surprised. I couldn't in a search. Maybe read the article before being so dismissive.

> As far as I know, there aren't any Rust based C compilers with the same capabilities. If you trained on a neutral representation like an AST or IR, then the source language shouldn't matter. * * I'm not familiar with how Anthropic builds their models, but training this way should nullify PL differences.

LLMs do not learn concepts perfectly across programming languages, just as they don't learn concepts perfectly across human languages.

Re: We tasked Opus 4.6 using agent teams to build a C Compiler

#544

Post by thesz »

charcircuit wrote: post #469
thesz wrote: post #466

Earlier quoted context omitted.

We are talking about compiler here and "performance" referred above is the performance of generated code. When you are optimizing a program, you have a specific part of code to improve. The part can be found with profiler. When you are optimizing a compiler generated code, you have many similar parts of code in many programs and not-so-specific part of compiler that can be improved.

Yes, performance of the generated code. You have some benchmark of using a handful of common programs going through common workflows and you measure the performance of the generated code. As tweaks are made you see how the different performance experiments effect the overall performance. Some strategies are always a win, but things like how you layout different files and functions in memory have different trade offs…

  > As tweaks are made...
  > ...how you layout different files and functions in memory have different trade offs and are hard to know up front without doing actual real world testing.
These are definitely not an algorithmic optimizations like privatization [1].

https://en.wikipedia.org/wiki/Privatization_(computer_progra...

To correctly apply privatization one has to have correct dependency analysis. This analysis uses results of many other analyses, for example, value range analysis, something like Fourier-Motzkin algorithm, etc.

So this agentic-optimized compiler has a program where privatization is not applied, what tweaks should agents apply?

Re: We tasked Opus 4.6 using agent teams to build a C Compiler

#545

Post by qarl »

lelanthran wrote: post #455
p-e-w wrote: post #445

Earlier quoted context omitted.

> I can write you an unoptimised C compiler that emits assembly for $20k You may be willing to sell your work at that price, but that’s not the market rate, to put it very mildly. Even 10 times that would be seriously lowballing in the realm of contract work, regardless of whether it’s “optimised” or not (most software isn’t).

> You may be willing to sell your work at that price, but that’s not the market rate, to put it very mildly. It is now. At any rate, this is my actual rate. I live in South Africa, and that's about 4 weeks of work for me, without an AI.

Deal. I'll pay you IF you can achieve the same level of performance. Heck, I'll double it.

You must provide the entire git history with small commits.

I won't be holding my breath.

Re: We tasked Opus 4.6 using agent teams to build a C Compiler

#546

Post by qarl »

pjmlp wrote: post #482
qarl wrote: post #438

Earlier quoted context omitted.

Oh yeah, I do. That whole internet thing was a total HOAX. I can't believe people bought into that. Can you imagine if Amazon, EBay, PayPal, or Saleforce existed today?

Well, how is your Solaris installation going? I also remember having gone into research, because there were no jobs available, and even though I was employed at the time, our salaries weren't being paid.

What does this even mean?

Re: We tasked Opus 4.6 using agent teams to build a C Compiler

#547

Post by thesz »

disgruntledphd2 wrote: post #540
thesz wrote: post #464

Earlier quoted context omitted.

> This test sorta definitely proves that AI is legit. This is an "in distribution" test. There are a lot of C compilers out there, including ones with git history, implemented from scratch. "In distribution" tests do not test generalization. The "out of distribution" test would be like "implement (self-bootstrapping, Linux kernel compatible) C compiler in J." J is different enough from C and I know of no such compile…

> This is an "in distribution" test. There are a lot of C compilers out there, including ones with git history, implemented from scratch. "In distribution" tests do not test generalization. It's still really, really impressive though. Like, economics aside this is amazing progress. I remember GPT3 not being able to hold context for more than a paragraph, we've come a long way since then. Hell, I remember bag of words…

  > It's still really, really impressive though.
Do we know how many attempts were done to create such compiler before during previous tests? Would Anthropic report on the failed attempt? Can this "really, really impressive" thing be a result of a luck?

Much like quoting Quake code almost verbatim not so long ago.

Re: We tasked Opus 4.6 using agent teams to build a C Compiler

#548

Post by embedding-shape »

iberator wrote: post #470
ndesaulniers wrote: post #165

I spent a good part of my career (nearly a decade) at Google working on getting Clang to build the linux kernel. https://clangbuiltlinux.github.io/ This LLM did it in (checks notes): > Over nearly 2,000 Claude Code sessions and $20,000 in API costs It may build, but does it boot (was also a significant and distinct next milestone)? (Also, will it blend?). Looks like yes! > The 100,000-line compiler can build a bootab…

Claude did not wrote it. you wrote it with PREVIOUS EXPERIENCE with 20.000 long commandshyellihg him exactly what to do. Real usable AI would create it with simple: 'make c compilers c99 faster than GCC'. AI usage should be banned in general. It takes jobs faster than creating new ones ..

> AI usage should be banned in general. It takes jobs faster than creating new ones ..

I don't have an strong opinion about that in either direction, but curious: Do you feel the same about everything, or is just about this specific technology? For example, should the nail gun have been forbidden if it was invented today, as one person with a nail gun could probably replace 3-4 people with normal "manual" hammers?

You feel the same about programmers who are automating others out of work without the use of AI too?

Re: We tasked Opus 4.6 using agent teams to build a C Compiler

#549

Post by socalgal2 »

Thinking about the this, while it’s a cool achievement, how useful is it really? It realizes on the fact there is a large comprehensive set of tests and a large number of available projects that can function as tests.

That situation is extremely uncommon for most development

Re: We tasked Opus 4.6 using agent teams to build a C Compiler

#550

Post by socalgal2 »

jillesvangurp wrote: post #499
btown wrote: post #26

> This was a clean-room implementation (Claude did not have internet access at any point during its development); it depends only on the Rust standard library. The 100,000-line compiler can build Linux 6.9 on x86, ARM, and RISC-V. It can also compile QEMU, FFmpeg, SQlite, postgres, redis, and has a 99% pass rate on most compiler test suites including the GCC torture test suite. It also passes the developer's ultimate…

You can use ai coding tools to create test suites, specifications, documentation, etc. And you can use them to scrutinize those, review them, criticize them, etc. Not having a test suite just means you start with creating one. Then the next question of course becomes "for what?". This indeed puts human prompters in a position where their job is to set the goals, outline the vision, ask for the right things, ask criti…

You would spend years verifying the tests actually work where as the tests for this accomplishment were already verified by humans over decades
Post reply on HN