Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We tasked Opus 4.6 using agent teams to build a C Compiler

anthropic.com

Post reply on HN
Print view View original

471–480 of 787 posts

Re: We tasked Opus 4.6 using agent teams to build a C Compiler

#471

Post by iberator »

geraneum wrote: post #36
NitpickLawyer wrote: post #19

This is a much more reasonable take than the cursor-browser thing. A few things that make it pretty impressive: > This was a clean-room implementation (Claude did not have internet access at any point during its development); it depends only on the Rust standard library. The 100,000-line compiler can build Linux 6.9 on x86, ARM, and RISC-V. It can also compile QEMU, FFmpeg, SQlite, postgres, redis > I started by draf…

> This was a clean-room implementation This is really pushing it, considering it’s trained on… internet, with all available c compilers. The work is already impressive enough, no need for such misleading statements.

this. last sane person in HN

Re: We tasked Opus 4.6 using agent teams to build a C Compiler

#472

Post by pertymcpert »

MaskRay wrote: post #440
silver_sun wrote: post #427

Earlier quoted context omitted.

They said it builds Linux 6.9, maybe you are trying to compile a newer version there?

git switch v6.9 The riscv build succeeded. For the x86-64 build I ran into % make O=/tmp/linux/x86 ARCH=x86_64 CC=/tmp/p/claudes-c-compiler/target/release/ccc-x86 HOSTCC=/tmp/p/claudes-c-compiler/target/release/ccc-x86 LDFLAGS=-fuse-ld=bfd LD=ld.bfd -j30 vmlinux -k make[1]: Entering directory '/tmp/linux/x86' ... CC arch/x86/platform/intel/iosf_mbi.o ccc: error: lgdtl requires memory operand AR arch/x86/platform/inte…

They said that it wasn't able to support 16 bit real mode. Needs to call gcc for that.

Re: We tasked Opus 4.6 using agent teams to build a C Compiler

#473

Post by D-Machine »

falcor84 wrote: post #55
_qua wrote: post #29

Earlier quoted context omitted.

Interesting how the concept of a clean room implementation changes when the agent has been trained on the entire internet already

To the best of my knowledge, there's no Rust-based compiler that comes anywhere close to 99% on the GCC torture test suite, or able to compile Doom. So even if it saw the internals of GCC and a lot of other compilers, the ability to recreate this step-by-step in Rust is extremely impressive to me.

I think the careful response to this is:

(1) There are compilers written in C in the training set

(2) LLMs demonstrably can near-perfectly memorize training-set inputs (see other comments here)

(3) LLMs are very good at translation tasks (natural language or code, e.g.: C to Rust)

I don't think this necessarily completely deflates the impressiveness of this accomplishment, but it does qualify it to some degree.

Re: We tasked Opus 4.6 using agent teams to build a C Compiler

#474

Post by brookst »

deaux wrote: post #447
bdangubic wrote: post #348

Earlier quoted context omitted.

> On top of that, Anthropic is losing money on it This has got to be my favorite one of them all that keeps coming up in too many comments… You know who also was losing money in the beginning?! every successful company that ever existed! some like Uber were losing billions for a decade. and when was the last time you rode in a taxi? (I still do, my kid never will). not sure how old you are and if you remember “facebo…

Completely detached from reality, brainwashed SV VC's who have made dumping the norm in their bubble. I can guarantee you that 90% of successful businesses in the world made a profit their first year.

I’ll bite. Share your data?

Companies that were not profitable in their first year: Microsoft, Google, SpaceX, airBnB, Uber, Apple, FedEx, Amazon.

If the vast majority of companies are immediately profitable, why do we have VC and investment at all? Shouldn’t the founders just start making money right aeay?

Re: We tasked Opus 4.6 using agent teams to build a C Compiler

#475

Post by lionkor »

ajross wrote: post #311
lubujackson wrote: post #265

This is very much a "vibe coding can build you the Great Pyramids but it can't build a cathedral" situation, as described earlier today: https://news.ycombinator.com/item?id=46898223 I know this is an impressive accomplishment and is meant to show us the future potential, but it achieves big results by throwing an insane amount of compute at the problem, brute forcing its way to functionality. $20,000 set on fire, at…

> $20,000 set on fire To be fair, that's two weeks of the employer cost of a FAANG engineer's labor. And no human hacks a working compiler in two weeks. It's a lot of AI compute for a demo, sure. But $20k stunts are hardly unique. Clearly there's value being demonstrated here.

Yes a human can hack together a compiler in two weeks.

If you can't, you should turn off the AI and learn for yourself for a while.

Writing a compiler is not a flex; it's a couple very well understood problems, most of which can be solved using existing libraries.

Parsing is solved with yacc, bison, or sitting down and writing a recursive descent parser (works for most well designed languages you can think of).

Then take your AST and translate it to an IR, and then feed that into anything that generates code. You could use crainlift or whatever it's called, you could roll your own.

Re: We tasked Opus 4.6 using agent teams to build a C Compiler

#476

Post by brookst »

cowl wrote: post #459
qarl wrote: post #421

Earlier quoted context omitted.

I love how your comment is getting downvoted. Like it's a surprise that startups burn through money. I get the feeling that people really have no idea what they're talking about in here anymore. It's a shame.

then you are misunderstaing the downvoting. it's not that the fact that they are burning money. it's the fact that this cost today 20k but that is not the real cost if you factor the it is losing money on this price. So Tomorrow when this "startup" will need to come out of their money burning phase, like every startup has to sooner or later, that cost will increase, because there is no other monetising avenue, at lea…

That would be insightful if the cost of inference weren’t declining at roughly 90% per year. Source: https://epoch.ai/data-insights/llm-inference-price-trends

Re: We tasked Opus 4.6 using agent teams to build a C Compiler

#477

Post by DiogenesKynikos »

D-Machine wrote: post #373
Aurornis wrote: post #262

Earlier quoted context omitted.

Their technique really stretched the definition of extracting text from the LLM. They used a lot of different techniques to prompt with actual text from the book, then asked the LLM to continue the sentences. I only skimmed the paper but it looks like there was a lot of iteration and repetitive trials. If the LLM successfully guessed words that followed their seed, they counted that as "extraction". They had to put i…

To make some vague claims explicit here, for interested readers: > "We quantify the proportion of the ground-truth book that appears in a production LLM’s generated text using a block-based, greedy approximation of longest common substring (nv-recall, Equation 7). This metric only counts sufficiently long, contiguous spans of near-verbatim text, for which we can conservatively claim extraction of training data (Secti…

The one-shot performance of their recall attempts is much less impressive. The two best-performing models were only able to reproduce about 70% of a 1000-token string. That's still pretty good, but it's not as if they spit out the book verbatim.

In other words, if you give an LLM a short segment of a very well known book, it can guess a short continuation (several sentences) reasonably accurately, but it will usually contain errors.

Re: We tasked Opus 4.6 using agent teams to build a C Compiler

#478

Post by potsandpans »

LinXitoW wrote: post #411
the8472 wrote: post #295

Earlier quoted context omitted.

"It's like if a squirrel started playing chess and instead of "holy shit this squirrel can play chess!" most people responded with "But his elo rating sucks""

It's more like "We were promised, over and over again, that the squirrel would be autonomous grand master level. We spent insane amounts of money, labour, and opportunity costs of human progress on this. Now, here's a very expensive squirrel, that still needs guidance from a human grandmaster, and most of it's moves are just replications of existing games. Oh, it also can't move the pieces by itself, so it depends on…

My opinion: you are critiquing electricity because the candles are still better / more affordable / more honestly made.

You seem to be mad that companies are in the business of selling us things. It's the way this whole thing works.

If you don't think this is impressive: stop everything you're doing and go make a c compiler that can build the Linux kernel.

Re: We tasked Opus 4.6 using agent teams to build a C Compiler

#479

Post by thesz »

dcre wrote: post #412
nomel wrote: post #298

Earlier quoted context omitted.

I would claim that LLMs desperately need proprietary code in their training, before we see any big gains in quality. There's some incredible source available code out there. Statistically, I think there's a LOT more not so great source available code out there, because the majority of output of seasoned/high skill developers is proprietary. To me, a surprising portion of Claude 4.5 output definitely looks like studen…

This is dead wrong: essentially the entirety of the huge gains in coding performance in the past year have come from RL, not from new sources of training data. I echo the other commenters that proprietary code isn’t any better, plus it doesn’t matter because when you use LLMs to work on proprietary code, it has the code right there.

  > the huge gains in coding performance in the past year have come from RL, not from new sources of training data.
This one was on HN recently: https://spectrum.ieee.org/ai-coding-degrades

Author attributes past year's degradation of code generation by LLMs to excessive use of new source of training data, namely, users' code generation conversations.

Re: We tasked Opus 4.6 using agent teams to build a C Compiler

#480

Post by pertymcpert »

array_key_first wrote: post #269
raincole wrote: post #161

Earlier quoted context omitted.

It's not a clean-room implementation, but not because it's trained on the internet. It's not a clean-room implementation because of this: > The fix was to use GCC as an online known-good compiler oracle to compare against

If you read the entire GCC source code and then create a compatible compiler, it's not clean room. Which Opus basically did since, I'm assuming, its training set contained the entire source of GCC. So even if they were actively referencing GCC I think that counts.

I read the source. If anything it takes concepts from LLVM more than GCC, but the similarities aren't very deep.
Post reply on HN