Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We tasked Opus 4.6 using agent teams to build a C Compiler

anthropic.com

Post reply on HN
Print view View original

731–740 of 787 posts

Re: We tasked Opus 4.6 using agent teams to build a C Compiler

#731

Post by thesz »

dcre wrote: post #725
thesz wrote: post #707

Earlier quoted context omitted.

>>> It is a very contrived under-specified prompt. No True Prompt can be such contrived and underspecified. The article about degradation is a case study (single prompt), weakest of the studies in hierarchy of knowledge. Case studies are basis for further, more rigorous studies. And author took the time to test his assumptions and presented quite clear evidence that such degradation might be present and that we shoul…

We have investigated. Millions of people are investigating all the time and finding that the coding capacity has improved dramatically over that time. A variety of very different benchmarks say the same. This one random guy’s stupid prompt says otherwise. Come on.

As far as I remember, article stated that he found same problematic behavior for many prompts, issued by him and his colleagues. The "stupid prompt" in article is for demonstration purposes.

Re: We tasked Opus 4.6 using agent teams to build a C Compiler

#732

Post by lelanthran »

qarl wrote: post #708
bee_rider wrote: post #623

Earlier quoted context omitted.

You seem to have doubled down on a bluff that was already called.

Naw. I got him to reveal himself, which was the whole point. It's amazing what you can get people to do.

> Naw. I got him to reveal himself, which was the whole point.

Reveal myself as ... a contractor agreeing to your bid?

> It's amazing what you can get people to do.

There's a ton of money now floating around in pursuit of "proving" how cost-efficient LLM coding is.

I'm sure they can spare you the $40k to put into escrow?

After all, if I don't deliver, then the AI booster community gets a huge win - highly respected ex-FAANG staff engineer with 30 years of verified dev experience could not match the cost efficiency of Claude Code.

I am taking you up on your original offer: $40k for a C compiler that does exactly what the CCC program in the video does.

Re: We tasked Opus 4.6 using agent teams to build a C Compiler

#733

Post by sarchertech »

signatoremo wrote: post #729
sarchertech wrote: post #721

Earlier quoted context omitted.

“clean room implementation” is a term of art with a specific meaning. It has no statutory definition though so you’re technically right. But it is a defense against copyright infringement because you can’t infringe on copyright without knowledge of the material. >During production, Connectix unsuccessfully attempted a Chinese wall approach to reverse engineer the BIOS, so its engineers disassembled the object code di…

Again, reverse engineering is a valid use case of clean room implementation as I posted above, so you don't have a point there. > “clean room implementation” is a term of art with a specific meaning. What is the specific meaning you are talking about? If I set out to do a clean room implementation of some software, what do I need to do specifically so that I will prevail any copyright infringement claims? The answer…

You’re getting confused because you are substituting the goal of a clean room implementation for its definition. And you are not understanding that “clean room implementation” is one specific type of reverse engineering.

The goal is to avoid copyright infringement claims. A specific clean room implementation may or may not be successful at that.

This does not mean that any reverse engineering attempt that successfully avoids copyright infringement was a clean room implementation.

A clean room implementation is a specific method of reverse engineering where one team writes a spec by reviewing the original software and the other team attempts to implement that spec. The entire point is so that the 2nd team has no knowledge of proprietary implementation details.

If the 2nd team has previously read the entire source code that defeats the entire purpose.

> That very much sounds like using GCC as an online known-good compiler oracle to compare against in this case.

Yes and that is absolutely fine to do in a clean room implementation. That’s not the part that makes this not a clean room implementation. That’s the part that makes it an attempt at reverse engineering.

Re: We tasked Opus 4.6 using agent teams to build a C Compiler

#734

Post by sarchertech »

signatoremo wrote: post #710
sarchertech wrote: post #396

Earlier quoted context omitted.

Doesn’t matter how you gain general knowledge of compiler techniques as long as you don’t have specific knowledge of the implementation of the compiler you are reverse engineering. If you have ever read the source code of the compiler you are reverse engineering, you are by definition not doing a clean room implementation.

> you are by definition not doing a clean room implementation. This makes no sense. Reverse engineering IS an application of clean room implementation. Citing Wikipedia: “Clean-room design (also known as the Chinese wall technique) is the method of copying a design by reverse engineering and then recreating it without infringing any of the copyrights associated with the original design” https://en.wikipedia.org/wiki/…

There are many ways to reverse engineer a piece of software.

A clean room implementation is one such method of reverse engineering.

A clean room implementation is always reverse engineering. Reverse engineering is not always done using a clean room method.

Re: We tasked Opus 4.6 using agent teams to build a C Compiler

#735

Post by friendzis »

ryanjshaw wrote: post #691
friendzis wrote: post #658

Earlier quoted context omitted.

Because literally no real-world non-research project starts with "we have an extremely comprehensive test suite and specification complete down to the most finite detail" and then searches for a way to turn it into code.

I’ve spent nearly 20 years working as a consultant writing software, I know that. How do you think humans solve that problem?

Typically by putting cost caps on deliverables.

Re: We tasked Opus 4.6 using agent teams to build a C Compiler

#736

Post by rhubarbtree »

qarl wrote: post #663
rhubarbtree wrote: post #607

Earlier quoted context omitted.

I didn’t say it was trivial. Just that I thought my course mates would be able to do it.

You thought your course mates would be able to write a C compiler that builds the Linux? Huh. Interesting. Like the other guy pointed out, compiler classes often get students to write toy C compilers. I think a lot of students don't understand the meaning of the word "toy". I think this thread is FULL of people like that.

If it helps, I did a PhD in computer science and went to plenty of seminars on languages, fuzz testing compilers, reviewed for conferences like PLDI. I’m not an expert but I think I know enough to say - this is conceptually within reach if a PITA.

Re: We tasked Opus 4.6 using agent teams to build a C Compiler

#737

Post by rhubarbtree »

stevejb wrote: post #693
qarl wrote: post #663

Earlier quoted context omitted.

You thought your course mates would be able to write a C compiler that builds the Linux? Huh. Interesting. Like the other guy pointed out, compiler classes often get students to write toy C compilers. I think a lot of students don't understand the meaning of the word "toy". I think this thread is FULL of people like that.

Hey! I built a Lego technic car once 20 years ago. I am fully confident that I can build an actual road worthy electric vehicle. It's just a couple of edge cases and a bit bigger right? /s

That's really helpful, actually, as you may be able to give me some other ideas for projects.

So, things you don't think I or my coursemates could do include writing a C compiler that builds a Linux kernel.

What else do you think we couldn't do? I ask because there are various projects I'll probably get to at some point.

Things on that list include (a) writing an OS microkernel and some of the other components of an OS. Don't know how far I'll take it, but certainly a working microkernel for one machine, if I have time I'll build most of the stack up to a window manager. (b) implementing an LLM training and inference stack. I don't know how close to the metal I'd go, I've done some low level CUDA a long time ago when it was very new and low-level, depends on time. I'll probably start the LLM stuff pretty soon as I'm keen to learn.

Are these also impossible? What other things would you add to the impossible list?

Re: We tasked Opus 4.6 using agent teams to build a C Compiler

#738

Post by 4corners4sides »

A compiler is another thing whose honor and pride that the models have taken from the nerds. In the past, people would debate for hours about the “dragon book” v.s. “writing interpreters” and present their cool bespoke compilers in Show HN articles. Now models can produce 100,000 lines of code over two weeks with no human intervention that actually work and can compile significant project. Which way now nerd? The models are getting better, are you?

The article has some really odd low level descriptions of bash orchestration which I suppose are important to illustrate how barebones it was. However I always feel it odd when we’re talking about agents that are lauded as borderline super intelligence and there is still low level bash being slung around – feels like we’re talking about things at the wrong level.

The point about writing extremely high quality tests reminds me a bit of the “hot mess theory of AI” (https://alignment.anthropic.com/2026/hot-mess-of-ai/) also made by anthropic where they essentially say that long horizon tasks are more likely to fall to incoherency than for a model to purposefully pursue incorrect results. This is phrased in the article as “Claude will work autonomously to solve whatever problem I give it. So it’s important that the task verifier is nearly perfect, otherwise Claude will solve the wrong problem”.

The author also observes something that I’ve realised after the initial joy of seeing an agent one shot a task wore off – for a 30 minute agent task, 25 minutes may be spent doing exploration of the environment. While it would be an offence to give a human unvetted model generated documentation and runbooks (I’m looking at you emoji ridden README.md files becoming more common across Show HN), models should commit things like this to memory for themselves to avoid repeatedly paying the “discovery tax” on every new action. Errors, hallucinations or changes cause the generated docs to fail create more busywork for the agent but agent time is less valuable than finite human life.

Re: We tasked Opus 4.6 using agent teams to build a C Compiler

#739

Post by ajross »

lionkor wrote: post #715
ajross wrote: post #606

Earlier quoted context omitted.

> Parsing is solved with yacc, bison, or sitting down and writing a recursive descent parser (works for most well designed languages you can think of). No human being writes a recursive descent parser for "Linux Kernel C" in two weeks, though. And AFAIK there's no downloadable BNF for that you can hand to an automatic generator either, you have to write it and test it and refine it. And you can't do it in two weeks.…

I wrote a couple hobby compilers. The only difficulty with C is the ambiguous syntax. Now compare the article's setup with a single senior engineer who uses an agent or two at the same time.

> I wrote a couple hobby compilers.

So did most of us, join the club. What you can't do is write such a compiler for $20k if you want to put food on the table, or do it in two weeks (what it costs to buy your time currently until AI eats your job). And let's be honest: it's not going to build something of the complexity of Linux either. Hobby compilers run hobby code. Giant decades-old source trees test edge cases like no one's business.

Re: We tasked Opus 4.6 using agent teams to build a C Compiler

#740

Post by mastermage »

galdauts wrote: post #680
friendzis wrote: post #658

Earlier quoted context omitted.

Because literally no real-world non-research project starts with "we have an extremely comprehensive test suite and specification complete down to the most finite detail" and then searches for a way to turn it into code.

Precisely. Figuring out what the specification is supposed to look like is often the hardest part.

100% agreed i use Claude often to just bounce ideas back and forth on specs i would like to create which I know will never gain traction because its either way too ambitious or too niche.

And the amount of times Claude proposes something thats completely contradictory in the same response. Or completely does a 180 after two more responses. Is ridiculous.

Post reply on HN