Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Claude’s C Compiler vs. GCC

harshanu.space

Post reply on HN
Print view View original

311–320 of 377 posts

Re: Claude’s C Compiler vs. GCC

#311

Post by vidarh »

jrflowers wrote: post #47

It seems like if Anthropic released a super cool and useful _free_ utility (like a compiler, for example) that was better than existing counterparts or solved a problem that hadn’t been solved before[0] and just casually said “Here is this awesome thing that you should use every day. By the way our language model made this.” it would be incredible advertising for them. But they instead made a blog post about how it w…

They made a blog post about it because it's an amazing test of the abilities of the models to deliver a working C-compiler, even with lots of bugs and serious caveats, for $20k of tokens, without a human babysitting it.

I'd challenge anyone who are negative to this to try to achieve what they did by hand, with the same restrictions (e.g. generating full SSA form instead of just directly emitting code, capable of compiling Linux), and log their time doing it.

Having written several compilers, I'll say with some confidence that not many developers would succeed. Far fewer would succeed fast enough to compete with $20k cost. Even fewer would do that and deliver decent quality code.

Now notice the part where they've done this experiment before. This is the first time it succeeded. Give it another model iteration or two, and expect quality to increase, and price to drop.

This is the new floor.

Re: Claude’s C Compiler vs. GCC

#312

Post by vidarh »

enum wrote: post #307
vidarh wrote: post #300

Earlier quoted context omitted.

It wasn't given gcc source code, and was not given internet access. It the extent it could translate gcc source code, it'd need to be able to recall all of the gcc source from its weights.

Right. And the arguably simpler problem, where the model gets the C code directly, is active research: https://www.darpa.mil/research/programs/translating-all-c-to... All of this work is extraordinarily impressive. It is hard to predict the impact of any single research project the week it is released. I doubt we'll ever throw away GCC/LLVM. But, I'd be surprised if the Claude C Compiler didn't have long-term impact…

I occasionally - when I have tokens to spare, a MAX subscription only lasts so far - have Claude working on my Ruby compiler. Far harder language to AOT compile (or even parse correctly). And even 6 months ago it was astounding how well it'd work, even without what I now know about good harnesses...

I think that is the biggest outcome of this: The notes on the orchestration and validation setup they used were far more interesting than the compiler itself. That orchestration setup is already somewhat quaint, but it's still far more advanced than what most AI users use.

Re: Claude’s C Compiler vs. GCC

#314

Post by troupo »

kleiba wrote: post #295
troupo wrote: post #208

Earlier quoted context omitted.

> but I cannot see any impoliteness here whatsoever. Ah yes. "It's AI I don't care" and "AI has very deep reasoning about code, prove me wrong" are the height of politeness.

Well, I wouldn't necessarily call it "going out of your way to be accommodating", but impolite is just not the word I'd choose to characterize it. I can see why others might but it's just my personal feeling that I don't think that this is the correct adjective here. That said, I don't feel like this topic is important enough to go on about it, I probably spend enough keystrokes on it already.

That's how politicians and passive-aggressive people hide their rudeness/impoliteness: under a veneer of polite-sounding phrases.

But I guess we could've used a few other synonyms: https://www.merriam-webster.com/thesaurus/impolite inconsiderate, thoughtless, impertinent (2nd meaning) :)

Re: Claude’s C Compiler vs. GCC

#315

Post by nopurpose »

direwolf20 wrote: post #259
nopurpose wrote: post #169

Earlier quoted context omitted.

You could make same argument in "information superhighway" days, but it turned out to be the opposite: no company monopolised internet services, despite trying hard. With so many companies in AI race it is already pretty competitive landscape and it doesnt seem likely to me that any of them can build deep enough moat to come ahead.

Internet services have been centralised into a few ISPs and a few websites everyone visits

a few? all sorts of websites and services are thriving on the Internet even after significant consolidation of attention social media caused. Not even close to a dystopian picture parent comment paints.

Re: Claude’s C Compiler vs. GCC

#316

Post by gordonhart »

bgirard wrote: post #30
marcus_holmes wrote: post #8

I think this is a great example of both points of view in the ongoing debate. Pro-LLM coding agents: look! a working compiler built in a few hours by an agent! this is amazing! Anti-LLM coding agents: it's not a working compiler, though. And it doesn't matter how few hours it took, because it doesn't work. It's useless. Pro: Sure, but we can get the agent to fix that. Anti: Can you, though? We've seen that the more c…

This to me sounds a lot like the SpaceX conversation: - Ohh look it can [write small function / do a small rocket hop] but it can't [ write a compiler / get to orbit]! - Ohh look it can [write a toy compiler / get to orbit] but it can't [compile linux / be reusable] - Ohh look it can [compile linux / get reusable orbital rocket] but it can't [build a compiler that rivals GCC / turn the rockets around fast enough] - T…

The difference I see is that, after "get to orbit", the goalposts for SpaceX are things that have never been done before, whereas for LLMs the goalposts are all things that skilled humans have been able to do for decades.

Re: Claude’s C Compiler vs. GCC

#317

Post by sungho_ »

dataflow wrote: post #34
marcus_holmes wrote: post #8

I think this is a great example of both points of view in the ongoing debate. Pro-LLM coding agents: look! a working compiler built in a few hours by an agent! this is amazing! Anti-LLM coding agents: it's not a working compiler, though. And it doesn't matter how few hours it took, because it doesn't work. It's useless. Pro: Sure, but we can get the agent to fix that. Anti: Can you, though? We've seen that the more c…

> Pro: Sure, maybe now, but the next generation will fix it. Do we need a c2 wiki page for "sufficiently smart LLM" like we do for https://wiki.c2.com/?SufficientlySmartCompiler ?

> This is still true - a compiler can never win this battle. All a human programmer has to do is take the output of the compiler and make a single optimisation, and he/she wins. This is the advantage that the human has - they can use any of a wide variety of tools at their disposal (including the compiler), whilst the compiler can only do what it was programmed. The best the compiler can hope for is a tie.

This is awkward

Re: Claude’s C Compiler vs. GCC

#318

Post by rayiner »

I don't understand how this isn't a bigger deal. Why are people are quibbling about how it isn't a particularly good C compiler. It seems earth shattering that an AI can write a C compiler in the first place.

Am I just old? "How did they fit those people into the television?!"

Re: Claude’s C Compiler vs. GCC

#319

Post by aurareturn »

layer8 wrote: post #279
aurareturn wrote: post #275

Earlier quoted context omitted.

Maybe I can't. But that's also why I'm invested into AI companies right now. Plan: 1. Keep my job for as long as I can by leveraging current AI tools to the best of my ability while my non-AI user colleagues lose earnings power or their job 2. Invest my money into AI companies and/or S&P500 3. If I'm truly rendered useless, live off of the investment You got any better ideas? How would you do this?

I believe that I’ll keep enough of an edge in my job that I’ll continue to be employed. (At present I still have zero pressure to use AI, though I do use it.) It’s of course possible for that to turn out to be wrong, but in that case I also see no chance to start a business, and society will be in a lot of trouble.

  I believe that I’ll keep enough of an edge in my job that I’ll continue to be employed.
I don't believe I can. If I don't use AI tools at work, I'm for sure going to get fired and/or passed up for promotions.

Re: Claude’s C Compiler vs. GCC

#320

Post by yourapostasy »

jeffreygoesto wrote: post #100
bgirard wrote: post #30

Earlier quoted context omitted.

This to me sounds a lot like the SpaceX conversation: - Ohh look it can [write small function / do a small rocket hop] but it can't [ write a compiler / get to orbit]! - Ohh look it can [write a toy compiler / get to orbit] but it can't [compile linux / be reusable] - Ohh look it can [compile linux / get reusable orbital rocket] but it can't [build a compiler that rivals GCC / turn the rockets around fast enough] - T…

All these people that built GCC and evolved the language did not have the end result in their training set. They invented it. They extrapolated from earlier experiences and knowledge, LLMs only ever accidentally stumble into "between unknown manifolds" when the temperature is high enough, they interpolate with noise (in so many senses). The people building GCC together did not only solve a to technical problem. They…

> LLMs are merely copying these decisions.

This I strongly suspect is the crux of the boundaries of their current usefulness. Without accompanying legibility/visibility into the lineage of those decisions, LLM's will be unable to copy the reasoning behind the "why", missing out on a pile of context that I'm guessing is necessary (just like with people) to come up to speed on the decision flow going forward as the mathematical space for the gradient descent to traverse gets both bigger and more complex.

We're already seeing glimmers of this as the frontier labs are reporting that explaining the "why" behind prompts is getting better results in a non-trivial number of cases.

I wonder whether we're barely scratching the surface of just how powerful natural language is.

Post reply on HN