Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

CompileBench: Can AI Compile 22-year-old Code?

quesma.com

Post reply on HN
Print view View original

61–69 of 69 posts

Re: CompileBench: Can AI Compile 22-year-old Code?

#61

Post by pabs3 »

gregsadetsky wrote: post #22

I recently downloaded the source code for Chocolate Doom [0], and even though a ton of human labor has been put into making it cross-platform and easy to build (and that work definitely deserves to be commended!), I still couldn't build it immediately on my M1 MacBook. Asking Claude Code to build it - literally prompting it "fix whatever needs to be fixed until you get the binary to run" - and waiting ~20 minutes was…

It builds fine on Linux arm64, what changes did you need to make?

https://buildd.debian.org/status/package.php?p=chocolate-doo...

Re: CompileBench: Can AI Compile 22-year-old Code?

#62

Post by VMG »

stared wrote: post #41
sujee_dev wrote: post #31

Earlier quoted context omitted.

I am doing this now. What are your instructions in CLAUDE.md? thx

Usually are long and project-dependent, so I won't share here. But just a copy-paste of a piece of a project of mine ## Core Principles (IMPORTANT) - **NO FALLBACKS EVER** - Fail fast, fail hard. If something is missing, crash immediately - **NO SILENT DEFAULTS** - Never use default values when files/data is missing - **NO TRY/EXCEPT WRAPPING** - Let exceptions propagate for easier debugging - **PROPER TYPES ONLY** -…

This is hilarious, I have almost exactly the same prompts. Why are the LLMs so afraid of breaking changes and propagating exceptions?

Re: CompileBench: Can AI Compile 22-year-old Code?

#64

Post by turtlebits »

magicalist wrote: post #55
gregsadetsky wrote: post #22

I recently downloaded the source code for Chocolate Doom [0], and even though a ton of human labor has been put into making it cross-platform and easy to build (and that work definitely deserves to be commended!), I still couldn't build it immediately on my M1 MacBook. Asking Claude Code to build it - literally prompting it "fix whatever needs to be fixed until you get the binary to run" - and waiting ~20 minutes was…

I mean this in the nicest possible way because you were just messing around on a fun thing, but... I feel like there's a real metaphor here. 86+ people did work over two decades to maintain a cross-platform codebase and that "definitely deserves to be commended", but what "definitely felt magical" was Claude bumbling through header tweaks from compilation errors until the project compiled. And in the end what has AI…

1M devs could have worked on it. You can neither fight bit rot nor predict the future.

The point was to get it running, not solve world peace. Without AI, the problem might not have been tackled at all.

Re: CompileBench: Can AI Compile 22-year-old Code?

#65

Post by groby_b »

saltcured wrote: post #53
groby_b wrote: post #52

Earlier quoted context omitted.

"I'll try this newfangled steel constructions if they actually forge each rebar on site". You are saying that you'd trust the new and unproven technology more if it didn't rely on old and proven technology and instead reinvented everything from scratch. That's a somewhat illogical take.

Of course I was being facetious. But by "correct", I meant that it would need to be able to work through such multi-level tasks as a compiler with semantic analysis, error checking, optimization, and code generation to reliably transcribe the source code. Not just emit lorum ipsum executables.

Why? Most engineers can't do that either. That's the whole point of having tools, that you don't need to stand their and mumble about "first principles".

It's about as useful as requiring your engineers to forge computers from sand on upwards.

Re: CompileBench: Can AI Compile 22-year-old Code?

#66

Post by camel-cdr »

gregsadetsky wrote: post #22

I recently downloaded the source code for Chocolate Doom [0], and even though a ton of human labor has been put into making it cross-platform and easy to build (and that work definitely deserves to be commended!), I still couldn't build it immediately on my M1 MacBook. Asking Claude Code to build it - literally prompting it "fix whatever needs to be fixed until you get the binary to run" - and waiting ~20 minutes was…

Ok, so I tried to build chocolate doom as well (on Debian WSL):

$ git clone --depth=1 https://github.com/chocolate-doom/chocolate-doom

$ cd c*doom; ls

Ok, there is a CMakeFile.txt, so it's probably a cmake project, so:

$ cmake .

Ok, that seems to work, but three libraries are missing, SDL2_Mixer, SDL2_Net and FluidSynth, so lets install them:

$ sudo apt install libsdl2-mixer-dev libsdl2-net-dev libfluidsynth-dev

Let's try again:

$ cmake .

Works, so now for compiling:

$ cmake --build . -j $(nproc)

Build completed in a few seconds first try.

Re: CompileBench: Can AI Compile 22-year-old Code?

#67

Post by gregsadetsky »

camel-cdr wrote: post #66
gregsadetsky wrote: post #22

I recently downloaded the source code for Chocolate Doom [0], and even though a ton of human labor has been put into making it cross-platform and easy to build (and that work definitely deserves to be commended!), I still couldn't build it immediately on my M1 MacBook. Asking Claude Code to build it - literally prompting it "fix whatever needs to be fixed until you get the binary to run" - and waiting ~20 minutes was…

Ok, so I tried to build chocolate doom as well (on Debian WSL): $ git clone --depth=1 https://github.com/chocolate-doom/chocolate-doom $ cd c*doom; ls Ok, there is a CMakeFile.txt, so it's probably a cmake project, so: $ cmake . Ok, that seems to work, but three libraries are missing, SDL2_Mixer, SDL2_Net and FluidSynth, so lets install them: $ sudo apt install libsdl2-mixer-dev libsdl2-net-dev libfluidsynth-dev Let'…

I’m on macOS so sometimes things aren’t as easy :) I’ll give it another try.

Re: CompileBench: Can AI Compile 22-year-old Code?

#68

Post by saltcured »

groby_b wrote: post #65
saltcured wrote: post #53

Earlier quoted context omitted.

Of course I was being facetious. But by "correct", I meant that it would need to be able to work through such multi-level tasks as a compiler with semantic analysis, error checking, optimization, and code generation to reliably transcribe the source code. Not just emit lorum ipsum executables.

Why? Most engineers can't do that either. That's the whole point of having tools, that you don't need to stand their and mumble about "first principles". It's about as useful as requiring your engineers to forge computers from sand on upwards.

Because I'm interested in acquiring more advanced but reliable tools, not simulations of error-prone humans.

Re: CompileBench: Can AI Compile 22-year-old Code?

#69

Post by peatmoss »

calebkaiser wrote: post #37
peatmoss wrote: post #33

Though this is more "LLM uses a variety of open source tools and compilers to compile source," I do wonder about whether there will eventually be a role for transformers in compiling code. I've mentioned this before, but "sufficiently smart compiler" would be the dream here. Start with high level code or pseudo code, end up with something optimized.

There's been a decent chunk of research in this direction over the years. Michael O'Boyle is pretty active as a researcher in the space, if you're looking for stuff to read: https://www.dcs.ed.ac.uk/home/mob/

Thank you! I'll take a read.
Post reply on HN