391–400 of 787 posts
Earlier quoted context omitted.
Do you think this was guided by a low quality Anthropic developer? You can give a developer the GCC test suite and have them build the compiler backwards, which is how this was done. They literally brute forced it, most developers can brute force. It also literally uses GCC in the background... Maybe try reading the article.
[flagged]
Earlier quoted context omitted.
> Still a really cool project! Yeah. This test sorta definitely proves that AI is legit. Despite the millions of people still insisting it's a hoax. The fact that the optimizations aren't as good as the 40 year gcc project? Eh - I think people who focus on that are probably still in some serious denial.
It's amazing that it "works", but viability is another issue. It cost $20,000 and it worked, but it's also totally possible to spend $20,000 and have Claude shit out a pile of nonsense. You won't know until you've finished spending the money whether it will fail or not. Anthropic doesn't sell a contract that says "We'll only bill you if it works" like you can get from a bunch of humans. Do catastrophic bugs exist in…
Earlier today, I couldn't get opus to replace useEffect-triggered-redux-dispatch nonsense with react-query calls. I already had a very nice react-query wrapper with tons of examples. But it just couldn't make sense of the useEffect rube goldberg machine.
To be fair, it was a pretty horrible mess of useEffects. But just another data point.
Also I was hoping opus would finally be able to handle complex typescript generics, but alas...
Earlier quoted context omitted.
100k lines of clean, bug free, optimized, and vulnerability free code or 100k lines of outsourced slop? Two very different price points.
A compiler that can build linux. That level of quality should be sufficient. Do you know any low quality programmers that write C compilers in rust THAT CAN BUILD LINUX? No you don't. They do not exist.
Earlier quoted context omitted.
Every S-curve looks like an exponential until you hit the bend.
Cool I guess. Kind of a meaningless statement yeah? Let's hit the bend, then we'll talk. Until then repeating, 'It's an S Curve guys and what's more, we're near the bend! trust me" ad infinitum is pointless. It's not some wise revelation lol.
History has a way of being surprisingly boring, so personally I'm not betting on the world order being transformed in five years, but I also have to take my own advice and take things a day at a time.
Earlier quoted context omitted.
"clean room" usually means "without looking at the source code" of other similar projects. But presumably the AIs training data would have included GCC, Clang, and probably a dozen other C compilers.
Suppose you the human are working on a clean room implementation of C compiler, how do you go about doing it? Will you need to know about: a) the C language, and b) the inner working of a compiler? How did you acquire that knowledge?
If you have ever read the source code of the compiler you are reverse engineering, you are by definition not doing a clean room implementation.
Hard to find fully specified problems like this in the wild.
I think this is more a testament to small, well-written tests than it is agent teams. I imagine you could do the same thing with any frontier model and a single agent in a linear flow.
I don’t know why people use parallel agents and increase accidental complexity. Isn’t one agent fast enough? Why lose accuracy over +- one week to write a compiler?
> Write extremely high-quality tests
> Claude will work autonomously to solve whatever problem I give it. So it’s important that the task verifier is nearly perfect, otherwise Claude will solve the wrong problem. Improving the testing harness required finding high-quality compiler test suites, writing verifiers and build scripts for open-source software packages, and watching for mistakes Claude was making, then designing new tests as I identified those failure modes.
> For example, near the end of the project, Claude started to frequently break existing functionality each time it implemented a new feature. To address this, I built a continuous integration pipeline and implemented stricter enforcement that allowed Claude to better test its work so that new commits can’t break existing code.
This is a much more reasonable take than the cursor-browser thing. A few things that make it pretty impressive: > This was a clean-room implementation (Claude did not have internet access at any point during its development); it depends only on the Rust standard library. The 100,000-line compiler can build Linux 6.9 on x86, ARM, and RISC-V. It can also compile QEMU, FFmpeg, SQlite, postgres, redis > I started by draf…
Kinda waiting for them to plateau so I can stop feeling so existential ¯\_(ツ)_/¯
It used the best tests it could find for existing compilers. This is effectively steering Claude to a well-defined solution. Hard to find fully specified problems like this in the wild. I think this is more a testament to small, well-written tests than it is agent teams. I imagine you could do the same thing with any frontier model and a single agent in a linear flow. I don’t know why people use parallel agents and i…
This is such a big and obvious cope. This is obviously a very real problem in the wild and there are many, many others like it. Probably most problems are like this honestly or can be made to be like this.
Earlier quoted context omitted.
You wouldn’t pay a human to write 100k LOC. Or at least you shouldn’t. You’d pay a human to write a working useful compiler that isn’t riddled with copyright issues. If you didn’t care about copying code, usefulness, or correctness you could probably get a human to whip you up a C compiler for a lot less than $20k.
Are you trolling me? Companies (made of humans) write 100,000 LOC all the time. And it's really expensive, despite your suspicions.
We figured out that LOC was a useless productivity metric in the 80s.