Porting the Linux kernel to 100% Rust should be the benchmark for AGI.
... and when done, please port SQLite too :)
Translating All C to Rust (TRACTOR)
41–50 of 403 posts
Re: Translating All C to Rust (TRACTOR)
#42That sounds ... hard. Especially as idiomatic Rust as written by skilled programmers looks nothing like C, and most interesting code is written in C++ anyway. Isn't it equivalent to statically determining the lifetimes of all allocations in the C program, including those that are implemented using custom allocators or which cross into proprietary libraries? There's been a lot of research into this sort of thing over…
Hard for humans. But it's DARPA, is it hard for AI? Image classification used to be hard also, today cars drive themselves. I'd say it's good timing.
Well, Claude 3.5 can do translation from one language to another in a fairly competent manner if the languages are close enough. I've used it for that task myself with success (Java -> JavaScript).
But, this isn't just about rewriting code from one language to another. It's about reverse engineering complex information out of the code, which may not be immediately visible in it, and then finding a way to make it "safe" according to Rust's type system. Where's the training data for that? It'd be really hard even for skilled humans.
Personally I think the most pragmatic way to make C/C++ memory safe quicker is one of two approaches:
1. Incrementally. Make std::vector[] properly bounds checked (still not done even in chrome!), convert allocations to allocations that know their own size and do bounds checking e.g. https://issues.chromium.org/issues/40285824
2. Or, go the whole hog and use runtime techniques like garbage collection and runtime bounds checks.
A good example of approach (2) is Managed Sulong, which extends the JVM to execute LLVM bitcode directly whilst exposing to the C/C++/FORTRAN a virtualized Linux syscall interface. The whole piece of code can be sandboxed with permissions, and memory safety errors are caught at runtime. The compiler tries to optimize out as many bounds checks as possible. The interesting thing about this approach is it doesn't require big changes to the source code (as long as it's already been ported to Linux), which means the work of making something safe can be done by teams independent of the original authors. In practice "rewrite it in Rust" will usually mean a fork, which introduces lots of complicated technical, cultural and economic issues.
Managed Sulong is also a research project and has a bunch of problems to solve, for instance it needs to lose the JITC dependency and go fully AOT compiled (doable, there's no theoretical issue with it and much of the needed infra already exists). And performance/memory usage can always be improved of course, it regresses vs the original C. But those are "just" systems engineering problems, not rewrite-the-world and solve-static-analysis problems.
Disclosure: I do work part time at Oracle Labs which developed Managed Sulong, but I don't work on it.
Re: Translating All C to Rust (TRACTOR)
#43Earlier quoted context omitted.
Genuine question: Would you mind explaining to a dev that doesn’t know much (anything) about Rust, how does this settle any debate?
Well, the general 'Rewrite All in Rust' consensus is that it solves all general programming problems, ever . Yet, the linked repository shows a huge list of cases in which simple, documented use of Rust can cause Undefined Behavior (a.k.a. 'UB') Pretty much every argument of Rust advocates against C/C++ boils down to either 'but memory safety' or 'but UB'. Yet there are many convincing counter-arguments that boil dow…
> Well, the general 'Rewrite All in Rust' consensus is that it solves all general programming problems, ever.
No, that’s not the consensus. This is a strawman.
Re: Translating All C to Rust (TRACTOR)
#44Earlier quoted context omitted.
There are some soundness holes in the implementation that can cause this. Just like any project, the compiler can have bugs. They’ll be fixed just like any bug.
Yes, in particular some interactions with LLVM have caused some frustrating UB. But those are considered implementation bugs, rather than user bugs, and all the conditions Miri states at the top are relevant primarily in unsafe code, which contradicts the OP's point, which is that there are tons of documented cases of UB in safe Rust. This is not true. There are a few documented cases, and most have been fixed. It's…
For sure, just making sure to acknowledge this is the case, before someone responded to your post with cve-rs. :)
Re: Translating All C to Rust (TRACTOR)
#45They didn't explain why they've chosen Rust. There are a lot of memory-safe languages besides Rust, especially in application-level area (not systems-level like Rust).
There are a lot of memory safe languages; there are fewer that have (1) marginal runtime requirements, (2) transparent interop/FFI with existing C codebases, (3) enable both spatial and temporal memory safety without GC, and (4) have significant development momentum behind them. Rust doesn't have to be unique among these qualifications, but it's currently preeminent.
Re: Translating All C to Rust (TRACTOR)
#46Earlier quoted context omitted.
Hard for humans. But it's DARPA, is it hard for AI? Image classification used to be hard also, today cars drive themselves. I'd say it's good timing.
Well, Claude 3.5 can do translation from one language to another in a fairly competent manner if the languages are close enough. I've used it for that task myself with success (Java -> JavaScript). But, this isn't just about rewriting code from one language to another. It's about reverse engineering complex information out of the code, which may not be immediately visible in it, and then finding a way to make it "saf…
std::vector [] has had bounds checking since forever if you set the correct compiler flag. Since they aren't using it this is a choice, presumably they prefer the speed gain.
Re: Translating All C to Rust (TRACTOR)
#47That sounds ... hard. Especially as idiomatic Rust as written by skilled programmers looks nothing like C, and most interesting code is written in C++ anyway. Isn't it equivalent to statically determining the lifetimes of all allocations in the C program, including those that are implemented using custom allocators or which cross into proprietary libraries? There's been a lot of research into this sort of thing over…
Hard for humans. But it's DARPA, is it hard for AI? Image classification used to be hard also, today cars drive themselves. I'd say it's good timing.
> today cars drive themselves
You can attach about a hundred asterisks to that.
If anything, I think self the failure to hit L5 driving after billions of dollars and millions of man hours invested is probably reflective of how automatic C to Rust translation will go. We'll cruise 90% of the way, but the last 10% will prove insurmountable with current technology.
Think about the number of C programs in the wild that rely on compiler-specific or libc-specific or platform-specific behavior, or even undefined behavior plus the dumb luck of a certain brittle combination of {compiler version} ∩ {libc version} ∩ {linker version} ∩ {build flags} emitting workable machine code. There's a huge chunk of C software where there's not enough context within the source itself (or even source plus build scripts) to understand the behavior. It's not even clear that this is a solvable problem in the abstract.
None of that is to say that DARPA shouldn't fund this. Research isn't always about finding an industrial strength end product; the knowledge and expertise gained along the way is important too.
Re: Translating All C to Rust (TRACTOR)
#48The one link for those who think that 'Rewrite it All in Rust' will, well, settle any debates: https://github.com/rust-lang/miri/
Genuine question: Would you mind explaining to a dev that doesn’t know much (anything) about Rust, how does this settle any debate?
It is a tool for checking that your unsafe code doesn't cause UB. It doesn't really settle anything, but the commenter uses it as a gotcha to say "rust is no better than C, because you still can compile code that contains UB".
Re: Translating All C to Rust (TRACTOR)
#49They didn't explain why they've chosen Rust. There are a lot of memory-safe languages besides Rust, especially in application-level area (not systems-level like Rust).
If you have your cross hair on c, then you want a language that can do whatever c does. That makes the list of memory safe languages a lot shorter.
Re: Translating All C to Rust (TRACTOR)
#50That sounds ... hard. Especially as idiomatic Rust as written by skilled programmers looks nothing like C, and most interesting code is written in C++ anyway. Isn't it equivalent to statically determining the lifetimes of all allocations in the C program, including those that are implemented using custom allocators or which cross into proprietary libraries? There's been a lot of research into this sort of thing over…
speaking of hard, the DOE actually funds a project that has been around for 20+ years now (ROSE) that involves (among other things) doing static analysis on and automatically translating between C/C++/Cuda and even high level languages like Python as well as HPC variants of C/C++. They have a combined AST that supports all of those languages with the same set of node types essentially. Quite cool. I got to work on it when I was an intern at Livermore, summer of 2014.
and it's open source as well! http://rosecompiler.org/ROSE_HTML_Reference/index.html