Live data from Hacker News

DARPA project for automated translation from C to Rust (2024)

darpa.mil

171–180 of 194 posts

Re: DARPA project for automated translation from C to Rust (2024)

#171

Earlier quoted context omitted.

(Note that I edited my comment for clarity since the original reply. My original comment just said "reference counting".)

The key part I agree with most in your edited comment is that unsafe raw pointers are the easiest way to deal with cycles. I think this is key. RC pointers don’t give you the ability to say: ok, I now know that this subset of the graph is dead because reasons so delete it. In RC, you’d have to walk the graph to break cycles. That’s a rather important difference from how you’d manage a cyclic data structure in C, C++,…

For a graph, I agree.

For a tree with parent pointers, I'd absolutely use Rc, with Weak for the parent pointers.

Re: DARPA project for automated translation from C to Rust (2024)

#172
post #158

Earlier quoted context omitted.

“ most people really want to be able to write code that doesn’t crash at all because of expressive limits in C’s semantics. “Memory safety” is a convenient term of art for this.” Rust doesn’t rely only on compile-time checks, so I find your claim dubious. Actually, the dynamic checks done by Rust are presented as useful features in themselves. The memory safety audience and most people seem like arbitrary groups defi…

I didn’t say it relies only on compile time checks. Where are you getting that from? The point about expressive semantics is that you can foreclose on vulnerabilities statically, not that everything becomes a static check. That’s why I mentioned UAFs and type confusions in the adjacent response below. > Taking standard C code and achieving the above would be amazing and much more useful than rewriting something in Ru…

> Fil-C bites the second bullet, under the theory that there’s a lot of userspace code that isn’t extremely performance sensitive. And that’s probably true, but I also think it misses the larger trend: that a lot of userspace code is abandonware, and that there’s sufficiently more interest in rewriting it than maintaining it.

Is there sufficiently more interesting in rewriting it, really? I know that there are a handful of userspace utils being rewritten in Rust, doing that is labor-intensive, and causes regressions (in perf and behavior), which then causes more labor (both for folks to deal with the fallout on the receiving end and to fix the regressions). And for every tool that has a rewrite in flight, there are maybe 100 others that don't.

Re: DARPA project for automated translation from C to Rust (2024)

#173
post #153

Earlier quoted context omitted.

Fil-C absolutely does fix uninitialized memory. It’s on my list to solve division. It’s easy to do and also not super important for the security angle that I’m addressing. But with doing precisely to provide clarity to these kinds of discussions. I’ve mostly tackled signed overflow. I’ve fixed all the cases where signed overflow would let you bypass Fil-C’s own bounds checks. It’s not hard to fix the remaining cases.…

How does Fil-C "fix" uninitialized memory?

By initializing it to zero.

Re: DARPA project for automated translation from C to Rust (2024)

#174

This is C to Rust, not C++ to Rust. That's important. DARPA was willing to award multiple contracts, but the only one awarded is to a group from several universities. [1] "The team’s approach, Formally-Verified Compositional Lifting of C to Rust (which forms the acronym ForCLift, pronounced “forklift”), uses Verified Lifting, which combines formal methods and program analysis with AI techniques such as Large Language…

> The test case which caused a segfault in the C version also caused a segfault in the unsafe Rust version. At least it generates compatible code. I mean... That's what I would expect. It's a translation not a rewrite. The question here is really “what's the end goal?”. Are you gonna refactor parts of the translations output, or is that too much work? Cause a pure translation won't give you anything besides a differe…

There's one reason I could think of for translating from C to Rust without further changes: simplifying the build. If you can remove the need for a build.rs script running make, that's one headache less. At that point you have exactly the same behaviour as wrapping the original C code in a -sys crate, so all of the usual caveats are warranted.

Re: DARPA project for automated translation from C to Rust (2024)

#175

Earlier quoted context omitted.

I didn’t say it relies only on compile time checks. Where are you getting that from? The point about expressive semantics is that you can foreclose on vulnerabilities statically, not that everything becomes a static check. That’s why I mentioned UAFs and type confusions in the adjacent response below. > Taking standard C code and achieving the above would be amazing and much more useful than rewriting something in Ru…

> Fil-C bites the second bullet, under the theory that there’s a lot of userspace code that isn’t extremely performance sensitive. And that’s probably true, but I also think it misses the larger trend: that a lot of userspace code is abandonware, and that there’s sufficiently more interest in rewriting it than maintaining it. Is there sufficiently more interesting in rewriting it, really? I know that there are a hand…

I think so: you're 100% right about the labor, but OSS isn't a rational labor market :-). People RIIR not because it's easy, but because they find it more fun than the alternative.

(If it was, we wouldn't have dozens of tools that all NIH the same thing.)

Re: DARPA project for automated translation from C to Rust (2024)

#176

Earlier quoted context omitted.

I didn’t say it relies only on compile time checks. Where are you getting that from? The point about expressive semantics is that you can foreclose on vulnerabilities statically, not that everything becomes a static check. That’s why I mentioned UAFs and type confusions in the adjacent response below. > Taking standard C code and achieving the above would be amazing and much more useful than rewriting something in Ru…

> Fil-C bites the second bullet, under the theory that there’s a lot of userspace code that isn’t extremely performance sensitive. And that’s probably true, but I also think it misses the larger trend: that a lot of userspace code is abandonware, and that there’s sufficiently more interest in rewriting it than maintaining it. Is there sufficiently more interesting in rewriting it, really? I know that there are a hand…

Fil-C is going to usually be a huge perf regression right? Sometimes I won't care, sometimes I will and this will vary between users with only some commonality.

I think any software on the "most people are annoyed" list will get a rewrite. All the tools where people cared specifically about perf anyway are already rewritten or being rewritten because there are so many performance opportunities. The out-of-box grep sucks compared to ripgrep for example.

It will suck to be someone with no programming ability for whom a tool most people aren't annoyed by is too slow with Fil-C for their niche use. But that's not so different from how it sucks when your local government forgets deaf people exist, or that time we made everything use touch screens so if your fingers don't "work" on a touch screen now you're crippled with no warning.

Re: DARPA project for automated translation from C to Rust (2024)

#177
post #153

Earlier quoted context omitted.

How does Fil-C "fix" uninitialized memory?

I assume under the same "memory safety" rationale it just zeroes the RAM. That's "safe" and compatible with C. In a good language this mistake is caught at compile time, like in Rust, the compiler says "Hey, I don't see how this variable is initialized before use" and you slap your forehead and fix it. But zeroing everything is technically safe. For the Casey "hand made" Muratori type zeroing might even seem like a b…

I guess I don't see how making inherently incorrect C code "safe" by sanitizing something it shouldn't be doing anyway is actually improving the C code.

Re: DARPA project for automated translation from C to Rust (2024)

#178

Earlier quoted context omitted.

Not my post, but I've heard the same concerns with Bevy as with the others. And the actual blog post linked by that post does mention Bevy many times. Infact Bevy might be a perfect example of my gripe(s) with Rust. Completely throwing a standard object system out the window in-favor of ECS which still hasn't caught on in even the most used game engines like Unity hurts adoption and usability.

I thought Unity and Unreal were transiting to ECS? I was under the impression at least that Bevy was adopting what the industry was moving towards, not exploring virgin territory.

I'd very much consider ECS virgin territory. DOTS (ECS) in Unity has been out for 5+ years with little to no real adoption. I'm unaware of any pretty much any use of ECS in Unreal (I think what exists is mostly focused on internal graphics / animation, not gameplay).

Re: DARPA project for automated translation from C to Rust (2024)

#179

Earlier quoted context omitted.

What are the specific aesthetic complaints here? In my limited rust experience, I’ve found that it does a pretty good job of using the ugliness of something like an explicit lifetime to signal to the developer and reader that the code is doing something complicated and non-obvious. Like “here’s a part where the types logic required more help than what the compiler could figure out on its own; here be dragons.” In tha…

I have no specific compaints, but here one example i saw online, and i'm talking from a C dev perspective. let x: Option , std::num::ParseIntError>> = Some(Ok(vec![1, 2, 3])); let flattened = x .map(|r| r.unwrap_or_default()) .unwrap_or_else(|| Vec:: ::new()); I have no idea what the code is doing here, but while reading python or JS code, i can make an educated guess what it's doing. I have no experience with Rust s…

It is of course impossible to know the context in which you saw this, but I will say this is a particular idiomatic way to write Rust which won't mesh with your C knowledge.

Also this code isn't a thing you'd actually do, it's maybe an illustration or part of an example I suppose

You see that vec![1, 2, 3] ? That's what we're getting at the end, a growable array with three integers in it. All the other stuff is machinery to handle errors which in fact have not happened.

    let flattened = vec![1, 2, 3];
Is the same effect, although probably for style you'd write:

    let flattened: Vec = vec![1, 2, 3];

Re: DARPA project for automated translation from C to Rust (2024)

#180

This is C to Rust, not C++ to Rust. That's important. DARPA was willing to award multiple contracts, but the only one awarded is to a group from several universities. [1] "The team’s approach, Formally-Verified Compositional Lifting of C to Rust (which forms the acronym ForCLift, pronounced “forklift”), uses Verified Lifting, which combines formal methods and program analysis with AI techniques such as Large Language…

> The test case which caused a segfault in the C version also caused a segfault in the unsafe Rust version. At least it generates compatible code. I mean... That's what I would expect. It's a translation not a rewrite. The question here is really “what's the end goal?”. Are you gonna refactor parts of the translations output, or is that too much work? Cause a pure translation won't give you anything besides a differe…

The end goal for DARPA's project is safe Rust. Translate unsafe constructs to safe constructs. C arrays of unknown length become Rust arrays or Vec items of known length. Pointer arithmetic becomes slices. (This covers most string manipulation, a high-risk activity in C.) Ownership translation probably will involve too much Rc/RefCell at first.

The key to this is using an LLM to recognize idioms, then formal methods to verify that the LLM guessed right. Which is what the current project is trying to do.

For example, consider C code with a function:

    void foo(int* tab, size_t len) { ... }
TRACTOR needs to do something like:

- Look at foo and its callers. Determine whether "tab" is an array or an indirect pass of a single integer.

- Look at foo and see if tab is ever modified. That determines whether it is mutable. It may be necessary to go down the call chain. This analysis may not be able to determine if it is mutable, but if determines it is immutable, tab can be passed immutably.

- Ask the LLM to figure out the size of tab from context in both foo and its callers. Is len the length of tab? That's a common idiom in C an LLM can recognize, but it might be a wrong guess. Is it the length in bytes or in ints? Remember, the programmer had to be able to figure this out, so the LLM has a good chance of doing it.

- If tab is immutable, hypothesize that the correct translation is

    fn foo(tab: &[int]) { ... }
and within "foo", len comes from tab.len()

Now attempt to use "lifting" formal analysis, as mentioned in the paper, to see if that changes the semantics of the program. Does "foo" subscript out of range? Can you prove that it doesn't?

This problem needs both LLMs and formal methods. LLMs to infer intent, and formal methods to check that inference.

Post reply on HN