Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Bugs You'll Probably Only Have in Rust

gankro.github.io

Post reply on HN
Print view View original

51–60 of 91 posts

Re: Bugs You'll Probably Only Have in Rust

#51

Post by vitalyd »

mcguire wrote: post #43

" The bug was a missing annotation, and the result was that users of Rust's stdlib could compile some incorrect programs that violated memory safety. " IIUC, technically, the bug was a missing implementation of a trait and the result was a data race (which I (weirdly, maybe) don't think of as memory safety). In other words, TL;DR: magic is neat, except that sometimes it really sucks. I may have misunderstood Ralf's b…

> I may have misunderstood Ralf's bug. Is it really the case that MutexGuard was seen as Sync if T was Send, rather that Sync? Wouldn't that be a bigger problem than just the case of MutexGuard?

So T: Sync if &T: Send. MutexGuard internally contains a &Mutex (and Poison, but that's irrelevant here). T was Cell. If you follow the rabbit hole, you'll net out that T was Send, and therefore MutexGuard was Sync.

Re: Bugs You'll Probably Only Have in Rust

#52

Post by dbaupp »

mcguire wrote: post #43

" The bug was a missing annotation, and the result was that users of Rust's stdlib could compile some incorrect programs that violated memory safety. " IIUC, technically, the bug was a missing implementation of a trait and the result was a data race (which I (weirdly, maybe) don't think of as memory safety). In other words, TL;DR: magic is neat, except that sometimes it really sucks. I may have misunderstood Ralf's b…

Data races can lead to other more conventional displays of memory unsafety. For instance, a read of a pointer length pair (a &[T] slice in Rust) while a write is occuring could resulting in a torn read where the pointer is the pre-write value but the length is the post-write one. If the original length was short than the new one, this can lead to a buffer overflow.

One framing of the MutexGuard problem is that the type wasn't declared in a way that reflected its semantics best, although it is clearly unfortunate that doing this is more complicated than the incorrect way.

Re: Bugs You'll Probably Only Have in Rust

#53

Post by vitalyd »

dbaupp wrote: post #47
mcguire wrote: post #44

Wait just a minute. Ralf Jung writes, " This means that the compiler considers a type like MutexGuard to be Sync if all its fields are Sync. " Is that true in general? Is a type thread safe if all its fields are thread safe individually?

Send and Sync are about data races, which lead to memory unsafety, not other forms of thread safety (like dead lock freedom, or maintaining non-unsafe relationships between fields). If there's no unsafe code, then there's no way to have a data race when the individual components are also data race free.

Somewhat tangential, but what ensures memory visibility in Rust? Say I allocate a struct (heap or stack), and then pass an immutable reference to a function that takes T: Sync. Assume the struct itself is Sync (e.g. bunch of integer fields). What ensures that the other thread sees all writes to this struct prior to the handoff?

Re: Bugs You'll Probably Only Have in Rust

#54

Post by dom0 »

tatterdemalion wrote: post #49
barsonme wrote: post #45

If you—like me—were interested in Diesel ORM's zero sized types thing, here's a pretty decent explanation: https://np.reddit.com/r/rust/comments/3ur9co/announcing_dies... edit: Go also has zero-sized types (struct{}), so I wonder if this is also possible? Probably not, I don't think, since the compiler doesn't see through interfaces.

> Go also has zero-sized types (struct{}), so I wonder if this is also possible? No. It specifically uses Rust's generics system, and the fact that generics are monomorphized at compile time, whereas Go interfaces are not. C++ templates can be used in similar ways.

Yes, but differently; C/C++ explicitly prohibit zero-sized types due to object identity.

Re: Bugs You'll Probably Only Have in Rust

#55

Post by Animats »

dbaupp wrote: post #41
burntsushi wrote: post #39

Earlier quoted context omitted.

> IIRC UTF8 stuff has been a perf bottleneck in the url crate Right, and this isn't surprising at all. It's not at all obvious to me that it's premature. It's really easy for from_utf8 to show up in a profile, and I've had occasion to either work around it safely (perhaps by staying in `&[u8]` land) or by resorting to `unsafe`. I did the latter in the CSV crate for managing `StringRecord`s.[1] The record is stored co…

I have found it very interesting to use the safe function when debug_asserts are on and the unsafe one otherwise (or more generally, some abstraction over safe/unsafe variants that compiles away to nothing in the optimised case). This plays really well with fuzzing/quickcheck (etc) too. My favourite example of doing this personally is: https://github.com/Aatch/ramp/pull/48

That's how you put in a buffer overflow backdoor.

Re: Bugs You'll Probably Only Have in Rust

#56

Post by dbaupp »

vitalyd wrote: post #53
dbaupp wrote: post #47

Earlier quoted context omitted.

Send and Sync are about data races, which lead to memory unsafety, not other forms of thread safety (like dead lock freedom, or maintaining non-unsafe relationships between fields). If there's no unsafe code, then there's no way to have a data race when the individual components are also data race free.

Somewhat tangential, but what ensures memory visibility in Rust? Say I allocate a struct (heap or stack), and then pass an immutable reference to a function that takes T: Sync. Assume the struct itself is Sync (e.g. bunch of integer fields). What ensures that the other thread sees all writes to this struct prior to the handoff?

It is the responsibility of cross-thread communication abstractions to use the right fencing (if it is touting itself as safe), probably with the various things in std::sync (especially ...::atomics) if it is pure Rust. For instance, spawning a thread, using a channel (std::sync::mpsc) or a mutex all do such things.

Just calling a function taking T: Sync doesn't need to do any of this, since that call happens all on a single thread. The function might do it internally if it needs to, but that is its own explicit implementation decision.

Re: Bugs You'll Probably Only Have in Rust

#57

Post by [deleted] »

vitalyd wrote: post #53
dbaupp wrote: post #47

Earlier quoted context omitted.

Send and Sync are about data races, which lead to memory unsafety, not other forms of thread safety (like dead lock freedom, or maintaining non-unsafe relationships between fields). If there's no unsafe code, then there's no way to have a data race when the individual components are also data race free.

Somewhat tangential, but what ensures memory visibility in Rust? Say I allocate a struct (heap or stack), and then pass an immutable reference to a function that takes T: Sync. Assume the struct itself is Sync (e.g. bunch of integer fields). What ensures that the other thread sees all writes to this struct prior to the handoff?

[deleted]

Re: Bugs You'll Probably Only Have in Rust

#58

Post by Gankro »

dom0 wrote: post #54
tatterdemalion wrote: post #49

Earlier quoted context omitted.

> Go also has zero-sized types (struct{}), so I wonder if this is also possible? No. It specifically uses Rust's generics system, and the fact that generics are monomorphized at compile time, whereas Go interfaces are not. C++ templates can be used in similar ways.

Yes, but differently; C/C++ explicitly prohibit zero-sized types due to object identity.

To clarify: C says it's UB, C++ rounds up to 1.

Re: Bugs You'll Probably Only Have in Rust

#59

Post by dbaupp »

Animats wrote: post #55
dbaupp wrote: post #41

Earlier quoted context omitted.

I have found it very interesting to use the safe function when debug_asserts are on and the unsafe one otherwise (or more generally, some abstraction over safe/unsafe variants that compiles away to nothing in the optimised case). This plays really well with fuzzing/quickcheck (etc) too. My favourite example of doing this personally is: https://github.com/Aatch/ramp/pull/48

That's how you put in a buffer overflow backdoor.

Could you be more specific about what you mean?

The relevant (i.e. when it isn't premature optimisation: benchmarks have been run) alternatives here are

1. always risk memory unsafety, fast

2. crash on conditions that would trigger memory unsafety, slow

3. use 1 in release builds and 2 in debug/testing builds.

This is effectively having an in-code sanitiser. You can definitely argue that using an actual sanitizer might be better/less error prone, but this scheme allows flagging more general invariants than what sanitisers currently understand, and, can/should be used in conjunction with sanitisers (more checking of `unsafe` is always better).

Of course, poorly implementing the abstraction might lead to behavioural differences between the two modes, but one has to be careful when writing unsafe code anyway. Additionally, typically these abstractions can be small and even written in ways that make it clear that there's only extra checking, returned results are the same (i.e. the only code difference in the abstraction's public API is some extra debug_assert calls).

Re: Bugs You'll Probably Only Have in Rust

#60

Post by grogers »

vitalyd wrote: post #51
mcguire wrote: post #43

" The bug was a missing annotation, and the result was that users of Rust's stdlib could compile some incorrect programs that violated memory safety. " IIUC, technically, the bug was a missing implementation of a trait and the result was a data race (which I (weirdly, maybe) don't think of as memory safety). In other words, TL;DR: magic is neat, except that sometimes it really sucks. I may have misunderstood Ralf's b…

> I may have misunderstood Ralf's bug. Is it really the case that MutexGuard was seen as Sync if T was Send, rather that Sync? Wouldn't that be a bigger problem than just the case of MutexGuard? So T: Sync if &T: Send. MutexGuard internally contains a &Mutex (and Poison, but that's irrelevant here). T was Cell . If you follow the rabbit hole, you'll net out that T was Send, and therefore MutexGuard was Sync.

My confusion (and I suspect others) is about what it means for &T to be Sync. Cell isn't safe to be shared across threads (so isn't Sync) but it is Send if T:Send. But that means &Cell is Sync? You can share a reference to something across threads but not the thing itself? What does that even mean?

You could imagine an alternate world where MutexGuard is Send, to allow transfer of ownership of a lock to a different thread while keeping the mutex locked. But that would mean &MutexGuard is Sync, WTF?

Post reply on HN