Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Type-Level Programming in Rust

willcrichton.net

Post reply on HN
Print view View original

11–20 of 29 posts

Re: Type-Level Programming in Rust

#11

Post by wcrichton »

couchand wrote: post #3

I'm immediately turned off by the entirely gratuitous use of unsafe here. I clicked through to the linked Reddit discussion and I'm not convinced the author has really thought it through. Writing, "pedagogically, that's irrelevant to the post, so I didn't highlight it" misunderstands the pedagogy here. However, there's some really neat stuff later, so it's wise to grit your teeth and get past it. In case anyone else…

Hi, author of the code here. Why do I use mem::transmute? To highlight the fact that when using typestate, it's useful to understand operations that change the the value of an object as distinct from those that change the type of an object. It's also a matter of efficiency -- you don't have to generate a new struct every time you change state.

You can avoid the overhead without transmute, but it's still unsafe and not easily explained: https://github.com/Munksgaard/session-types/commit/3a310d205...

Re: Type-Level Programming in Rust

#12

Post by chrismorgan »

couchand wrote: post #3

I'm immediately turned off by the entirely gratuitous use of unsafe here. I clicked through to the linked Reddit discussion and I'm not convinced the author has really thought it through. Writing, "pedagogically, that's irrelevant to the post, so I didn't highlight it" misunderstands the pedagogy here. However, there's some really neat stuff later, so it's wise to grit your teeth and get past it. In case anyone else…

Further to this: transmute is really pretty dangerous, and you’ve got to be very careful if you use it, or you’ll invoke undefined behaviour. And I believe the transmutes in the original article do invoke undefined behaviour, since the layout of the struct is undefined (you’d need to use something like `#[repr(C)]` on the struct to make it be defined). In practice, these particular examples are at least 99.9999% sure to do the right thing in all versions of Rust ever, but you just shouldn’t ever do this—you’re playing with delayed-action napalm. With his background, I feel that Will should have known better than to use unsafe here in this way: because it’s wrong.

Admittedly the SecureVec example is more understandable: because the generic is passed through to Item as well, you’ll need to convert the Vec> to a Vec>>, which… well, it can be done readily enough without unsafe code, by consuming the Vec, mapping each item and collecting to a new vec, but short of a kind of specialisation magic that I don’t think will kick in in this case, that’s going to entail allocating a new vector, so that I would be inclined to use unsafe code here for efficiency; but care should still be taken that it doesn’t invoke undefined behaviour.

But one more word in defence of the transmutes: it makes it obvious that it’s either zero-cost or just a copy, whereas more involved things can become subject to compiler optimisations, so that you don’t know that the whole supposedly-noop transformation will be being optimised out of existence. That confidence and clearly-stated intent is valuable.

Re: Type-Level Programming in Rust

#13

Post by kevincox »

One minor note is that I don't think PhantomData is required here.

  _state: PhantomData
Since State is zero-sized it is free to store it in the struct.

  state: State
IIUC PhantomData is only required when you need to indicate that you are logically storing something of that type but don't want to actually store it. Since storing is free in this case you may as well just store it.

Re: Type-Level Programming in Rust

#14

Post by kevincox »

wcrichton wrote: post #11
couchand wrote: post #3

I'm immediately turned off by the entirely gratuitous use of unsafe here. I clicked through to the linked Reddit discussion and I'm not convinced the author has really thought it through. Writing, "pedagogically, that's irrelevant to the post, so I didn't highlight it" misunderstands the pedagogy here. However, there's some really neat stuff later, so it's wise to grit your teeth and get past it. In case anyone else…

Hi, author of the code here. Why do I use mem::transmute? To highlight the fact that when using typestate, it's useful to understand operations that change the the value of an object as distinct from those that change the type of an object. It's also a matter of efficiency -- you don't have to generate a new struct every time you change state. You can avoid the overhead without transmute, but it's still unsafe and no…

Generating new structs is free in Rust. There is only cost if you need to send it somewhere and the layout differs from the original.

Since `transmute` will be undefined behaviour if the layout differs you will only be saving cost if you would be invoking undefined behaviour.

I recommend checking the generated asm and reporting a missed-optimization bug if it differs.

Re: Type-Level Programming in Rust

#15

Post by kd5bjo »

chrismorgan wrote: post #12
couchand wrote: post #3

I'm immediately turned off by the entirely gratuitous use of unsafe here. I clicked through to the linked Reddit discussion and I'm not convinced the author has really thought it through. Writing, "pedagogically, that's irrelevant to the post, so I didn't highlight it" misunderstands the pedagogy here. However, there's some really neat stuff later, so it's wise to grit your teeth and get past it. In case anyone else…

Further to this: transmute is really pretty dangerous, and you’ve got to be very careful if you use it, or you’ll invoke undefined behaviour. And I believe the transmutes in the original article do invoke undefined behaviour, since the layout of the struct is undefined (you’d need to use something like `#[repr(C)]` on the struct to make it be defined). In practice, these particular examples are at least 99.9999% sure…

> In practice, these particular examples are at least 99.9999% sure to do the right thing in all versions of Rust ever

I’m pretty sure that declaring both `Item` and `SecureVec` as `#[repr(transparent)]` would cover that last 0.0001% chance: it protects against the extra fields becoming non-zero-size by accident and forces the binary representation to be identical to the first field.

At that point, he’s transmuting between things with the same representation; all that’s left to verify is semantics.

Re: Type-Level Programming in Rust

#16

Post by chrismorgan »

kd5bjo wrote: post #15
chrismorgan wrote: post #12

Earlier quoted context omitted.

Further to this: transmute is really pretty dangerous, and you’ve got to be very careful if you use it, or you’ll invoke undefined behaviour. And I believe the transmutes in the original article do invoke undefined behaviour, since the layout of the struct is undefined (you’d need to use something like `#[repr(C)]` on the struct to make it be defined). In practice, these particular examples are at least 99.9999% sure…

> In practice, these particular examples are at least 99.9999% sure to do the right thing in all versions of Rust ever I’m pretty sure that declaring both `Item` and `SecureVec` as `#[repr(transparent)]` would cover that last 0.0001% chance: it protects against the extra fields becoming non-zero-size by accident and forces the binary representation to be identical to the first field. At that point, he’s transmuting b…

I knew there was another #[repr(…)] value, just couldn’t remember its name and didn’t look it up. Thanks for pointing it out. Yep, #[repr(transparent)] is a correct way of fixing the undefined behaviour—and if it won’t compile, then #[repr(C)] will do.

Re: Type-Level Programming in Rust

#17

Post by wcrichton »

kd5bjo wrote: post #15
chrismorgan wrote: post #12

Earlier quoted context omitted.

Further to this: transmute is really pretty dangerous, and you’ve got to be very careful if you use it, or you’ll invoke undefined behaviour. And I believe the transmutes in the original article do invoke undefined behaviour, since the layout of the struct is undefined (you’d need to use something like `#[repr(C)]` on the struct to make it be defined). In practice, these particular examples are at least 99.9999% sure…

> In practice, these particular examples are at least 99.9999% sure to do the right thing in all versions of Rust ever I’m pretty sure that declaring both `Item` and `SecureVec` as `#[repr(transparent)]` would cover that last 0.0001% chance: it protects against the extra fields becoming non-zero-size by accident and forces the binary representation to be identical to the first field. At that point, he’s transmuting b…

I didn't know about #[repr(transparent)], I will add that to the blog post. Thank you!

Re: Type-Level Programming in Rust

#18

Post by lilyball »

wcrichton wrote: post #11
couchand wrote: post #3

I'm immediately turned off by the entirely gratuitous use of unsafe here. I clicked through to the linked Reddit discussion and I'm not convinced the author has really thought it through. Writing, "pedagogically, that's irrelevant to the post, so I didn't highlight it" misunderstands the pedagogy here. However, there's some really neat stuff later, so it's wise to grit your teeth and get past it. In case anyone else…

Hi, author of the code here. Why do I use mem::transmute? To highlight the fact that when using typestate, it's useful to understand operations that change the the value of an object as distinct from those that change the type of an object. It's also a matter of efficiency -- you don't have to generate a new struct every time you change state. You can avoid the overhead without transmute, but it's still unsafe and no…

Why can’t that just be written

  fn cast(self) -> Chan {
    Chan(self.0, self.1, PhantomData)
  }

Re: Type-Level Programming in Rust

#19

Post by lilyball »

kevincox wrote: post #13

One minor note is that I don't think PhantomData is required here. _state: PhantomData Since State is zero-sized it is free to store it in the struct. state: State IIUC PhantomData is only required when you need to indicate that you are logically storing something of that type but don't want to actually store it. Since storing is free in this case you may as well just store it.

This sounds reasonable, but personally I prefer the PhantomData as it makes it immediately obvious to the reader that this is a zero-sized type.

Re: Type-Level Programming in Rust

#20

Post by evincarofautumn »

chrismorgan wrote: post #12
couchand wrote: post #3

I'm immediately turned off by the entirely gratuitous use of unsafe here. I clicked through to the linked Reddit discussion and I'm not convinced the author has really thought it through. Writing, "pedagogically, that's irrelevant to the post, so I didn't highlight it" misunderstands the pedagogy here. However, there's some really neat stuff later, so it's wise to grit your teeth and get past it. In case anyone else…

Further to this: transmute is really pretty dangerous, and you’ve got to be very careful if you use it, or you’ll invoke undefined behaviour. And I believe the transmutes in the original article do invoke undefined behaviour, since the layout of the struct is undefined (you’d need to use something like `#[repr(C)]` on the struct to make it be defined). In practice, these particular examples are at least 99.9999% sure…

Has there been any work in Rust on something like Haskell’s ‘Coercible’? Namely, it defines the precise criteria for safe zero-cost coercions between types that provably have the same representation, and derives the coercions automatically. It guarantees that ‘coerce’ is identical to ‘fmap coerce’, which saves the cost of that exact traverse+reconstruct pattern to change out a phantom type parameter.

There are some deficiencies, most prominently that the “role” system is aggressively monomorphic, so you can’t always prove that two things are coercible—the compiler must conservatively assume that type parameters with unknown roles cannot be coerced—but overall it’s much better than what we had before, and I expect it’ll continue to improve.

Post reply on HN