Curious how Rust's ownership interplays with this memory management approach. Does it actually get in the way of implementing this?
Handles Are the Better Pointers (2018)
71–80 of 113 posts
Re: Handles Are the Better Pointers (2018)
#72Earlier quoted context omitted.
In my Perfect Programming Language That Doesn't Have To Deal With The Problems Of Actually Existing, serialization of data structures into memory is explicitly defined in the language, just as one would define a JSON or Protobuf serialization. One could define a "Point3{x,y,z}", and hypothetically define one array of them in the conventional manner, define another array as a columnar serialization, and potentially ev…
The idea of "univalence", being pursued in Homotopy Type Theory (AKA HoTT), is that isomorphic types (i.e. we can convert back and forth without information loss) are equal , and hence one can be used in place of the other. The dream scenario is to write all of our libraries and application code using types that are the most straightforward (e.g. counting in unary, mapping over linked-lists, looking up values from li…
I remember when that first came out. I have not checked in on it in a while. Do you have any links to communities/reddits/etc working with that? (/r/haskell was very excited for a while, but then the energy went somewhere else, and I don't know where.)
One of the things I only alluded to in my third paragraph, but very much had this sort of thing in mind, is that it's going to take a lot of type work and a lot of thinking in properties (like Traversable, Functor, etc. in Haskell) to make this work out well, because a lot of these rewrites to underlying representation will have manifestations at the property level (this array is now Traversable but no longer RandomAccessible, etc.).
Re: Handles Are the Better Pointers (2018)
#73Curious how Rust's ownership interplays with this memory management approach. Does it actually get in the way of implementing this?
On the contrary, it kind of encourages it https://kyren.github.io/2018/09/14/rustconf-talk.html
Re: Handles Are the Better Pointers (2018)
#74Earlier quoted context omitted.
In my Perfect Programming Language That Doesn't Have To Deal With The Problems Of Actually Existing, serialization of data structures into memory is explicitly defined in the language, just as one would define a JSON or Protobuf serialization. One could define a "Point3{x,y,z}", and hypothetically define one array of them in the conventional manner, define another array as a columnar serialization, and potentially ev…
I was privy to a fairly high-wattage conversation between Walid Taha, Jeremy Siek, Todd Veldhuizen, et al, where they discussed this issue at length. They were concerned that languages almost universally equate data types with data structures . Their opinion (drunkenly expressed) was that the program semantics should be based on both. The wrinkle was was happened if you wanted to expose transformations of the underly…
Yes, as I was typing this up I began to realize that's where I was really going. Essentially turning an entire struct of, say, { int A; string B; Thingy C } into only a set of promises about being able to get an int, a string, a Thingy, etc., and promises around being able to set them (let's just do a struct for now, no methods or hiding), and then decoupling the type, being the set of promises being made, from the structure, being the concrete layout in memory and corresponding code capable of fulfilling those promises.
It's all easy in the simple struct case, gets more interesting when you start trying to manipulate them by putting them into arrays or associative maps, etc.
On the one hand, this is nearly revolutionary in some sense, but in another it's simply an incremental advance in a direction we're already going. "Object's are a poor man's closures, closures are a poor man's objects" being an old saying leaning in this direction, and it's hardly a new observation in Haskell that there's not a heck of a lot of practical difference between an associative array of X -> Y and a function (x -> y), at least when it comes to reading them.
Re: Handles Are the Better Pointers (2018)
#75Earlier quoted context omitted.
Typically functional programming isn’t considered to be CPU friendly. Specifically typical functional programming allocates memory all over the heap, especially given the prevalence of things like linked lists and so on. This obviously makes for crumby cache performance and memory pressure. I’m not sure in what sense FP lends itself to columnar data structures, but those would obviously have still better cache and ve…
The difference is that if you're doing functional programming with vectors as the fundamental data type (rather than scalars in linked lists), then the implementation could be hyper optimized underneath the programmer to maximize computational throughout. Whereas with imperative for loops, the implementation is somewhat fixed because python, C, and C++ force you to choose row (array of struct), or columnar (struct of…
Also I always find it's so easy to make everything data parallel if you program in a functional way.
Sure it might not be the most efficient implementation possible, but due to the ease of parallelizing, you are more likely to just try it out.
Re: Handles Are the Better Pointers (2018)
#76MacOS 1.0 in 1984 used handles in order to allow the heap to be coalesced more easily (although locked handles and regular pointers often created islands). Given the first Mac had 128K RAM and no MMU, it was highly necessary at the time.
Microsoft Windows did the same:
https://devblogs.microsoft.com/oldnewthing/20041104-00/?p=37...
In later versions, heap memory handles were even implemented in Intel x86 hardware:
https://devblogs.microsoft.com/oldnewthing/20041105-00/?p=37...
Re: Handles Are the Better Pointers (2018)
#77“We can solve any problem by introducing an extra level of indirection.”
— “Fundamental theorem of software engineering”
https://en.wikipedia.org/wiki/Fundamental_theorem_of_softwar...
Re: Handles Are the Better Pointers (2018)
#78Earlier quoted context omitted.
The idea of "univalence", being pursued in Homotopy Type Theory (AKA HoTT), is that isomorphic types (i.e. we can convert back and forth without information loss) are equal , and hence one can be used in place of the other. The dream scenario is to write all of our libraries and application code using types that are the most straightforward (e.g. counting in unary, mapping over linked-lists, looking up values from li…
HoTT changes what equal means. In order for univalence to work you must be comfortable with arbitrary code running to perform the necessary substitutions. Isomorphism isn't for free, you can do some fancy cast-conversions (where cast is an identity function) that has been explored in generic zero-cost reuse [1] but this is not HoTT and uses essentially an UIP equality (which is inconsistent with HoTT). Your usage of…
That sounds like, for performance reasons, you want control over the actual types used. But you could eliminate those changes from affecting the semantic representation of your program. Or at the very least with an easy check that the optimization doesn't affect your semantics.
Re: Handles Are the Better Pointers (2018)
#79Thinking out loud: Since handles refer to offer rather than absolute addresses, this sort of scheme should allow for "serializing" data by memmapping chunks of memory to disk. Obviously, this scheme wouldn't be useful for cross-platform save files, and might not even translate across builds. Still, as long as you have a robust scheme for invalidating data when a new version is deployed, I could see this being useful…
Excellent insight, thank you.
Re: Handles Are the Better Pointers (2018)
#80Does this assume that the handle consumers have to somehow 'release' the handle in addition to 'deleting' it? As was explained with an example of delayed release of some deleted but still GPU queued element. I'm not sure if the suggested generation counter approach is the optimal way to deal with the delayed release. Also, when it comes down to own memory managers, benchmarking is a must. Added complexities may event…
The "generation counter" suggested in the article has nothing to do with delayed release, as far as I understand. It is used as a strategy to avoid returning colliding handles close in the time domain, reducing the chances for a collision of the "unique bit pattern", which would allow non-detectable dangling access conditions.
> ... non-detectable dangling access conditions.
My understanding is that the dangling access conditions potentially arise due to the delayed release of deleted handles.
["... Instead the rendering system would only mark the resource object for destruction when the user code requests its destruction, but the actual destruction happens at a later time when the GPU no longer uses the resource."]