Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Native Reflection in Rust

jack.wrenn.fyi

Post reply on HN
Print view View original

31–40 of 68 posts

Re: Native Reflection in Rust

#31

Post by estebank »

dgb23 wrote: post #25
estebank wrote: post #9

Earlier quoted context omitted.

It's a fancy way of saying "every time this type is used, replace all the generic type params with what was used and generate code for it". It's how generics are implemented in Rust. If you have struct Foo (T); And you create Foo(42i32) and Foo(0.0f64), the compiler will create the equivalent to struct Fooi32(i32); struct Foof64(f64); In other languages like Java, generics are implemented the way that Rust does "trai…

Not exactly the same thing but JITs can turn dynamic objects into structs if the structure is consistent. JS runtimes and Julia do this as far as I know.

Yes! Java as well. And this is how those languages can show impressive benchmarks for consistent workloads. In theory they can even surpass AoT languages. In practice it depends on the specifics.

Re: Native Reflection in Rust

#32

Post by loeg »

Animats wrote: post #30

"When you call .reflect on a dyn Reflect value, deflect figures out its concrete type in four steps:" * invokes local_type_id to get the memory address of your value’s static implementation of local_type_id * maps that memory address to an offset in your application’s binary * searches your application’s debug info for the entry describing the function at that offset * parses that debugging information entry (DIE) to…

> This is a rather strange thing to bolt onto a language.

It can just be an extremely fun and cute demo, without practical application.

Re: Native Reflection in Rust

#33

Post by planede »

estebank wrote: post #5
Thaxll wrote: post #3

Today I learn that Rust does not have reflection.

Reflection is usually not available in AoT compiled languages. The prevalent Rust coding styles rely heavily on monomorphic data types and functions, meaning there's nothing left to reflect at runtime. But if you want to deal with trait objects and need to access the underlying type, you need to use Any::downcast or rely on annotations on every type you want to reflect on. Or now, leverage DWARF info on Linux with de…

That's runtime reflection.

Compile time reflection AFAIK is available in D and Zig, and is planned for C++.

Re: Native Reflection in Rust

#34

Post by HideousKojima »

jeroenhd wrote: post #10

Does using DWARF info imply that this will break when you strip the resulting executable? I often strip my Rust binaries because it practically halves the application size, which can become quite a lot in a language where you're statically linking everything. Regardless, quite an ingenious use of standard ELF features, I didn't think this would be possible in Rust without adding some kind of VM around reflection code…

C# has similar issues where they have to be conservative about what them trim from binaries for AoT in case it is used for reflection, so I imagine you'd run into the same issues for almost any compiled language you want to implement reflection for.

Re: Native Reflection in Rust

#35

Post by mmis1000 »

dgb23 wrote: post #25
estebank wrote: post #9

Earlier quoted context omitted.

It's a fancy way of saying "every time this type is used, replace all the generic type params with what was used and generate code for it". It's how generics are implemented in Rust. If you have struct Foo (T); And you create Foo(42i32) and Foo(0.0f64), the compiler will create the equivalent to struct Fooi32(i32); struct Foof64(f64); In other languages like Java, generics are implemented the way that Rust does "trai…

Not exactly the same thing but JITs can turn dynamic objects into structs if the structure is consistent. JS runtimes and Julia do this as far as I know.

Firefox's js runtime also do tricks like generate multi copy of optimized function when the function has multi call site instead make one with lots of if else. So it no longer suffer from the problem that function that frequently get multi different type of parameters from different call site has poor performance.

It's probably exactly how templates work, except the details are invisible to users.

https://hacks.mozilla.org/2020/11/warp-improved-js-performan...

Re: Native Reflection in Rust

#36

Post by olvy0 »

I've used very similar method, at work, to provide C++ "reflection" between my own system and a system from another team.

Basically, the other system is a dynamic library which sends and receives C structures from my application. Those structures are then mapped into a buffer that is supposed to have the same size and there are pointers with metadata pointing into the buffer that are supposed to be exactly like the struct elements. Those structures can have arbitrary complexity, and are passed around through type erasure (essentially char*).

I wrote a "reflection" code for the other team, which runs when they register the struct instance to be sent, checks if there's a matching PDB [0] around, reads it, and outputs a json including the metadata needed, which can then be used to define the structures' metadata on our side correctly.

This is all in C/C++ since in some contexts we have soft real-time requirements, else I would have used any of the many RPC frameworks available.

This has been working for several years now.

This is not a generic solution but it's good enough for in-house communication between 2 systems that are maintained by different parts of the organization, where the API between them, that like I said is based on passing around char* buffers, has been more or less set in stone a long time ago. Conway's law [1] and all that. Sigh.

[0] We are a Windows shop although the same thing should work with DWARF info, same as the OP library works. In fact he says "It may never work on Windows, which does not use DWARF to encode debug info" but I can say that the same approach does work on Windows, for C++ at least. The PDB format might be a tad undocumented, but its documentation has been improved in the last decade or so since I started working on my library. Writing some small test programs is enough to understand how to access it, if all you need is meta info on C-style structures. Other stuff is more... challenging. But it wasn't necessary for my use-case.

[1] https://en.wikipedia.org/wiki/Conway%27s_law

Re: Native Reflection in Rust

#37

Post by More-nitors »

loeg wrote: post #32
Animats wrote: post #30

"When you call .reflect on a dyn Reflect value, deflect figures out its concrete type in four steps:" * invokes local_type_id to get the memory address of your value’s static implementation of local_type_id * maps that memory address to an offset in your application’s binary * searches your application’s debug info for the entry describing the function at that offset * parses that debugging information entry (DIE) to…

> This is a rather strange thing to bolt onto a language. It can just be an extremely fun and cute demo, without practical application.

how about adding this to debuggers for better object-views? (could it be possible to provide near-js/python/java level of obj view?)

Re: Native Reflection in Rust

#38

Post by pcwalton »

Animats wrote: post #30

"When you call .reflect on a dyn Reflect value, deflect figures out its concrete type in four steps:" * invokes local_type_id to get the memory address of your value’s static implementation of local_type_id * maps that memory address to an offset in your application’s binary * searches your application’s debug info for the entry describing the function at that offset * parses that debugging information entry (DIE) to…

> This is a rather strange thing to bolt onto a language. I could see this as an external tool.

It is an external tool. This is a crate, not a part of the compiler.

Re: Native Reflection in Rust

#39

Post by adgjlsfhk1 »

dgb23 wrote: post #25
estebank wrote: post #9

Earlier quoted context omitted.

It's a fancy way of saying "every time this type is used, replace all the generic type params with what was used and generate code for it". It's how generics are implemented in Rust. If you have struct Foo (T); And you create Foo(42i32) and Foo(0.0f64), the compiler will create the equivalent to struct Fooi32(i32); struct Foof64(f64); In other languages like Java, generics are implemented the way that Rust does "trai…

Not exactly the same thing but JITs can turn dynamic objects into structs if the structure is consistent. JS runtimes and Julia do this as far as I know.

Julia doesn't do this. It just has structs in the first place.

Re: Native Reflection in Rust

#40

Post by jerf »

loeg wrote: post #32
Animats wrote: post #30

"When you call .reflect on a dyn Reflect value, deflect figures out its concrete type in four steps:" * invokes local_type_id to get the memory address of your value’s static implementation of local_type_id * maps that memory address to an offset in your application’s binary * searches your application’s debug info for the entry describing the function at that offset * parses that debugging information entry (DIE) to…

> This is a rather strange thing to bolt onto a language. It can just be an extremely fun and cute demo, without practical application.

It can also be something that looks cool and doesn't necessarily ever get past "kinda works", but piques the interest of the core dev team and they take steps to make it work even better, resulting in the ultimate "deprecation" of this sort of thing by virtue of it being even better integrated into the core.

I don't have the context to judge the probability of that in this specific case (lots of technical nitty-gritty comes in to this sort of thing), but I've certainly seen similar things happen in other communities.

Post reply on HN