Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Neat Rust Tricks: Passing Closures to C

blog.seantheprogrammer.com

Post reply on HN
Print view View original

51–60 of 69 posts

Re: Neat Rust Tricks: Passing Closures to C

#51

Post by a1369209993 »

richardwhiuk wrote: post #28
gpderetta wrote: post #26

Earlier quoted context omitted.

Then it wouldn't be a lexical closure.

It can be from the perspective of the calling code. Roughly speaking: thread_local! { static CBQ: Option i32>>; } #[no_mangle] extern "C" fn qsort(array: *mut i32, val: usize, callback: impl FnMut(i32, i32) -> i32); pub fn rust_qsort(array : Vec , callback: impl FnMut(i32, i32) -> i32){ CBQ.replace(Box::new(callback)).unwrap_none(); unsafe { qsort(array.as_mut_ptr(), array.len(), &rust_qsort_callback); } CBQ.take().u…

This fails horribly if called recursively (or from a signal handler). You need something like:

  wrapped_qsort(/* array,callback */)
    {
    auto tmp = CBQ;
    CBQ = wrap(callback);
    qsort(array.ptr,array.len,cbq_callback);
    CBQ = tmp; /* pop old value from stack */
    }

Re: Neat Rust Tricks: Passing Closures to C

#52

Post by Gibbon1 »

KenanSulayman wrote: post #41

Interestingly this is very similar to how I implemented passing closures into JavaScriptCore as hooks for JS class invocations ("function calls"). [0] Essentially it's taking advantage of the fact that closures are static methods with "implicit" data pointers. It should be fairly obvious that this is a massive violation of safety and undefined behavior, and most likely to break when debugging symbols etc. are inserte…

I read an article by a guy talking about stupid C tricks. One of them was to 'mangle' raw pointers into an index before passing them. And then de-magle them to get back a raw pointer. Advantages are you can pass meta data with the 'pointer'. Which also allows you to invalidate a pointer. The pointer can't be dereferenced. The enclosed variables/data isn't accessible and cannot be modified by the target.

For callbacks the overhead likely isn't significant.

Re: Neat Rust Tricks: Passing Closures to C

#53

Post by 0db532a0 »

loeg wrote: post #29
tedunangst wrote: post #14

Now call qsort with a closure.

I don't think qsort changes anything about the mechanism described in the blog post, but maybe I'm missing something. (I.e., use qsort_r...) (qsort is really only for C. Other languages can potentially inline the comparison function, so using FFI for that is kind of insane.)

Gets me thinking: I wonder how good Intel CPUs are with dealing with this sort of thing. Can the CPU detect repeated jumps to comparators and in-line them from there? I’d be interested to see a comparative benchmark.

Re: Neat Rust Tricks: Passing Closures to C

#54

Post by marcan_42 »

cheez wrote: post #2

Is this a neat trick or just standard operating procedure for calling C from ? As it was billed as a trick, I was expecting some sort of runtime code generation to pass the data pointer and some jump instruction to jump to the right spot and unpack the data pointer. Maybe I just overcomplicate things ;-)

It's standard procedure. I've done the exact same thing when wrapping C APIs into Python using Cython, several times. You pass the Python closure as the void *data and then register a shared generic callback which casts it and calls it. Easy. Getting the memory management right is slightly tricky, but not too bad.

Fun fact: you can't safely do this with ctypes. Since it is called as pure Python, it cannot do watertight Python exception handling in a callback context (because even if you have a try/except block, an exception can always happen right before or after it), and ctypes provides no usable internal way of doing it - it just eats exceptions inside callbacks. This is what motivated me to rewrite Ceph's librbd bindings from ctypes to Cython.

Re: Neat Rust Tricks: Passing Closures to C

#57

Post by alkonaut »

dmitrygr wrote: post #55

> If you’re not familiar with C’s syntax, here’s the equivalent signature in Rust Author is hilarious. Who is familiar with that but not c?

I came to Rust without writing C before. Most of my experience with C comes from problems exactly like this. I doubt I'm alone in this.

Re: Neat Rust Tricks: Passing Closures to C

#58

Post by pjmlp »

jonny383 wrote: post #30

Rust is already doomed. The amount of literature being published about either comparisons or compatibility with C is a strong indicator C is here to stay.

Rome wasn't built in a day.

As for C staying around, unfortunately yes, until we get rid of POSIX based OSes, C will be around.

After all we need to keep those Security conferences alive. /s

Re: Neat Rust Tricks: Passing Closures to C

#59

Post by pjmlp »

jonny383 wrote: post #45
MaulingMonkey wrote: post #43

Earlier quoted context omitted.

> It shows that people are still struggling with changing the way in which they write software to the "rust" way. It shows no such thing. I generally work on relatively small ~1MLOC C++ codebases. There are codebases out there measured in the hundreds of MLOC. These are not the comparatively tiny javascript codebases you find React used in - where additional milliseconds of download / parse / evaluation time has a me…

>There are codebases out there measured in the hundreds of MLOC. I agree with your argument, but I think in practice trying to "port" something with hundreds of MLOC is a losing battle (especially away from C). By the time you finished porting to rust (or your other language of the week), rust will likely have come and gone and will be been replaced by something either better or "better". IMO people should spend less…

Given the 30+ years of proven security exploits due to memory corruption, the C community has proven multiple times that it doesn't care about those solutions, except when required to do so in certified software.

That Solaris, iOS and in the future Android, pursue hardware memory tagging as workaround for memory corruption exploits, it is a proof how bad the situation in terms of security is.

Re: Neat Rust Tricks: Passing Closures to C

#60

Post by barrkel »

mmastrac wrote: post #13
twic wrote: post #10

Earlier quoted context omitted.

The whole point of jgtrosh's link is that there is a way to hide data behind a function pointer, so Rust could convert any closure to a function pointer. But it requires writable-and-executable memory, so it's a pretty bad idea (in GCC's implementation, that memory is on the stack, which is an extra bad idea, but i don't think it needs to be).

Definitely. Technically this can also be done via static code trampolines that are mmap'd as well [1]. That approach has been used on iOS in the past to turn blocks into raw function pointers. If you have a platform that allows W+X on code (yikes!), you can do [2] as well. [1] https://github.com/plausiblelabs/plblockimp/blob/master/Sour... [2] https://www.mikeash.com/pyblog/friday-qa-2010-02-12-trampoli...

For example, every platform that has a virtual machine with JIT compilation support.
Post reply on HN