Neat Rust Tricks: Passing Closures to C

Post by richardwhiuk » Tue, Nov 26, 2019, 10:25 AM UTC

a1369209993 wrote: post #51

richardwhiuk wrote: post #28

Earlier quoted context omitted.

It can be from the perspective of the calling code. Roughly speaking: thread_local! { static CBQ: Option i32>>; } #[no_mangle] extern "C" fn qsort(array: *mut i32, val: usize, callback: impl FnMut(i32, i32) -> i32); pub fn rust_qsort(array : Vec , callback: impl FnMut(i32, i32) -> i32){ CBQ.replace(Box::new(callback)).unwrap_none(); unsafe { qsort(array.as_mut_ptr(), array.len(), &rust_qsort_callback); } CBQ.take().u…

This fails horribly if called recursively (or from a signal handler). You need something like: wrapped_qsort(/* array,callback */) { auto tmp = CBQ; CBQ = wrap(callback); qsort(array.ptr,array.len,cbq_callback); CBQ = tmp; /* pop old value from stack */ }

Post by cheez » Tue, Nov 26, 2019, 3:20 PM UTC

marcan_42 wrote: post #54

cheez wrote: post #2

Is this a neat trick or just standard operating procedure for calling C from ? As it was billed as a trick, I was expecting some sort of runtime code generation to pass the data pointer and some jump instruction to jump to the right spot and unpack the data pointer. Maybe I just overcomplicate things ;-)

It's standard procedure. I've done the exact same thing when wrapping C APIs into Python using Cython, several times. You pass the Python closure as the void *data and then register a shared generic callback which casts it and calls it. Easy. Getting the memory management right is slightly tricky, but not too bad. Fun fact: you can't safely do this with ctypes. Since it is called as pure Python, it cannot do watertig…

Post by dfox » Tue, Nov 26, 2019, 6:13 PM UTC

wahern wrote: post #44

dfox wrote: post #35

Earlier quoted context omitted.

qsort(3) or even ftw(3) is the simple case. You can either dynamically generate trampoline code with exactly bounded dynamic scope (and even do the gcc-style executable stack hack) or simply stash the whole context in some TLS region and completely sidestep the whole issue. Side point: ftw(3) is much more interesting unix API to call from some FFI layer than qsort(3). And I spent about a year pestering people from Su…

And it appears that you succeeded. Awesome! It seems Solaris has been adding many BSD and, especially, Linux compatibility APIs lately. It seems too little, too late; or perhaps the initiative is part of their effort to EoL Solaris, providing an upgrade path to Linux.

Post by imstuff » Wed, Nov 27, 2019, 6:58 AM UTC

tedunangst wrote: post #14

Now call qsort with a closure.

    extern "C" {
        #![allow(improper_ctypes)]
        fn qsort(
            ptr: *mut (), count: usize, size: usize, 
            comp: extern "C" fn (*const (), *const ()) -> i32,
        );
        fn qsort_r(
            ptr: *mut (), count: usize, size: usize,
            comp: extern "C" fn (*const (), *const (), *mut ()) -> i32,
            context: *mut ()
        ) -> usize;
    }
    #[derive(Debug)]
    pub struct CQsortError;
    /// calls qsort or qsort_r from libc
    pub fn rust_qsort i32>(items: &mut [T], mut compare: F) -> Result {
        #![allow(improper_ctypes)]
        extern "C" fn call_cmp_fn_with_context Out>(
            a: *const (), b: *const (), context: *mut ()
        ) -> Out {
            unsafe { 
                (context as *mut F).as_mut().unwrap()(
                    (a as *const T).as_ref().unwrap(), 
                    (b as *const T).as_ref().unwrap(),
                ) 
            }
        }
        /// for zero-sized F
        extern "C" fn call_cmp_fn_static Out>(
            a: *const (), b: *const (),
        ) -> Out {
            unsafe {
                // HACK: 
                // if we use core::ptr::null_mut, the compiler thinks it's invalid and makes the funtion always crash
                // however, it can be anything since it "points" to a zst
                // the compiler thinks 1 is a valid pointer
                std::mem::transmute::(1_usize)(
                    (a as *const T).as_ref().unwrap(), 
                    (b as *const T).as_ref().unwrap(),
                )
            }
        }

        unsafe {
            // it doesn't have any state, call qsort without context
            if std::mem::size_of::() == 0 {
                qsort(items.as_mut_ptr() as *mut (), items.len(), std::mem::size_of::(), 
                    call_cmp_fn_static::);
                Ok(())
            } else {
                if qsort_r(items.as_mut_ptr() as *mut (), items.len(), std::mem::size_of::(),
                    call_cmp_fn_with_context::, &mut compare as *mut _ as *mut ()) == 0 
                {
                    Ok(())
                } else {
                    Err(CQsortError)
                }
            }
        }
    }

Post by gpderetta » Thu, Nov 28, 2019, 5:28 PM UTC

richardwhiuk wrote: post #28

gpderetta wrote: post #26

Earlier quoted context omitted.

Then it wouldn't be a lexical closure.

It can be from the perspective of the calling code. Roughly speaking: thread_local! { static CBQ: Option i32>>; } #[no_mangle] extern "C" fn qsort(array: *mut i32, val: usize, callback: impl FnMut(i32, i32) -> i32); pub fn rust_qsort(array : Vec , callback: impl FnMut(i32, i32) -> i32){ CBQ.replace(Box::new(callback)).unwrap_none(); unsafe { qsort(array.as_mut_ptr(), array.len(), &rust_qsort_callback); } CBQ.take().u…