Rust: Rust functions that can be called from C
We have a lot of scenarios now where people creating bindings want to be able to provide a callback that a C function can call. The current solution is to write a native C function that uses some unspecified internal APIs to send a message back to Rust code. Ideally it involves writing no C code.
Here is a minimal solution for creating functions in Rust that can be called from C code. The gist is: 1) we have yet another kind of function declaration, 2) this function cannot be called from Rust code, 3) it's value can be taken as an opaque unsafe pointer, 4) it bakes in the stack switching magic and adapts from the C ABI to the Rust ABI.
Declarations of C-to-Rust (crust) functions:
crust fn callback(a: *whatever) {
}
Getting an unsafe pointer to a C ABI function:
let callbackptr: *u8 = callback;
We could also define some type specifically for this purpose.
Compiler implementation:
It's mostly straightforward, but trans gets ugly. In trans we will need to do basically the opposite of what we do for native mod functions:
- Generate a C ABI function using the declared signature
- Generate a shim function that takes the C arguments in a struct
- The C function stuffs the arguments into a struct
- The C function calls upcall_call_shim_on_rust_stack with the struct of arguments and the address of the shim function
- Generate a Rust ABI function using the declared signature
- The shim function pulls the arguments out of the struct and calls the Rust function
Runtime implementation:
The runtime has to change in a few ways to make this happen:
- A new upcall for switching back to the Rust stack
- Tasks need to maintain a stack of Rust contexts and C contexts
- Needs a strategy to deal with failure after reentering the Rust stack
- Needs a strategy to deal with yielding after reentering the Rust stack
Failure:
We can't simply throw an exception after reentering the Rust stack because there's no guarantee the native code can be unwound with C++ exceptions. The Go language apparently will just skip over all the native frames in this scenario, leaking everything along the way. We, instead will abort - if the user want's to avoid catastrophic failure they should use their Rust callback to dispatch a message and immediately return.
Yielding:
Without changes to the way we handle C stacks we cannot allow Rust functions to context-switch to the scheduler after reentering the Rust stack from C code. I see two solutions:
1) Yielding is different after reentering the Rust stack and simply blocks. Tasks that want to do this should make sure they have their own scheduler (#1721).
2) Instead of running native code using the scheduler's stack, tasks will check out C stacks from a pool located in each scheduler. Each time a task reenters the C stack it will check if it already has one and reuse it, otherwise it will request a new one from the scheduler. This would allow Rust code to always yield normally without tying up the scheduler.
I prefer the second option.
See also #1508
brson
All 16 comments
Sorry to jump in here, but I'd like to stress that calling C functions should be fast, as in _blazing_ fast. If I want to write a game in rust using a C library like Allegro, SDL, or Opengl, this is essential. Otherwise the game will slow down in the rendering code where there are a lot of C calls, which is unacceptable. The default Go language compiler with cgo has such problems.
So I'd prefer a solution that is fast, even though it may restrict what the function on the Rust side can do.
Also, wouldn't it be an idea to use "native fn" in stead of "crust fn" or does that have another planned meaning?
beoran
on 2 Feb 2012
@beoran you have this backward. We're talking about C calling Rust. Calling C functions from Rust is already pretty fast (could be made somewhat faster).
nikomatsakis
on 2 Feb 2012
Ok, I see. Is there any way I can help with speeding up calling C from rust?
beoran
on 2 Feb 2012
@beoran as I said, it's kind of orthogonal to this issue... but probably the best thing you could do is make up a benchmark showing how the performance is inadequate. :)
nikomatsakis
on 2 Feb 2012
OK, I will do that when I get far enough in wrapping Allegro to compare the overhead of calling it from rust Rust with calling it from C. I'll leave this issue alone for now and I'll open a new one once I have the benchmark.
beoran
on 2 Feb 2012
Could we avoid one of the copies of the arguments by having the C function write directly to the Rust stack, as Rust->C calls currently do?
pcwalton
on 6 Feb 2012
I'm hoping that the final copy of the arguments out of the shim struct and into the arguments of the rust function will be eliminated through inlining. I'm not sure if that's what you are referring to though.
I believe that our C calls currently copy the arguments into a struct on the Rust stack then copy that struct to the C stack.
brson
on 6 Feb 2012
@pcwalton Rust->C do not currently write directly into the C stack because that is specific to i386. I wanted to avoid having to write code that was specific to a particular calling convention, even at the price of some performance, in order to get 64-bit working. (I think such optimizations might make sense now, though---particularly as #1402 points out that LLVM doesn't really handle the calling conventions completely _anyway_)
nikomatsakis
on 6 Feb 2012
After reading the stack switching function, the arg struct isn't copied across stacks at all, the pointer to the previous stack is just passed to the function that runs on the new stack, which makes perfect sense.
brson
on 8 Feb 2012
The arg struct is not copied, but the shim function will load values out of it and re-push them onto the new stack. The old code used to literally write the argument values directly into the target stack. This made sense on i386 but on x86_64 it's much more complex to figure out which values will go on the stack etc.
nikomatsakis
on 8 Feb 2012
After some testing I've found that it is going to be very difficult for crust functions to guarantee that they don't fail. What currently happens is that, when a top-level task (like main) fails every task is told to fail, so as soon as the callback tries to send a message (or when it returns from sending a message) it can end up failing and causing the runtime to abort abnormally.
brson
on 14 Feb 2012
I guess we can change rust_task to ignore kill requests once tasks have reentered the rust stack. For tasks implementing event loops they can peek at some monitor port looking for a message indicating that the runtime is failing and figure out how to terminate gracefully.
brson
on 14 Feb 2012
So when a top-level task fails, it propagates errors to its children? I guess I don't understand our error propagation model, I thought it went from the leaves up. It seems like code running on C stacks ought to be able to run in an unsupervised task or something like that.
nikomatsakis
on 14 Feb 2012
It basically acts as if 'main' is supervised by the kernel, so if main fails everything fails.
brson
on 14 Feb 2012
I'm calling this done. There's a little cleanup, and I've filed separate bugs for remaining issues.
brson
on 14 Feb 2012
Please forgive this resurrection, but this issue is linked from a y-combinator piece and a few other sites, and I recently had a noob ask about it, so I'm noting that, per this issue & later changes, calling Rust from C is simple:
#[no_mangle]
pub extern fn hello_rust() -> *const u8 {
"Hello, world!\0".as_ptr()
}
#include "stdio.h"
const char *hello_rust(void);
int main(void) {
printf("%.32s\n", hello_rust());
}
alexchandel
on 14 Jul 2016
Related issues
modsec
·
3Comments
drewcrawford
·
3Comments
dwrensha
·
3Comments
eddyb
·
3Comments
defuz
·
3Comments
Most helpful comment
Please forgive this resurrection, but this issue is linked from a y-combinator piece and a few other sites, and I recently had a noob ask about it, so I'm noting that, per this issue & later changes, calling Rust from C is simple: