925: feat(runtime-core) Support closures with a captured environment as host functions r=Hywan a=Hywan
Reboot of #882 and #219.
For the moment, host functions (aka imported functions) can be regular function pointers, or (as a side-effect) closures without a captured environment. This PR extends the support of host functions to closures with a captured environment. This is required for many other features (incl. the Python integration, the Ruby integration, WebAssembly Interface Types [see #787], and so on).
This PR is the culmination of previous ones, notably #915, #916 and #917.
### General idea
The user-defined host function is wrapped inside a `wrap` function. This wrapper function initially receives a `vm::Ctx` as its first argument, which is passed to the host function when necessary. The patch keeps this behavior but it comes from `vm::FuncCtx`, which is a new structure. A `vm::FuncCtx` is held by `vm::ImportedFunc` such as:
```rust
#[repr(C)]
pub struct ImportedFunc {
pub(crate) func: *const Func,
pub(crate) func_ctx: NonNull<FuncCtx>,
}
```
where `vm::FuncCtx` is:
```rust
#[repr(C)]
pub struct FuncCtx {
pub(crate) vmctx: NonNull<Ctx>,
pub(crate) func_env: Option<NonNull<FuncEnv>>,
}
```
where `vm::FuncEnv` is:
```rust
#[repr(transparent)]
pub struct FuncEnv(pub(self) *mut c_void);
```
i.e. a raw opaque pointer.
So the wrapper function of a host function receives a `vm::Ctx`, which is used to find out the associated `FuncCtx` (by using the import backing), which holds `vm::FuncEnv`. It holds a pointer to the closure captured environment.
### Implementation details
#### How to get a pointer to a closure captured environment
A closure with a captured environment has a memory size greater than zero. This is how we detect it:
```rust
if mem::size_of::<Self>() != 0 { … }
```
To get a pointer to its captured environment, we use this statement:
```rust
NonNull::new(Box::into_raw(Box::new(self))).map(NonNull::cast)
```
(in `typed_func.rs`, in the `wrap` functions).
To reconstruct the closure based on the pointer, we use this statement:
```rust
let func: &FN = {
let func: NonNull<FN> = func_env.cast();
&*func.as_ptr()
};
```
That's basically how it works. And that's the core idea of this patch.
As a side effect, we have removed an undefined behavior (UB) in 2 places: The `mem::transmute(&())` has been removed (it was used to get the function pointer of `FN`). The transmute is replaced by `FuncEnv`, which provides a unified API, erasing the difference between host functions as closures with a captured environment, and host functions as function pointer. For a reason I ignore, the UB wasn't showing himself until this PR and a modification in the Singlepass backend. But now it's fixed.
#### Impact on `Backing`
After the modification on the `typed_func` and the `vm` modules, this is the other core idea of this patch: Updating the `backing` module so that `vm::ImportedFunc` replaces `vm::Ctx` by `vm::FuncCtx`.
When creating `vm::ImportedFunc`, a new `vm::FuncCtx` is created and its pointer is used. We are purposely leaking `vm::FuncCtx` so that the pointer is always valid. Hence the specific `Drop` implementation on `ImportBacking` to dereference the pointer, and to drop it properly.
#### Impact on the backends
Since the structure of `vm::ImportedFunc` has changed, backends must be updated. We must deref `FuncCtx` to reach its `vmctx` field.
#### Impact on `Instance`
Because `vm::ImportedFunc` has changed, it has a minor impact on the `instance` module, nothing crazy though.
Co-authored-by: Ivan Enderlin <ivan.enderlin@hoa-project.net>
The `field-offset` crate is unmaintained. When using its `offset_of!`
macro on a struct with a field of type `std::ptr::NonNull`, in release
mode, it generates a sigill.
This patch removes the `field-offset` crate, and implements a custom
`offset_of!` macro.
952: Generate C preprocessor code to hide things not on Windows r=MarkMcCaskey a=MarkMcCaskey
resolves#804
# Review
- [x] Add a short description of the the change to the CHANGELOG.md file
Co-authored-by: Mark McCaskey <mark@wasmer.io>
Co-authored-by: nlewycky <nick@wasmer.io>
923: Add probable fix of memory leak in trampoline code r=MarkMcCaskey a=MarkMcCaskey
might be what's needed for #810 ; but despite my best efforts I could not get asan working on osx, so I did not test it.
By my count this accounts for 40 bytes, so there may be another issue
edit: unless Rust is optimizing out turning a zero-sized type into a Box, in which case, this should account for 48 bytes
Co-authored-by: Mark McCaskey <mark@wasmer.io>
942: Deny missing docs in runtime core and add missing docs r=bjfish a=bjfish
<!--
Prior to submitting a PR, review the CONTRIBUTING.md document for recommendations on how to test:
https://github.com/wasmerio/wasmer/blob/master/CONTRIBUTING.md#pull-requests
-->
# Description
Deny missing docs in runtime core and add missing docs
<!--
Provide details regarding the change including motivation,
links to related issues, and the context of the PR.
-->
# Review
- [x] Add a short description of the the change to the CHANGELOG.md file
Co-authored-by: Brandon Fish <brandon.j.fish@gmail.com>
