wasmer/lib/spectests

Wasmer Spectests

Wasmer is a standalone JIT WebAssembly runtime, aiming to be fully compatible with Emscripten, Rust and Go. Learn more.

This crate allows to test the Wasmer runtime against the official specification test suite.

This lib contains tests for the core WebAssembly semantics, as described in Semantics.md and specified by the spec interpreter.

SIMD wast specs are also added here.

These files should be a direct copy of the original WebAssembly spec tests.

Tests are written in the S-Expression script format defined by the interpreter.

Version

The spectests were last updated at WebAssembly/spec commit a221f2574d7106e92cf8abaf05d5bb1131b19d76.

Testcases

Currently supported command assertions:

• module fully implemented
• assert_return fully implemented
• assert_return_canonical_nan fully implemented
• assert_return_arithmetic_nan fully implemented
• assert_trap fully implemented
• assert_invalid fully implemented (it should not require validation to be performed separate from compilation)
• assert_malformed fully implemented
• assert_uninstantiable not implemented, no usages found
• assert_exhaustion fully implemented
• register fully implemented
• perform_action fully implemented

Covered spec tests

See tests/excludes.txt for current coverage.

Specific non-supported cases

There are some cases that we decided to skip for now to accelerate the release schedule:

• SKIP_CALL_INDIRECT_TYPE_MISMATCH: we implemented traps in a fast way. We haven't yet covered the type mismatch on call_indirect. Specs affected:

  • call_indirect.wast

• SKIP_CALL_UNDEFINED_ELEMENT Tables are imported into every spec module, even for modules that don't expect it. We need to figure out a way to prevent importing of objects that are not explicitly imported into the module.

Currently cranelift_wasm::ModuleEnvironment does not provide declare_table_import, etc. so there is no meaningful way of fixing this yet.

• call_indirect.wast

• SKIP_SHARED_TABLE [elem.wast] Currently sharing tables between instances/modules does not work. Below are some of the reasons it is hard to achieve:

  • Rust naturally prevents such because of the possibility of race conditions

  • ImportObject is just a wrapper, what we really care about is references to its content.

  • Instance::new contains a mutation points, the part where after getting the object (memory or table) we push values to it table[table_element_index] = func_addr

  • Instance has its own created memories and tables and references to them must outlive Instance::new()

  • Possible strategy:

    // ImportObject should be passed by ref
    Instance::new<'a>(..., &ImportObject);
    
    // Add OwnedData to Instance struct
    struct OwnedData;
    
    // For parts where mutatation is really needed
    fn get_mut(&import) -> &mut ImportObject {
        unsafe { transmute::<&ImportObject, &mut ImportObject>(import) }
    }
    

  • elem.wast

• SKIP_UNARY_OPERATION [memory_grow.wast] In some versions of MacOS this is failing (perhaps because of the chip). More info here:

Executing function c82_l299_action_invoke
thread 'test_memory_grow::test_module_5' panicked at 'assertion failed: `(left == right)`
 left: `Ok([I32(0)])`,
right: `Ok([I32(31)])`', /Users/distiller/project/target/release/build/wasmer-spectests-98805f54de053dd1/out/spectests.rs:32304:5
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace.


failures:
   test_memory_grow::test_module_5

https://circleci.com/gh/wasmerio/wasmer/9556

Development

To test locally, try the following commands:

RUST_BACKTRACE=1 cargo test --features clif -- --nocapture
RUST_BACKTRACE=1 cargo test --features llvm -- --nocapture
RUST_BACKTRACE=1 cargo +nightly test --features singlepass -- --nocapture