wasmer/lib/spectests/spectests

This directory contains tests for the core WebAssembly semantics, as described in Semantics.md and specified by the spec interpreter.

These files should be a direct copy of the original WebAssembly spec tests.

Tests are written in the S-Expression script format defined by the interpreter.

Autogenerated Rust test cases

These files will serve as a base for autogenerating Rust testcases when WASM_GENERATE_SPECTESTS=1 cargo build is executed (src/build_spectests.rs).

The resulting autogenerated spec tests live in the src/spectests.

Testcases

Currently supported command assertions:

• module mostly implemented (it should support named modules (module $Xx)).
• assert_return mostly implemented (it should support calls to named modules (invoke $Xx "call")).
• assert_return_canonical_nan fully implemented
• assert_return_arithmetic_nan fully implemented
• assert_trap fully implemented
• assert_invalid fully implemented (it should not require validation to be performed separate from compilation)
• assert_malformed fully implemented
• assert_uninstantiable not implemented yet
• assert_exhaustion not implemented yet
• register not implemented yet
• perform_action partially implemented, only function invocations for now

Covered spec tests

The following spec tests are currently covered:

• address.wast
• align.wast
• binary.wast
• block.wast
• br.wast
• br_if.wast
• br_table.wast
• break-drop.wast
• call.wast
• call_indirect.wast
• comments.wast
• const.wast
• conversions.wast
• custom.wast
• data.wast
• elem.wast
• endianness.wast
• exports.wast
• f32.wast
• f32_bitwise.wast
• f32_cmp.wast
• f64.wast
• f64_bitwise.wast
• f64_cmp.wast
• fac.wast
• float_exprs.wast
• float_literals.wast
• float_memory.wast
• float_misc.wast
• forward.wast
• func.wast
• func_ptrs.wast
• get_local.wast
• globals.wast
• i32.wast
• i64.wast
• if.wast
• imports.wast
• inline-module.wast
• int_exprs.wast
• int_literals.wast
• labels.wast
• left-to-right.wast
• linking.wast
• loop.wast
• memory.wast
• memory_grow.wast
• memory_redundancy.wast
• memory_trap.wast
• names.wast
• nop.wast
• return.wast
• select.wast
• set_local.wast
• skip-stack-guard-page.wast
• stack.wast
• start.wast
• store_retval.wast
• switch.wast
• tee_local.wast
• token.wast
• traps.wast
• type.wast
• typecheck.wast
• unreachable.wast
• unreached-invalid.wast
• unwind.wast
• utf8-custom-section-id.wast
• utf8-import-field.wast
• utf8-import-module.wast
• utf8-invalid-encoding.wast

Specific non-supported cases

There are some cases that we decided to skip for now to accelerate the release schedule:

• SKIP_MUTABLE_GLOBALS: Right now the Wasm parser can't validate a module with imported/exported mut globals. We decided to skip the tests until Cranelift and wasmparser can handle this (see original spec proposal). Spec tests affected:

  • globals.wast

• SKIP_CALL_INDIRECT_TYPE_MISMATCH: we implemented traps in a fast way. We haven't yet covered the type mismatch on call_indirect. Specs affected:

  • call_indirect.wast

• SKIP_CALL_UNDEFINED_ELEMENT Tables are imported into every spec module, even for modules that don't expect it. We need to figure out a way to prevent importing of objects that are not explicitly imported into the module.

Currently cranelift_wasm::ModuleEnvironment does not provide declare_table_import, etc. so there is no meaningful way of fixing this yet.

• call_indirect.wast

• SKIP_SHARED_TABLE [elem.wast] Currently sharing tables between instances/modules does not work. Below are some of the reasons it is hard to achieve:

  • Rust naturally prevents such because of the possibility of race conditions

  • ImportObject is just a wrapper, what we really care about is references to its content.

  • Instance::new contains a mutation points, the part where after getting the object (memory or table) we push values to it table[table_element_index] = func_addr

  • Instance has its own created memories and tables and references to them must outlive Instance::new()

  • Possible strategy:

    // ImportObject should be passed by ref
    Instance::new<'a>(..., &ImportObject);
    
    // Add OwnedData to Instance struct
    struct OwnedData;
    
    // For parts where mutatation is really needed
    fn get_mut(&import) -> &mut ImportObject {
        unsafe { transmute::<&ImportObject, &mut ImportObject>(import) }
    }
    

  • elem.wast

• SKIP_UNARY_OPERATION [memory_grow.wast] In some versions of MacOS this is failing (perhaps because of the chip). More info here:

Executing function c82_l299_action_invoke thread 'test_memory_grow::test_module_5' panicked at 'assertion failed: (left == right) left: Ok([I32(0)]), right: Ok([I32(31)])', /Users/distiller/project/target/release/build/wasmer-spectests-98805f54de053dd1/out/spectests.rs:32304:5 note: run with RUST_BACKTRACE=1 environment variable to display a backtrace.

failures: test_memory_grow::test_module_5

  https://circleci.com/gh/wasmerio/wasmer/9556