wasmer/lib/runtime/spectests/README.md

This directory contains tests for the core WebAssembly semantics, as described in Semantics.md and specified by the spec interpreter.

This files should be a direct copy of the original WebAssembly spec tests.

Tests are written in the S-Expression script format defined by the interpreter.

Autogenerated Rust test cases

This files will serve as base for autogenerating Rust testcases when WASM_GENERATE_SPECTESTS=1 cargo build is executed (src/build_spectests.rs).

The result autogenerated spectests live in the src/spectests directory.

Testcases

Currently supported command assertions:

• Module mostly implemented (it should support named modules (module $Xx)).
• AssertReturn mostly implemented (it should support calls to named modules (invoke $Xx "call")).
• AssertReturnCanonicalNan fully implemented
• AssertReturnArithmeticNan fully implemented
• AssertTrap fully implemented
• AssertInvalid Fully implemented (it should not require to do validation separate from compilation)
• AssertMalformed Fully implemented
• AssertUninstantiable not implemented yet
• AssertExhaustion not implemented yet
• Register not implemented yet
• PerformAction partially implemented, only function invokations for now

Covered spectests

This spectests are currently covered:

• address.wast ✅
• align.wast ✅
• binary.wast ✅
• block.wast ✅
• br.wast ✅
• br_if.wast ✅
• br_table.wast ✅
• break-drop.wast ✅
• call.wast ✅
• call_indirect.wast ✅
• comments.wast ✅
• const.wast ✅
• conversions.wast ✅
• custom.wast ✅
• data.wast ✅
• elem.wast
• endianness.wast ✅
• exports.wast ✅
• f32.wast ✅
• f32_bitwise.wast ✅
• f32_cmp.wast ✅
• f64.wast ✅
• f64_bitwise.wast ✅
• f64_cmp.wast ✅
• fac.wast ✅
• float_exprs.wast ✅
• float_literals.wast ✅
• float_memory.wast ✅
• float_misc.wast ✅
• forward.wast ✅
• func.wast ✅
• func_ptrs.wast ✅
• get_local.wast ✅
• globals.wast ✅
• i32.wast ✅
• i64.wast ✅
• if.wast ✅
• imports.wast
• inline-module.wast
• int_exprs.wast ✅
• int_literals.wast ✅
• labels.wast ✅
• left-to-right.wast ✅
• linking.wast
• loop.wast ✅
• memory.wast ✅
• memory_grow.wast ✅
• memory_redundancy.wast ✅
• memory_trap.wast ✅
• names.wast ✅
• nop.wast ✅
• return.wast ✅
• select.wast ✅
• set_local.wast ✅
• skip-stack-guard-page.wast
• stack.wast ✅
• start.wast ✅
• store_retval.wast ✅
• switch.wast ✅
• tee_local.wast ✅
• token.wast ✅
• traps.wast ✅
• type.wast ✅
• typecheck.wast ✅
• unreachable.wast
• unreached-invalid.wast
• unwind.wast ✅
• utf8-custom-section-id.wast
• utf8-import-field.wast
• utf8-import-module.wast
• utf8-invalid-encoding.wast

Specific non-supported cases

There are some cases that we decided to skip for now to fasten the time to release:

• SKIP_MUTABLE_GLOBALS: Right now the WASM parser can't validate a module with imported/exported mut globals. We decided to skip the tests until Cranelift and wasmparser can handle this (original spec proposal: https://github.com/WebAssembly/mutable-global). Spectests affected:

  • globals.wast

• SKIP_CALL_INDIRECT_TYPE_MISMATCH: we implemented traps in a fast way. We haven't covered yet the type mismatch on call_indirect. Specs affected:

  • call_indirect.wast

• SKIP_CALL_UNDEFINED_ELEMENT Tables are imported into every spec module, even for modules that don't expect it. We need to figure out a way to prevent import of objects that are not explicitly imported into the module.

Currently cranelift_wasm::ModuleEnvironment does not provide declare_table_import, etc. so there is no meaningful way of fixing this yet.

• call_indirect.wast

• SKIP_SHARED_TABLE [elem.wast] Currently sharing tables between instances/modules does not work. Below are some of the reasons it is hard to achieve.

  • Rust naturally prevents such because of the possibility of race conditions

  • ImportObject is just a wrapper, what we really care about is references to its content.

  • Instance::new contains a mutation points, the part where after getting the object (memory or table) we push values to it table[table_element_index] = func_addr

  • Instance has its own created memories and tables and references to them must outlive Instance::new()

  • Possible strategy

    // ImportObject should be passed by ref
    Instance::new<'a>(..., &ImportObject);
    
    // Add OwnedData to Instance struct
    struct OwnedData;
    
    // For parts where mutatation is really needed
    fn get_mut(&import) -> &mut ImportObject {
        unsafe { transmute::<&ImportObject, &mut ImportObject>(import) }
    }
    

  • elem.wast