Implement debug compilation mode for runtime sanity checking and assertions

[osa1]

There are lots of sanity checks and assertions that will help testing and debugging the runtime system but are also too expensive (in terms of runtime and binary size) to enable always. One example is resetting the scratch space in the current copying garbage collector: it cost us significant gas when it was enabled in #1750 so we had to remove it.

For these checks I think we'll need a compilation mode (let's call it "debug", maybe enable with -debug passed to moc) that will generate the final .wasm file with the assertions and sanity checks. These binaries will be used when testing and debugging. In normal operation we'll build without -debug.

Initially debug compilation mode could simply link moc-generated code with the Rust parts of the RTS built in debug mode (instead of release mode). In Rust, debug mode does overflow and underflow checking in arithmetic (unless methods like wrapping_add used, which allow overflows and underflows) which is already quite useful (having debugged underflows in a GC in the past ...). We could then add more checks manually using if cfg!(debug_assertions) { ... } or debug_assert_eq! etc.

@nomeata @ggreif what do you think?

[nomeata]

Yes, sounds useful, absolutely in favor.

We probably want to run the test suite in both modes then? Would be a shame to not see a bug that’s hidden by resetting the scratch space. The output should be identical, so we could use the same ok/… files, right?

Implementation-wise, I’d say we build both mo-rts.wasm and mo-rts.wasm using rts/Makefile; embed both into the moc binary, and then use the right one based on -debug or not.

The IR typechecker is a similar thing that we would probably not want to to run by default.

[osa1]

We probably want to run the test suite in both modes then? Would be a shame to not see a bug that’s hidden by resetting the scratch space. The output should be identical, so we could use the same ok/… files, right?

Yes to all.

Implementation-wise, I’d say we build both mo-rts.wasm and mo-rts.wasm using rts/Makefile; embed both into the moc binary, and then use the right one based on -debug or not.

👍

The IR typechecker is a similar thing that we would probably not want to to run by default.

Yeah once we have the flag we can also do more compile-time checking as well. Or maybe generate different code that does sanity checking. The latter would be difficult if we had separate compilation (as debug code would have to be compatible with non-debug code generated for libraries), but with whole-program compilation it's easier.