Understanding every byte in a WASM module
danielmangum.com
Understanding every byte in a WASM module
1–10 of 38 posts
Re: Understanding every byte in a WASM module
#2This is pretty great! I highly recommend building a parser for Wasm, it shows off the nice design that it can be compiled in a streaming fashion and other nice properties.
I built a small parser for a wasm runtime I was going to hack at, although that lost steam and now I just contribute to wasmtime, but it was a great way to learn. The source if you’re interested in C++20 and coroutines: https://github.com/rockwotj/wasmcc
Re: Understanding every byte in a WASM module
#3A while back I ran through some exercises that had me writing WAT directly. It's a great way to understand what WASM is doing, and is surprisingly easy to pick up (though of course you wouldn't want to write an actual program with it). It's just system calls with s-expressions.
Re: Understanding every byte in a WASM module
#4This is pretty great! I highly recommend building a parser for Wasm, it shows off the nice design that it can be compiled in a streaming fashion and other nice properties. I built a small parser for a wasm runtime I was going to hack at, although that lost steam and now I just contribute to wasmtime, but it was a great way to learn. The source if you’re interested in C++20 and coroutines: https://github.com/rockwotj/…
> the nice design
But integers are read 7 bits at a time. Wouldn't it be much easier to use 8 bits?
Re: Understanding every byte in a WASM module
#5I really appreciate this article. I’ve noticed that modern streaming web players have started using WASM as part of their obfuscation toolbox. Understanding WASM modules is important for reverse engineering.
Re: Understanding every byte in a WASM module
#6This is pretty great! I highly recommend building a parser for Wasm, it shows off the nice design that it can be compiled in a streaming fashion and other nice properties. I built a small parser for a wasm runtime I was going to hack at, although that lost steam and now I just contribute to wasmtime, but it was a great way to learn. The source if you’re interested in C++20 and coroutines: https://github.com/rockwotj/…
> the nice design But integers are read 7 bits at a time. Wouldn't it be much easier to use 8 bits?
If you use all 8 bits of a byte for data, you need a separate way to store the length, or you need to make everything fixed-width which wastes space.
"Varints" or variable-length integers, with 7 bits of data and 1 continuation bit per byte, are used in a fairly wide variety of formats (e.g. Protobuf, SQLite and Lucene). They're compact when most of your integers are small, while still being quite simple and fast to encode/decode.
Faster alternatives tend to be more complicated, and more compact alternatives tend to be slower.
Re: Understanding every byte in a WASM module
#7This is pretty great! I highly recommend building a parser for Wasm, it shows off the nice design that it can be compiled in a streaming fashion and other nice properties. I built a small parser for a wasm runtime I was going to hack at, although that lost steam and now I just contribute to wasmtime, but it was a great way to learn. The source if you’re interested in C++20 and coroutines: https://github.com/rockwotj/…
> the nice design But integers are read 7 bits at a time. Wouldn't it be much easier to use 8 bits?
This is basically the design of UTF-8 or varint encoding. The integer's representation is read one byte (8 bits) at a time, which can then be interpreted by looking at the first bit which says whether the integer is "done" or more bytes need to be read.
(If we were using all 8 bits for the representation and an additional leading bit for whether it continues, we'd end up having to read 9 bits or something, which is awkward and not aligned with what the machine actually supports reading.)
Re: Understanding every byte in a WASM module
#8This is pretty great! I highly recommend building a parser for Wasm, it shows off the nice design that it can be compiled in a streaming fashion and other nice properties. I built a small parser for a wasm runtime I was going to hack at, although that lost steam and now I just contribute to wasmtime, but it was a great way to learn. The source if you’re interested in C++20 and coroutines: https://github.com/rockwotj/…
> the nice design But integers are read 7 bits at a time. Wouldn't it be much easier to use 8 bits?
It is a balance between simplicity, speed and size. They choose size, honesty leb128 isn’t *too* complex (you can see an implementation in my repo I pointed too). But yes they don’t take all the simplicity tradeoffs.
Re: Understanding every byte in a WASM module
#9A while back I ran through some exercises that had me writing WAT directly. It's a great way to understand what WASM is doing, and is surprisingly easy to pick up (though of course you wouldn't want to write an actual program with it). It's just system calls with s-expressions.
WAT is a fun format! I'm curating some readable samples here -- https://github.com/eliben/wasm-wat-samples/
Re: Understanding every byte in a WASM module
#10A while back I ran through some exercises that had me writing WAT directly. It's a great way to understand what WASM is doing, and is surprisingly easy to pick up (though of course you wouldn't want to write an actual program with it). It's just system calls with s-expressions.
For some time I've been fascinated by the codebase of a small WAT compiler written in JavaScript.
https://github.com/stagas/wat-compiler/blob/main/story.txt
And I mean "small" as a real complement to how readable and compact the entire compiler is. It's been a great way to appreciate the design of the WASM text format and WASM overall. It's not a Lisp but has a similar feel to it.
I've been meaning to get more fluent at writing WAT directly, not for any practical purpose but just for pleasure of it. I could see myself gradually building up some abstractions to help me deveolp larger programs, perhaps a slightly higher-level language.