I only spent about an hour on the rust + wasm app for fun, so I treated wasm-bindgen and wasm-pack as black boxes and didn't really try to understand the inner workings. You're probably right about serialization, but I'm not sure it has to copy huge buffers around.
Look, the supported types in wasm-bindgen are somewhat restricted. The full list is here [1]. Other than "atomic" types like numbers and pointers (I assume the opaque types are treated as pointers too), the interesting one are str/String (basically &[u8] under the hood) and number slices. It's apparently not great if you have to copy these around, but since these are well-aligned, why can't you just pass the starting address and length? Again, I don't know enough about wasm or its current state to say if this is doable. Maybe it doesn't like "someone else's memory" at the moment.
(By the way, wasm-bindgen allows you to access pub fields of structs from JS, but they have to be Copy. That is to say, the opaque types are not completely opaque, but I'm not sure if the pub field access is achieved through implicit getter methods or serialized in the first place.)
Anyway, the frictions might be a big problem if you're doing React-style DOM re-renders all the time, but in my app, I'm only running the computation heavy and memory sensitive tasks in wasm, and communications are few and far between, making it a non-issue.
[1] https://rustwasm.github.io/wasm-bindgen/reference/types.html