Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

An Urgent Notice from AssemblyScript

assemblyscript.org

Post reply on HN
Print view View original

41–50 of 55 posts

Re: An Urgent Notice from AssemblyScript

#41

Post by paulddraper »

kevingadd wrote: post #29
paulddraper wrote: post #27

Earlier quoted context omitted.

I don't understand. Can't you just abstract this? Python has several internal string representations to reduce conversions. Really "UTF-16 string" and "UTF-8 string" are code smells. Applications should be using character/code point sequences or byte sequences. Using code unit sequences is....bizzare. (Yes I know Java, C#, JS has chosen to do that, but a new language has an opportunity to improve.)

You can abstract it, but AssemblyScript did not. So it's not a trivial change, it's a complex migration. Similarly, you can use UTF-8 in Java and C#, but you can't just "switch" them over to the encoding directly, it has to be exposed via new types/etc.

If I understand correctly, AssemblyScript has been building ahead of specs on Interface Types?

I sympathize with pain, but the bleeding edge of tech does...bleed.

Re: An Urgent Notice from AssemblyScript

#42

Post by paulddraper »

trusktr wrote: post #30
kevingadd wrote: post #29

Earlier quoted context omitted.

You can abstract it, but AssemblyScript did not. So it's not a trivial change, it's a complex migration. Similarly, you can use UTF-8 in Java and C#, but you can't just "switch" them over to the encoding directly, it has to be exposed via new types/etc.

Note that AssemblyScript rides on TypeScript language syntax. How would ``` let foo: string = "whatever" ``` be able to work in any similar sense as TS/JS if? How can that map to multiple string types? The idea is both AS and TS use the same syntax for strings, and are compatible across boundaries (TS for JS side, AS for Wasm side). Having multiple string types is possible, but this would greatly reduce developer erg…

First you have to decide whether AssemblyScript is a language, or a compiler for an existing language.

If it's a language, then it gets to decide what ```let foo: string = "whatever"``` means.

If it's a compiler for an existing language, then semantics have already been decided, and the compiler has to implement it.

But none of the precludes abstraction to reduce data type conversions.

Re: An Urgent Notice from AssemblyScript

#43

Post by trusktr »

kevingadd wrote: post #28
zodiakzz wrote: post #24

Earlier quoted context omitted.

JavaScript is called the entire "web" now? HTML and CSS work with UTF-8 just fine and the majority of the WWW uses UTF-8 to serve them.

The canonical representation of DOM content is DOMString ( https://developer.mozilla.org/en-US/docs/Web/API/DOMString ), which is not UTF-8. Your HTML being encoded in UTF-8 is irrelevant, it gets decoded when it's loaded into whatever the canonical representation is. Your HTML could be in Shift-JIS or ASCII or whatever and not UTF-8, same difference.

This is exactly right. UTF-8 is the transmission format that your HTML gets sent in, but it is not the format of strings in JavaScript at runtime.

The problem being discussed is about runtime interoperability between JS (with WTF-16 string format) and WebAssembly.

Re: An Urgent Notice from AssemblyScript

#44

Post by trusktr »

paulddraper wrote: post #42
trusktr wrote: post #30

Earlier quoted context omitted.

Note that AssemblyScript rides on TypeScript language syntax. How would ``` let foo: string = "whatever" ``` be able to work in any similar sense as TS/JS if? How can that map to multiple string types? The idea is both AS and TS use the same syntax for strings, and are compatible across boundaries (TS for JS side, AS for Wasm side). Having multiple string types is possible, but this would greatly reduce developer erg…

First you have to decide whether AssemblyScript is a language, or a compiler for an existing language. If it's a language, then it gets to decide what ```let foo: string = "whatever"``` means. If it's a compiler for an existing language, then semantics have already been decided, and the compiler has to implement it. But none of the precludes abstraction to reduce data type conversions.

AssemblyScript is a compiler that aims to compile TypeScript code (with slight differences to be able to make sense in Wasm, though trying to minimize those differences) into Wasm. To remain compatible with TypeScript (which is AssemblyScript's goal) and be an optimal language for Wasm that will communicate with TS/JS on the other side, the type `string` would need to be in the same format to avoid any perf hit or data loss while passing thing if that type across the Wasm-JS boundary.

I'm not sure what type of compiler to label that as, but that's the goal.

Re: An Urgent Notice from AssemblyScript

#45

Post by paulddraper »

trusktr wrote: post #44
paulddraper wrote: post #42

Earlier quoted context omitted.

First you have to decide whether AssemblyScript is a language, or a compiler for an existing language. If it's a language, then it gets to decide what ```let foo: string = "whatever"``` means. If it's a compiler for an existing language, then semantics have already been decided, and the compiler has to implement it. But none of the precludes abstraction to reduce data type conversions.

AssemblyScript is a compiler that aims to compile TypeScript code (with slight differences to be able to make sense in Wasm, though trying to minimize those differences) into Wasm. To remain compatible with TypeScript (which is AssemblyScript's goal) and be an optimal language for Wasm that will communicate with TS/JS on the other side, the type `string` would need to be in the same format to avoid any perf hit or da…

> I'm not sure what type of compiler to label that as, but that's the goal.

A compiler. From TypeScript to web assembly.

Certainly it has its work cut out for it to have TS semantics (that is, JS semantics) and be optimal for target web assembly.

I can't think of any scripting languages that are optimal for targeting low-level runtimes.

I wish AssemblyScript the best; seems like a hard problem.

Re: An Urgent Notice from AssemblyScript

#46

Post by dcode »

conrad-watt wrote: post #39

Full disclosure, I am an active participant in WebAssembly standardisation, my github is here ( https://github.com/conrad-watt ). What follows is purely my personal opinion. This announcement is deliberately phrased to scare people who do not have sufficient context. I don't know why some AssemblyScript maintainers have decided to act in this extreme way over what is quite a niche issue. The vote that this announceme…

The security aspect is separate from whether UTF-16 lowering and lifting is supported. One simply cannot roundtrip every possible DOMString, C#, Java etc. String through the single concept of Unicode Scalar Values without either introducing lots of surface area for a) silent data corruption (what you call "recommended security practice", i.e. strings not comparing equal anymore after what appears to be an innocent function call) or b) for (deliberate) denial of service (when erroring instead). I mean, there is a good reason why all these languages do not do that in between function calls, but actually try very hard to guarantee integrity. And in a real program, an Interface Types function looks like any other function to the developer, just imported somewhere in the codebase, so good luck documenting that. Other than that I do not know how to respond to your subtle insults, please forgive my ignorance.

Re: An Urgent Notice from AssemblyScript

#47

Post by duped »

trusktr wrote: post #33
duped wrote: post #2

Can the authors expound on the reasons why they can't compile their language's string semantics into whatever representation will be used by WASI? Both C++ and Rust support numerous string representations, C++ even more so than Rust.

How would an end developer write code in one fashion (f.e. `let foo: string = "hello "`) while the compiler makes that work perfectly in every scenario? It would take a high amount of engineering effort compared to having one format that works well in the web to begin with. How does a compiler ensure that when that string is passed to a Rust Wasm module it goes to it in UTF-8 and then when moments later the same stri…

I don't really understand the questions. Have you looked at any prior art to see how C++ and Rust handle different string representations? C++ is probably the best influence due to type coercion since it sounds like you care about ergonomics over correctness.

For FFI there's nothing a compiler can do. That's why FFI is unsafe and restricted to rudimentary types in most languages - it's up to the caller to ensure the data is laid out as the callee expects.

I also don't know what interface types have to do with anything. Wasm is far lower level than interfaces, and nothing is stopping you from implementing interfaces in your language and doing automatic type conversion through them to handle string representations as required.

Look past the web for a moment - wasm is a competitor with the JVM, GraalVM, and LLVM as a platform and implementation independent byte code. Think about how your language would be implemented on those targets before the web.

Re: An Urgent Notice from AssemblyScript

#48

Post by trusktr »

conrad-watt wrote: post #39

Full disclosure, I am an active participant in WebAssembly standardisation, my github is here ( https://github.com/conrad-watt ). What follows is purely my personal opinion. This announcement is deliberately phrased to scare people who do not have sufficient context. I don't know why some AssemblyScript maintainers have decided to act in this extreme way over what is quite a niche issue. The vote that this announceme…

If Interface Types doesn't convert WTF strings to UTF, the flaw won't have to be documented, and the risk that someone forgets to do the right thing and causes a program to break will be eliminated. This seems like a better outcome. I haven't been able to think of downsides of not "sanitizing". Can you list those?

Re: An Urgent Notice from AssemblyScript

#49

Post by conrad-watt »

dcode wrote: post #46
conrad-watt wrote: post #39

Full disclosure, I am an active participant in WebAssembly standardisation, my github is here ( https://github.com/conrad-watt ). What follows is purely my personal opinion. This announcement is deliberately phrased to scare people who do not have sufficient context. I don't know why some AssemblyScript maintainers have decided to act in this extreme way over what is quite a niche issue. The vote that this announceme…

The security aspect is separate from whether UTF-16 lowering and lifting is supported. One simply cannot roundtrip every possible DOMString, C#, Java etc. String through the single concept of Unicode Scalar Values without either introducing lots of surface area for a) silent data corruption (what you call "recommended security practice", i.e. strings not comparing equal anymore after what appears to be an innocent fu…

For reference, this is the lead maintainer of AS.

You previously posted yourself that documenting sanitisation at the component boundary would be an acceptable solution: (https://web.archive.org/web/20210726140105if_/https://github...).

I don't understand why you have so radically changed your opinion since then.

Re: An Urgent Notice from AssemblyScript

#50

Post by dcode »

conrad-watt wrote: post #49
dcode wrote: post #46

Earlier quoted context omitted.

The security aspect is separate from whether UTF-16 lowering and lifting is supported. One simply cannot roundtrip every possible DOMString, C#, Java etc. String through the single concept of Unicode Scalar Values without either introducing lots of surface area for a) silent data corruption (what you call "recommended security practice", i.e. strings not comparing equal anymore after what appears to be an innocent fu…

For reference, this is the lead maintainer of AS. You previously posted yourself that documenting sanitisation at the component boundary would be an acceptable solution: ( https://web.archive.org/web/20210726140105if_/https://github... ). I don't understand why you have so radically changed your opinion since then.

At that time, it was ~ "you either stop advocating for the security concern, or we make sure you get nothing at all". In fact, I believe we would need to trap to at least make the breakage non-silent, but that would break AssemblyScript even more obviously and only exchange data corruption with denial of service. Not necessarily an improvement if you care about people using what you are building. In hindsight I am not proud of that comment and should have protested, no matter how dire the situation.
Post reply on HN