Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Towards a JavaScript Binary AST

yoric.github.io

Post reply on HN
Print view View original

181–190 of 211 posts

Re: Towards a JavaScript Binary AST

#181

Post by iainmerrick »

Yoric wrote: post #149
s3th wrote: post #140

i'm very skeptical about the benefits of a binary JavaScript AST. The claim is that a binary AST would save on JS parsing costs. however, JS parse time is not just tokenization. For many large apps, the bottleneck in parsing is instead in actually validating that the JS code is well-formed and does not contain early errors. The binary AST format proposes to skip this step [0] which is equivalent to wrapping function…

Early benchmarks seem to support the claim that we can save a lot on JS parsing costs. We are currently working on a more advanced prototype on which we will be able to accurately measure the performance impact, so we should have more hard data soon.

It seems like one big benefit of the binary format will be the ability to skip sections until they're needed, so the compilation can be done lazily.

But isn't it possible to get most of that benefit from the text format already? Is it really very expensive to scan through 10-20MB of text looking for block delimiters? You have to check for string escapes and the like, but it still doesn't seem very complicated.

Re: Towards a JavaScript Binary AST

#182

Post by c-smile »

To be honest I (as an author of the Sciter [1]) do not expect too much gain from that.

Sciter contains source code to bytecode compiler. Those bytecodes can be stored to files and loaded bypassing compilation phase. There is not too much gain as JS alike grammar is pretty simple.

In principle original ECMA-262 grammar was so simple that you can parse it without need of AST - direct parser with one symbol lookahead that produces bytecodes is quite adequate.

JavaScript use cases require fast compilation anyway. As for source files as for eval() and alike cases like onclick="..." in markup.

[1] https://sciter.com And JS parsers used to be damn fast indeed, until introduction of arrow functions. Their syntax is what requires AST.

Re: Towards a JavaScript Binary AST

#183

Post by lomnakkus »

Yoric wrote: post #106
sjrd wrote: post #103

Earlier quoted context omitted.

That seems awesome. I'm glad that it's on your radar. If you can point me to the best place to suggest ideas or spend my spare cycles, I would gladly do so. At the very least, I can comment on how we serialize positions in the (tree-based) Scala.js IR, which is size-optimized.

The tracker on which you're posting is a good place, thanks a lot :)

Off-topic: I'm sorry to go a little squishy, but I thought I should say that I appreciate the work both of you do very much... though sjrd's work is a little bit more "direct-impact-for-me" at the moment, I must admit. :p

Of course, as you just both said, your work is kind of complementary... which is always nice. :)

Anyway, thanks for the long-term thinking to the both of you.

Re: Towards a JavaScript Binary AST

#184

Post by tomdale »

tolmasky wrote: post #127
tomdale wrote: post #122

Earlier quoted context omitted.

I don't think the binary AST proposal changes the accessibility status quo. In my mind, the best analogy is to gzip, Brotli, etc. If you had to have a complicated toolchain to produce gzipped output to get the performance boost, that would create a performance gap between beginners and more experienced developers. But today, almost every CDN worth its salt will automatically gzip your content because it's a stateless…

I actually think gzip serves as a good example of this issue: this comment alone is daunting to a beginner programmer and it really shouldn't. This chrome/cdn thing could ALSO be auto-gzipping for you so that a beginner throwing files on a random server wouldn't need to know whether it supports gzip or not. I think we really take for granted the amount of stuff completely unrelated to programming we've now had to lea…

The process of getting content on to the web has historically been pretty daunting, and is IMO much easier now than the bad old days when a .com domain cost $99/year and hosting files involved figuring out how to use an FTP client.

In comparison, services like Now from Zeit, Netlify, Surge, heck, even RunKit, make this stuff so much easier in comparison now. As long as the performance optimizations are something that can happen automatically with tools like these, and are reasonable to use yourself even if you want to configure your own server, I think that's a net win.

I do agree with you though that we ought to fight tooth and nail to keep the web as approachable a platform for new developers as it was when we were new to it.

On balance, I'm more comfortable with services abstracting this stuff, since new developers are likely to use those services anyway. That's particularly true if the alternative is giving Google even more centralized power, and worse, access to more information that proxying all of those AST files would allow them to snoop on.

Re: Towards a JavaScript Binary AST

#185

Post by lomnakkus »

sjrd wrote: post #177
human-factor wrote: post #134

Earlier quoted context omitted.

A lot of the languages you mentioned have different enough memory allocation characteristics than JavaScript due to immutability and functional style that they would probably benefit from having a garbage collector tuned to their purposes in webassembly. There's a reason we don't have one common garbage collector for all the managed languages. I do recognize that this is a side point, but I think it's worth mentionin…

A lot of the mentioned languages also allow deep interoperability between their heap and the JavaScript heap, e.g., circular references between objects of the "two heaps", and free access to fields and methods of objects of the other language. That's very hard (if not impossible) to achieve without leak and performance degradation if the two languages have their own GC, with their own heaps. Compiling a language to J…

> Compiling a language to JS is not about making it work. That's easy (it becomes hard to cite a language that does not do it). It's about designing the language to interoperate with JS, and making that work. That is the real challenge.

It's very interesting that Scala and Scala.js have such a relatively painless interaction, but in general I'd say interoperation is "technically" simple by just employing an FFI?

Obviously, words like "seamless" and "effortless" start to enter the vocabulary here, but I'm not entirely these targets are worth it. Are they, do you think?

(I mean, obviously, Scala.js must have seamless 'interop' to Scala, but is 'seamless' introp with JS worth it, or should you require explicit FFI? I'm not sure, but I think you ultimately chose FFI-via-annotations, but there's a lot of fuzziness wrt. js.Dynamic.)

Re: Towards a JavaScript Binary AST

#186

Post by timdorr »

c-smile wrote: post #182

To be honest I (as an author of the Sciter [1]) do not expect too much gain from that. Sciter contains source code to bytecode compiler. Those bytecodes can be stored to files and loaded bypassing compilation phase. There is not too much gain as JS alike grammar is pretty simple. In principle original ECMA-262 grammar was so simple that you can parse it without need of AST - direct parser with one symbol lookahead th…

How do you do one-symbol-lookahead with things like function hoisting?

https://developer.mozilla.org/en-US/docs/Web/JavaScript/Refe...

Re: Towards a JavaScript Binary AST

#187

Post by limeblack »

c-smile wrote: post #182

To be honest I (as an author of the Sciter [1]) do not expect too much gain from that. Sciter contains source code to bytecode compiler. Those bytecodes can be stored to files and loaded bypassing compilation phase. There is not too much gain as JS alike grammar is pretty simple. In principle original ECMA-262 grammar was so simple that you can parse it without need of AST - direct parser with one symbol lookahead th…

I have never been a fan of the => syntax. Although that really isn't the point. What about the syntax slows parsing?

Re: Towards a JavaScript Binary AST

#189

Post by pharrington »

MadcapJake wrote: post #157
Yoric wrote: post #68

Earlier quoted context omitted.

> How can you say it's "compressed" over "compiled" when you are actually parsing it into an AST and then (iiuc) converting that to binary? That's exactly what compilers do. You are in fact going to a new source format (whatever syntax/semantics your binary AST is encoded with) so you really are compiling. I am not sure but there may be a misunderstanding on the word "binary". While the word "binary" is often used to…

> Here, "binary" simply means "not text", just as for instance images or zipped files are binary. If it's not text, then what is it? I'm not sure "not text" is a good definition of the word "binary". > A compiler typically goes from a high-level language to a lower-level language, losing data. I don't agree, I don't think there is any loss in data, the compiled-to representation should cover everything you wanted to…

Text means a sequence of characters conforming to some character encoding. Yoric's binary AST is not a sequence of characters conforming to a character encoding.

Compilation maps a program in a higher level language to a program in a lower level language. The map is not required to be one-to-one: the colloquial term for this is "lossy."

Re: Towards a JavaScript Binary AST

#190

Post by comex »

iainmerrick wrote: post #181
Yoric wrote: post #149

Earlier quoted context omitted.

Early benchmarks seem to support the claim that we can save a lot on JS parsing costs. We are currently working on a more advanced prototype on which we will be able to accurately measure the performance impact, so we should have more hard data soon.

It seems like one big benefit of the binary format will be the ability to skip sections until they're needed, so the compilation can be done lazily. But isn't it possible to get most of that benefit from the text format already? Is it really very expensive to scan through 10-20MB of text looking for block delimiters? You have to check for string escapes and the like, but it still doesn't seem very complicated.

Well, for one thing, a binary format’s inherent “obfuscatedness” actually works in its favor here. If Binary AST is adopted, I’d expect that in practice, essentially all files in that format will be generated by a tool specifically designed to work with Binary AST, that will never output an invalid file unless there’s a bug in the tool. From there, the file may still be vulnerable to random corruption at various points in the transit process, but a simple checksum in the header should catch almost all corruption. Thus, most developers should never have to worry about encountering lazy errors.

By contrast, JS source files are frequently manipulated by hand, or with generic text processing tools that don’t understand JS syntax. In most respects, the ability to do that is a benefit of text formats - but it means that syntax errors can show up in browsers in practice, so the unpredictability and mysteriousness of lazy errors might be a bigger issue.

I suppose there could just be a little declaration at the beginning of the source file that means “I was made by a compiler/minifier, I promise I don’t have any syntax errors”…

In any case, parsing binary will still be faster, even if you add laziness to text parsing.

Post reply on HN