Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Towards a JavaScript Binary AST

yoric.github.io

Post reply on HN
Print view View original

191–200 of 211 posts

Re: Towards a JavaScript Binary AST

#191

Post by iainmerrick »

comex wrote: post #190
iainmerrick wrote: post #181

Earlier quoted context omitted.

It seems like one big benefit of the binary format will be the ability to skip sections until they're needed, so the compilation can be done lazily. But isn't it possible to get most of that benefit from the text format already? Is it really very expensive to scan through 10-20MB of text looking for block delimiters? You have to check for string escapes and the like, but it still doesn't seem very complicated.

Well, for one thing, a binary format’s inherent “obfuscatedness” actually works in its favor here. If Binary AST is adopted, I’d expect that in practice, essentially all files in that format will be generated by a tool specifically designed to work with Binary AST, that will never output an invalid file unless there’s a bug in the tool. From there, the file may still be vulnerable to random corruption at various poin…

a simple checksum in the header should catch almost all corruption

For JavaScript, you have to assume the script may be malicious, so it always has to be fully checked anyway.

It's true that the binary format could be more compact and a bit faster to parse. I just feel that the size difference isn't going to be that big of a deal after gzipping, and the parse time shouldn't be such a big deal. (Although JS engine creators say parse time is a problem, so it must be harder than I realise!)

Re: Towards a JavaScript Binary AST

#193

Post by Akkuma »

pier25 wrote: post #153
Akkuma wrote: post #121

Earlier quoted context omitted.

Sure, why not? The difference is that WASM is aiming for near native performance of code, while the binary JS would still be limited to JS performance.

Yes, but it would have access to DOM, GC, etc.

But that's what languages that compile to JavaScript support already today. The binary JS shouldn't prevent this behaviour if it is just a binary representation of a text version.

Re: Towards a JavaScript Binary AST

#194

Post by comex »

iainmerrick wrote: post #191
comex wrote: post #190

Earlier quoted context omitted.

Well, for one thing, a binary format’s inherent “obfuscatedness” actually works in its favor here. If Binary AST is adopted, I’d expect that in practice, essentially all files in that format will be generated by a tool specifically designed to work with Binary AST, that will never output an invalid file unless there’s a bug in the tool. From there, the file may still be vulnerable to random corruption at various poin…

a simple checksum in the header should catch almost all corruption For JavaScript, you have to assume the script may be malicious, so it always has to be fully checked anyway. It's true that the binary format could be more compact and a bit faster to parse. I just feel that the size difference isn't going to be that big of a deal after gzipping, and the parse time shouldn't be such a big deal. (Although JS engine cre…

> For JavaScript, you have to assume the script may be malicious, so it always has to be fully checked anyway.

The point I was trying to make isn't that a binary format wouldn't have to be validated, but that the unpredictability of lazy validation wouldn't harm developer UX. It's not a problem if malicious people get bad UX :)

Anyway, I think you're underestimating the complexity of identifying block delimiters while tolerating comments, string literals, regex literals, etc. I'm not sure it's all that much easier than doing a full parse, especially given the need to differentiate between regex literals and division...

Re: Towards a JavaScript Binary AST

#195

Post by c-smile »

timdorr wrote: post #186
c-smile wrote: post #182

To be honest I (as an author of the Sciter [1]) do not expect too much gain from that. Sciter contains source code to bytecode compiler. Those bytecodes can be stored to files and loaded bypassing compilation phase. There is not too much gain as JS alike grammar is pretty simple. In principle original ECMA-262 grammar was so simple that you can parse it without need of AST - direct parser with one symbol lookahead th…

How do you do one-symbol-lookahead with things like function hoisting? https://developer.mozilla.org/en-US/docs/Web/JavaScript/Refe...

Not clear how hoisting is related to parsing really ...

Compiler builds table of variables (registers map) of the function to generate proper bytecodes to access registers. At this point hoisting has some effect. But it has nothing with parse/AST phase.

Re: Towards a JavaScript Binary AST

#196

Post by c-smile »

limeblack wrote: post #187
c-smile wrote: post #182

To be honest I (as an author of the Sciter [1]) do not expect too much gain from that. Sciter contains source code to bytecode compiler. Those bytecodes can be stored to files and loaded bypassing compilation phase. There is not too much gain as JS alike grammar is pretty simple. In principle original ECMA-262 grammar was so simple that you can parse it without need of AST - direct parser with one symbol lookahead th…

I have never been a fan of the => syntax. Although that really isn't the point. What about the syntax slows parsing?

As I said original JS syntax is pretty simple - parsing is comparable with just scanning source buffer char by char.

Not too much different from HTML and CSS. Where 99% of time takes actual DOM creation.

Like this XML/HTML scanner that I did while ago: https://www.codeproject.com/Articles/14076/Fast-and-Compact-... - it as fast as access to memory of HTML source. It even does not allocate any memory while parsing.

Re: Towards a JavaScript Binary AST

#197

Post by vvanders »

spc476 wrote: post #179
vvanders wrote: post #64

Lua has something very similar(bytecode vs AST) via luac for a long while now. We've used to to speed up parse times in the past and it helps a ton in that area.

There's a problem---you can break out of Lua with malformed bytecode [1] and the Lua team don't want to spend the time trying to validate Lua byte code [2]. That's why the latest Lua version has an option to ignore precompiled Lua scripts. And sadly, I can see the same happening here. [1] https://www.corsix.org/content/malicious-luajit-bytecode [2] The overhead will eat any performance gained by using precompiled Lua…

Sure, if you're loading scripts from an untrusted source then don't use bytecode. They're pretty clear about that in the docs. However probably about 90% of Lua's use cases are embedded and so in that case it works just fine.

Re: Towards a JavaScript Binary AST

#198

Post by Yoric »

mnemotronic wrote: post #192

Yea! A whole new attack surface. A hacked AST file could cause memory corruption and other faults in the browser-side binary expander.

Sure, a new file format always introduces a new risk. This has never prevented browsers from adding support for new image formats or compression schemes, though.

Re: Towards a JavaScript Binary AST

#200

Post by hashseed »

apaprocki wrote: post #51

From an alternate "not the web" viewpoint, I am interested in this because we have a desktop application that bootstraps a lot of JS for each view inside the application. There is a non-insignificant chunk of this time spent in parsing and the existing methods that engines expose (V8 in this case) for snapshotting / caching are not ideal. Given the initial reported gains, this could significantly ratchet down the par…

Do you have any details on what could be improved wrt V8's snapshotting? We have recently extended the feature set a lot in order to be able to snapshot full Blink contexts, and there are some efforts on-going to implement that in Node.js as well. Soon crutches like electron-link won't be necessary anymore.

Some context:

https://v8project.blogspot.com/2017/05/energizing-atom-with-...

https://github.com/nodejs/node/issues/13877

Post reply on HN