Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Node Modules at War: Why CommonJS and ES Modules Can’t Get Along

redfin.engineering

Post reply on HN
Print view View original

131–140 of 152 posts

Re: Node Modules at War: Why CommonJS and ES Modules Can’t Get Along

#131

Post by WorldMaker »

jkrems wrote: post #122
Macha wrote: post #81

Earlier quoted context omitted.

It's not possible to write a nice automated transparent one for arbitrary libraries because of top level await, but it's totally possible for the 95%.

You cannot write a CJS shim for any library because the CJS shim cannot expose any ESM file with a synchronous API because importing ESM always returns a Promise. You can - and people widely do - compile a CJS version of your library from ESM sources. You'll accept that there may be multiple copies of your library in the program but it does work. But in any case there's no reason to actually author in CJS, everything…

It is possible, but you have to hack the Node require() loader to do it, teach it a lot about ESM loading, and then hope the downstream CJS don't have timing bugs around "very long" synchronous require() loads.

https://github.com/standard-things/esm

Re: Node Modules at War: Why CommonJS and ES Modules Can’t Get Along

#132

Post by WorldMaker »

z3t4 wrote: post #129
samth wrote: post #75

Earlier quoted context omitted.

No, this is not correct. The split long predates async in JS The fundamental reason behind the difference between ES modules and CJS is that require() does synchronous I/O in a function, but that's not acceptable in the browser context. So you either need to make require an asynchronous function (meaning with a callback, back in 2010 when this happened) or move the I/O out of the execution of module itself, as import…

Sync xhr is being depricated. Meanwhile Google and Facebook analytics/tracking freeze the main thread for a whooping two seconds. Why not push the module dependencies with http2 push? The problem is not technical. Its big money wanting to break backwards compatibility.

Even HTTP/2 Push is an order of magnitude slower than local File I/O on a good day with perfect (internet) weather. ESM isn't even the first approach to try to deal with this, for instance AMD had nothing (at first) to do with "big money" and everything to do with average web developers trying their best to practically solve the module problem for the web and CommonJS was never going to be an option in a web browser (even if AMD or ESM spec authors had magic foresight to HTTP/2 or HTTP/3 they'd still have the same concerns with CommonJS).

CommonJS was a mistake that the JS world is going to live with for years more.

Re: Node Modules at War: Why CommonJS and ES Modules Can’t Get Along

#133

Post by WorldMaker »

pansa2 wrote: post #73
gridlockd wrote: post #71

Earlier quoted context omitted.

The reason why async/await is a good fit for Javascript is that it neatly wraps up asynchronous callback hell and Promises, which were already ubiquitous in Javascript code, because the web runtime enforces non-blocking IO.

Do Promises and async-await work together better than, say, Promises and Lua-like stackful coroutines?

This description is probably far more abstract than most would like (and it is probably wrong by not being exact enough, I am not a mathematician I am a software developer), but might be useful to some: coroutines are a join calculus and Promises are a monad. async/await is syntactic sugar for a monad (less general than Haskell's do-notation, more general than only Promises, though rarely in practice used much beyond Promises). You could plumb Promises on top of coroutines easily (and while JS engines are more traditionally event loops; other languages that support async/await such as Python use much more coroutine-like underbellies), and while you don't get the full "join calculus" power/flexibility by putting the monad abstraction on top of the join calculus, you get the benefit of monad transformers like async/await.

Those benefits being that the async/await syntax and semantics for working with Promises more closely resembles the imperative code it replaces, whereas the join calculus is its own [sometimes easy, admittedly] learning curve with its own syntax and semantics. Most of the complaints about Promises are the usual complaints about monadic wrapping types that the types become "viral" and "color" all associated code. But that just means that the failure cases are more easily picked up by a static type checker than semantic failures in learning the join calculus.

Re: Node Modules at War: Why CommonJS and ES Modules Can’t Get Along

#134

Post by WorldMaker »

Humphrey wrote: post #41

Wow, I've been coding JS/Typescript frontends for years and somehow completely missed out knowing anything about CJS & ESM or their differences. I guess it's all irrelevant when you don't use Node, aren't publishing libraries, and are compiling all JavaScript into bundles for deployment and never read the bundled code.

Typescript and your bundler both have to know a lot about it. Typescript went all in on ESM syntax since 1.x so you at least know ESM syntax by now if you are in Typescript and can pretend that CJS is a bad dream, but Typescript will happily spit out CJS for you with a compiler flag if you need it for node.

So much of the complexity of webpack, parcel, rollup, pika, snowpack, et al is dealing with the weird mix of CJS and ESM spread across npm at this point.

One of the main differences between the bundlers at this point is their opinion of CJS and ESM handling. You may not notice it as a bundler user, but you probably see a lot of indirect results in things like bundle size and ease of use. So it is not entirely irrelevant to frontend work, but you just may not realize that why you prefer one bundler over another today is heavily influenced by their approach to the CJS / ESM split. (A good bundler you shouldn't have to realize that behavior is what contributes so much to the bundler's "feel" and your gut-reaction/instinct to working with it.) Learning the differences can be useful if you need to better pick your next bundler based on technical underpinnings, though admittedly that sort of "comparison shopping" is probably a rare need.

Re: Node Modules at War: Why CommonJS and ES Modules Can’t Get Along

#135

Post by WorldMaker »

djohnston wrote: post #91

it seems so insane that there are competing module patterns in a single language, but perhaps i'm ignorant. are there other languages with such incompatibilities on a fundamental component?

Lisp is the easiest classic example that took ages to settle on a module standard and has had different competing standards over the years.

PHP and Perl are probably more recent examples comparable to JS here that provided mostly only low level plumbing (include/require) support and nearly wound up with multiple incompatible module loading patterns.

Even C/C++ the "module" system started as a text processing hack in a macro pre-processor that used to be a separate tool (though I don't think any modern C/C++ toolchain uses a separate macro pre-processor, as there are tons of optimization layers now and some of the macro pre-processor knowledge makes its way all the way through to the linker these days).

Historically, modules were always just "smash these two files together in this order" and it was only later that languages started asking hard questions such as "which symbols from this other file are available".

The modern concept of modules as a fixed "shape" of self-describing public symbols is as much a later abstraction out of Object-Oriented Programming as anything else. Our idea of what a module should be in 2020 is much more informed by "recent" notions such as Component systems like COM and languages such as Java and C#.

Re: Node Modules at War: Why CommonJS and ES Modules Can’t Get Along

#136

Post by WorldMaker »

desmap wrote: post #43

fwiw, TypeScript and ES Modules are just not compatible and never will be: https://github.com/microsoft/TypeScript/issues/18442#issueco...

That chart shows that Typescript is incompatible with CJS modules, because it inherits ES Modules incompatibilities with CJS, not that it it is incompatible with ES Modules.

Re: Node Modules at War: Why CommonJS and ES Modules Can’t Get Along

#137

Post by Androider »

wildpeaks wrote: post #92

Honestly, all I need is a standardized way to publish and consume packages directly as Typescript (right now you can point "main" & "types to it, but I'd rather something like export maps) because the transpilation should be done by the end user as you can't assume which build target they might prefer.

Typescript transpilation is already pretty slow when just used for your own application, if all the node_modules directory entries were in TS I'd probably be looking at hours for the transpilation.

Re: Node Modules at War: Why CommonJS and ES Modules Can’t Get Along

#138

Post by djohnston »

WorldMaker wrote: post #135
djohnston wrote: post #91

it seems so insane that there are competing module patterns in a single language, but perhaps i'm ignorant. are there other languages with such incompatibilities on a fundamental component?

Lisp is the easiest classic example that took ages to settle on a module standard and has had different competing standards over the years. PHP and Perl are probably more recent examples comparable to JS here that provided mostly only low level plumbing (include/require) support and nearly wound up with multiple incompatible module loading patterns. Even C/C++ the "module" system started as a text processing hack in…

very cool thanks for the context!

Re: Node Modules at War: Why CommonJS and ES Modules Can’t Get Along

#139

Post by jkrems »

WorldMaker wrote: post #131
jkrems wrote: post #122

Earlier quoted context omitted.

You cannot write a CJS shim for any library because the CJS shim cannot expose any ESM file with a synchronous API because importing ESM always returns a Promise. You can - and people widely do - compile a CJS version of your library from ESM sources. You'll accept that there may be multiple copies of your library in the program but it does work. But in any case there's no reason to actually author in CJS, everything…

It is possible, but you have to hack the Node require() loader to do it, teach it a lot about ESM loading, and then hope the downstream CJS don't have timing bugs around "very long" synchronous require() loads. https://github.com/standard-things/esm

I'm not sure that's really relevant here because esm is just one of the transpilers for module syntax, it's not running native modules at all. The same can be achieved with the babel CLI or TSC etc. and the right configuration. But it's not really "ESM loading". It's compiling ESM to CJS and then running CJS. It's true that transpiling to CJS will continue to work, no matter what.

Re: Node Modules at War: Why CommonJS and ES Modules Can’t Get Along

#140

Post by fireattack »

desmap wrote: post #61
fireattack wrote: post #32

Another issue about module import I encountered before and wasted me 1 hour to find the culprit: When converting TypeScript that using `import default xxx` to JS using `tsc`, it would convert it to something like `exports.default = xxx`. To import it, you have to use `const XXX = require('./mo.js').default` instead of just `const XXX = require('./mo.js')` in plain ES6 JS. This is fine but Babel used to [1] be able to…

But how do you solve it in a tsc context? Using the default prop doesn't work in tsc and breaks all type information because tsc just doesn't know of any default prop.

Not exactly sure what you mean but you can check this: https://github.com/microsoft/TypeScript/issues/2719
Post reply on HN