Live data from Hacker News

Node Modules at War: Why CommonJS and ES Modules Can’t Get Along

redfin.engineering

61–70 of 152 posts

Re: Node Modules at War: Why CommonJS and ES Modules Can’t Get Along

#61

Another issue about module import I encountered before and wasted me 1 hour to find the culprit: When converting TypeScript that using `import default xxx` to JS using `tsc`, it would convert it to something like `exports.default = xxx`. To import it, you have to use `const XXX = require('./mo.js').default` instead of just `const XXX = require('./mo.js')` in plain ES6 JS. This is fine but Babel used to [1] be able to…

But how do you solve it in a tsc context?

Using the default prop doesn't work in tsc and breaks all type information because tsc just doesn't know of any default prop.

Re: Node Modules at War: Why CommonJS and ES Modules Can’t Get Along

#62
post #10

JavaScript just gives to much options. I keep having problems with libraries because ES6 allows import x from "library"; import {x} from "library"; import * as x from "library"; import {x as y} from "library"; Just to compare Java uses this: import java.lang.Double; import java.lang.*; No renaming, not a gazillion options during in- and export, but one statement. If you want to rename something, assign it to a variab…

Java's packaging system sucks as does all the similar ones like C#, C++ etc compared to JavaScript's IMO. All of those the space of namespaces is global. You can't have 2 modules named math.quaterinions in any of those languages. You can in JavaScript. To put it another way, in order to prevent name clashes in C++/C#/Java you have to know the name of every other project on the planet. If someone uses the same namespa…

> Java's packaging system sucks as does all the similar ones like C#, C++ etc compared to JavaScript's IMO.

There is one gazillion things that are much more a problem than "namespace conflicts" in packaging and package management.

Including security & code authenticity, parallel multi-version handling, sensibility to side effects, speed, auditing and reproducibility.

And in these domain the JavaScript packaging system generally range from being 'catastrophic' to 'absolutely terrible'.

If I have to take an example package management system for the world I would certainly quote cargo, Nix, Guix or Spack but certainly not npm which is quite close to what you should really not do for a package manager.

Re: Node Modules at War: Why CommonJS and ES Modules Can’t Get Along

#63
post #48

Earlier quoted context omitted.

2to3 was still useful because all code was in python 2 when python 3 first came out.

Sure, but maybe upgrading the existing Python 2 code wasn't the problem: the vast majority of the Python 2 code alive when Python 3 was released was probably scrapped by the time Python 2 was EOL'd . Over a decade is a very long time for a codebase to live. (Again, there are exceptions, but the vast majority of websites or data processing scripts or whatnot are significantly re-written or decommissioned in – pulling…

> Porting Python 2 code to Python 3 was never the problem.

You're, frankly, high as a kite. As the author of one such port on a large codebase, porting Python 2 code to Python 3 was absolutely a problem.

The issue with 2to3 is that it assumed you'd do the conversion as a one-shot and straight flip over with entirely separate codebases (or dropping Python 2 entirely) from there on, which is completely insane.

What the vast majority of transitioning projects needed was a way to migrate progressively through cross-version codebases:

* libraries weren't going to drop older Python versions, the very few which attempted that (I'm aware of dateutil at least) were extremely painful to users and IIRC ended up rolling back to a more progressive codebase

* programs being distributed were not going to drop Python 2 until EOF at least

* large codebases simply could not afford to perform a completely untested and uncheckable one-shot transition

That's why the community created cross-version support tools like six and friends, and lobbied hard for the reintroduction of cross-version features into Python 3 e.g. reintroduction of `callable` (3.2), reintroduction of the "u" prefix (3.3), reintroduction of % on bytes (3.5) and I'm sure others.

> Again, there are exceptions, but the vast majority of websites or data processing scripts or whatnot are significantly re-written or decommissioned in – pulling a number out of my arse – five years or so. […] Python 3 because it would be incompatible with the Python 2 ecosystem

What do you think "the python 2 ecosystem is" except for a ton of projects which needed to be ported from Python 2 to Python 3 exactly? "the vast majority of websites or data processing scripts" is not an ecosystem, tooling and libraries are.

Do you somehow think numpy and lxml and django just scrapped their entire Python 2 codebase and rewrote the entire thing from scratch? Of course not.

> That problem could have been prevented with a 3to2 tool, which would compile the Python 3 code to Python 2, allowing it to be used with the existing ecosystem.

3to2 would not have worked any better better than 2to3. The syntactic incompatibilities between the version were not the hard part, the semantic ones were, and 3to2 would not have succeeded any better than 2to3 there because it simply couldn't.

All 3to2 would have given you is a broken Python 2 codebase from a Python 3 codebase you could not even run.

Re: Node Modules at War: Why CommonJS and ES Modules Can’t Get Along

#64
Not taking seamless CJS interop into account was IMHO the biggest failure of TC39 till date.

Given the widespread popularity of CJS they should have prioritized CJS backward compatibility over things like better support for circular dependencies and mutable exports. Many languages are doing quite fine without them.

Re: Node Modules at War: Why CommonJS and ES Modules Can’t Get Along

#65
post #49
post #37

I’ll conclude with three guidelines for library authors to follow: Provide a CJS version of your library Provide a thin ESM wrapper for your CJS Add an exports map to your package.json This seems like the same fallacy of python’s 2to3, and why they should have written 3to2 instead. If ESM is the new way, library authors should instead have a CJS shim to prevent fossilization of cruft.

> This seems like the same fallacy of python’s 2to3, and why they should have written 3to2 instead. This is incredibly insightful. I'm frankly a bit stunned. I would never have thought of this (and clearly the Python people also didn't) but it is absolutely true. It's also a very transferable strategy for any time compatibility is to be broken. Is this an original thought of yours or do you have references to further…

This is something like what the future package and tools do, though I can say from experience it's not without its issues.

Re: Node Modules at War: Why CommonJS and ES Modules Can’t Get Along

#66
post #48

Earlier quoted context omitted.

2to3 was still useful because all code was in python 2 when python 3 first came out.

Sure, but maybe upgrading the existing Python 2 code wasn't the problem: the vast majority of the Python 2 code alive when Python 3 was released was probably scrapped by the time Python 2 was EOL'd . Over a decade is a very long time for a codebase to live. (Again, there are exceptions, but the vast majority of websites or data processing scripts or whatnot are significantly re-written or decommissioned in – pulling…

good theory, doesn't align with my (limited) experience. writing new code to be both 2 and 3 compatible was quite possible and not too tricky. six helped, `from __future__` imports, etc. however, everywhere i've worked still had some python 2 only code running - even to this day unfortunately.

the real issue is 2to3 didn't work so well, or exposed the exact issues why python 3's breaking changes were needed (80% sloppy bytes/str/unicode handling).

Re: Node Modules at War: Why CommonJS and ES Modules Can’t Get Along

#68
post #34

The description of how require() works is not quite correct. Node does not always fully initialize the dependency module before returning from require(). It can't work that way, because Node allows circular require() dependencies. Try it!

The reason is not correct. Nodejs cache the require lot and so circular dependencies can work without returning half finished loads.

Nodejs does cache the results of require(), but that doesn't help with this problem. Example: https://repl.it/repls/SwiftOlivedrabBrace

Re: Node Modules at War: Why CommonJS and ES Modules Can’t Get Along

#69
post #59
post #22

Earlier quoted context omitted.

It means you can get a half-initialized module back from a require() call.

Does that mean the required modules are not run?

They are run, but if there are circular dependencies they are run _after_ require() returns.
Post reply on HN