The Hitchhiker's Guide to Modern JavaScript Tooling

We talk about "modern JavaScript tooling" but year after year, the list essentially stays the same.

Maybe few new players have appeared (Gulp, WebPack, Babeljs) but they do exactly the same thing that the tools we had before (e.g. Grunt, Browserify, Traceur).

It occurs to me that "modern JavaScript tooling" is growing only vertically (better tools to build, better tools to modularize, better tools to transpilation), but not horizontally. If we see some "brave new tool" on the scene, this tool will do exactly what previous tools did, only better.

I would like to have some decent tools for:

* analysing of project structure (e.g. dependencies between modules, graphs, trees etc.)

* code visualising (not toy! Gource is beautiful, but pretty useless. JSCity... I also don't see much use of it. I would see something that would allow me to draw some useful information from code visualisation. Something that would allow to understand better. But I see only beautiful animations and abstract 3D scenes)

* maintaining code (something that would allow me to conduct massive scale refactoring, or automatically convert code from one framework to another etc.)

* better editors for HTML/CSS, maybe even some decent WYSIWYG

Okay. Plain build systems and transpilers also are super useful. I think that Gulp, Babel.js, Browserify etc. are greeeat. But I think we need more. Something different. There is still room for innovation. Projects grow bigger and I think that we need something that helps us

* to understand easily new codebase

* to navigate codebase, conduct semantic search etc.

* to maintaining, refactoring etc.

I feel that some important tools are missing, not created yet.

I think it's good to note that you don't actually need a single one of these tools to make functioning software. You can pack your scripts with cat and download libraries with wget and use regular old JavaScript without transpilers. When you have a problem with this, you can use tools... But you don't have to top-load every project with a whole suite of complex tools just for the sake of it.

I'd rather use JS than add shell mixed in with Makefile. Two extra languages just to run a build system seems like adding unnecessary complexity.

Esprima [1] is a full-featured and quite extensible ES5 parser, which has been used as the basis for a lot of the sort of analysis tools you're thinking about here. You might want to take a look at it, and at the software that's been written around it. In particular, there are several static analysis tools you might find of interest; I don't have a handy list, but searching "esprima static analysis" should find you plenty of candidates for further investigation. Refactoring tools shouldn't be hard to write, too, given an AST. (But I'd tend to suspect that automatic conversion from one framework to another is probably a pipe dream, at least for any nontrivial code base. Frameworks tend to come with too many mutually incompatible assumptions for that to be possible without an outright rewrite. Hell, that's even true of Angular 1.x and Angular 2!)

I'd be curious to know what you mean by "better", in your fourth bullet. I can think of a few things, but I doubt they're the same things you've thought of.

(And, while I have extensive experience with WYSIWYG HTML/CSS editors, that experience lies far in my professional past, because I found they were always too inaccurate to be worth the effort. Besides, it's not like there's anything particularly difficult about just viewing your changes in an actual browser; with tools like LiveReload, you needn't even go to the modicum of effort required to hit F5.)

What kind of codebase visualization would you consider "useful"? I think that's really the hard question to answer there; implementing a visualization is probably pretty easy, compared to coming up with a visualization from which you can easily derive information that's hard or impossible to obtain any other way. (I surmise this to be a difficult question based on the fact that no such visualization exists, or at least if it does it's not well known. Otherwise, it'd probably be part of the standard toolkit by now.)

[1] http://esprima.org