Hi Ben,
> It looks like a nice project, but I'm still a little confused about its purpose. You mention your JS background, so is it all about familiarity to npm?
I think familiarity with npm is important here!
I mentioned that npm allows you to have all dependencies of the project local.
Now if we want the same experience for a native project it gets more complex: we can't just install OCaml/Reason/C/C++ code into node_modules, it's not going to work.
So what esy does is it constructs for each package a "perfectly" isolated build environment based on dependencies declared in package.json, which is then used to produce build output (executable or library). Such environment contains things like `$PATH`, `$OCAMLPATH`, `$PKG_CONFIG_PATH` and etc, so these are all environment variable which C/OCaml/Reason/... software can understand.
Now it looks like this (super simplified!):
esy build: package.json -> isolated build environment -> build output
Now because such
isolated build environment depends only on the package.json (I simplify a bit here... later on that) and the build output depends only on its build environment we can easily compute a cache key for each package build output just by looking at package.json of the package.
So esy has smart cache for builds which makes each package build work like this:
let KEY = hash(package.json) # this is very cheap
if ~/.esy/store/KEY
done
else
build(package.json, ~/.esy/store/KEY) # this is not cheap!
It means in practice that if you have completely separate esy projects which
depend on the same version of `ocaml` (or `gcc` or any other package) then esy
will build that version of `ocaml` just once. So those "every local"
dependencies are super cheap once you have esy cache warm.
(the key isn't really hash(package.json) but more like hash(package.json +
esy.lock) as we need to resolve dependency constraints to concrete versions
first using "esy install" command)
> Or are there other things you gain when you run esy on top of dune?
The correct comparison is to dune on top of opam, I think (opam, the command line program, not opam, the registry as esy supports opam registry too!):
- with opam you manage switches manually (switches are like installed package sets), if you need a switch per project - you create a local switch per project
- with opam every time you create switch you need to compile everything from scratch, with esy we try to reuse as much as possible from the build cache
- you use dune with esy the same way you use dune with opam but you need to prefix dune invocations with esy: "esy dune ...", so that way dune sees esy build environment
- with esy you can pull packages from npm/opam/github/tarballs/...
- there's mechanism called `link:`[2] which allows to develop multiple packages as parts of the same project (so called monorepo workflow).
- ...
Some of these things are possible with opam too (not sure about smart cache though and I think it's pretty important) but with esy we've tried to provide the same experience as when you develop with npm/yarn (of course we wanted to solve gigantic node_modules issue and thus we have pnp, and flaky dependency constraints and thus we have a proper SAT solver, the same used by opam). So you might consider this a different UX to develop native code which is focused on Reason/OCaml.
> And my other question? It doesn't run with bucklescript - right? Are there specific reasons?
It doesn't integrate with bucklescript very well now but you still can use esy with bucklescript projects so you can install Merlin (IDE assistant) or other tools from Reason / OCaml ecosystem.
The way how bucklescript works right now isn't compatible with esy - for example we use pnp[1] installation mechanism (same as yarn) so we don't produce node_modules and thus we are fast and package sources are cached globally - for now bucklescript doesn't understand how resolve package sources from pnp.js runtime and thus it can't work with esy directly.
If bucklescript implements support with pnp then it will be perfectly usable with esy but for now you can use package.json for npm/yarn for bucklescript dependencies and esy.json (esy looks for esy.json before package.json) for
native dependencies.
[1]: https://github.com/yarnpkg/rfcs/pull/101
[2]: https://esy.sh/docs/en/linking-workflow.html