[flagged]
Node.js needs a virtual file system
51–60 of 275 posts
Re: Node.js needs a virtual file system
#52Yarn, pnpm, webpack all have solutions for this. Great to see this becoming a standard. I have a project that is severely handicapped due to FS. Running 13k tests takes 40 minutes where a virtual file system that Node would just work with it would cut the run time to 3 minutes. I experimented with some hacks and decided to stay with slow but native FS solution. What I really want is a way of swapping FS with VFS in a…
The way to do this today is to do it outside of node. Using an overlay fs with the overlay being a ramfs. You can even chroot into it if you can't scope the paths you need to be just downstream from some directory. Or, just use docker.
Re: Node.js needs a virtual file system
#53Taking the question of whether this would be a useful addition to Node.js core or aside, it must be noted that this 19k LoC PR was mostly generated by Claude Code and manually reviewed by the submitter which in my opinion is against the spirit of the project and directly violates the terms of Developer's Certificate of Origin set in the project's CONTRIBUTING.md
Large PRs could follow the practices that the Linux kernel dev lists follow. Sometimes large subsystem changes could be carried separately for a while by the submitter for testing and maintenance before being accepted in theory, reviewed, and if ready, then merged.
While the large code changes were maintained, they were often split up into a set of semantically meaningful commits for purposes of review and maintenance.
With AI blowing up the line counts on PRs, it's a skill set that more developers need to mature. It's good for their own review to take the mass changes, ask themselves how would they want to systematically review it in parts, then split the PR up into meaningful commits: e.g. interfaces, docs, subsets of changed implementations, etc.
Re: Node.js needs a virtual file system
#54The Node team has lost the plot IMO. By far the most critical issue is the over reliance on third party NPM packages for even fundamental needs like connecting to a database.
What would a Node-native database connection layer look like? What other platforms have that? Databases are third party tech, I don’t think it’s unreasonable to use a third party NPM module to connect to them.
Bun provides native MySQL, SQlite, and Postgres drivers.
I'm not saying Node should support every db in existence but the ones I listed are critical infrastructure at this point.
When using Postgres in Node you either rely on the old pg which pulls 13 dependencies[1] or postgres[2] which is much better and has zero deps but mostly depends on a single guy.
Re: Node.js needs a virtual file system
#55Re: Node.js needs a virtual file system
#56[dead]
Re: Node.js needs a virtual file system
#57Earlier quoted context omitted.
How exactly does it violate the Developer's Certificate of Origin clause?
The submitted code must adhere to either of (a), (b), (c), and separately a (d) clause of: https://github.com/nodejs/node/blob/main/CONTRIBUTING.md#dev... If submitter picks (a) they assert that they wrote the code themselves and have right to submit it under project's license. If (b) the code was taken from another place with clear license terms compatible with the project's license. If (c) contribution was written…
It would be considered (a) since the author would own the copyright on the code.
Re: Node.js needs a virtual file system
#58Earlier quoted context omitted.
Deno has some pretty nice unique features like sandboxing that, afaik, don't exist in other runtimes (yet). It's enough of a draw that it's the recommended runtime for projects like yt-dlp: https://github.com/yt-dlp/yt-dlp/issues/14404
Node has sandboxing these days: https://nodejs.org/api/permissions.html
No it doesn't, unfortunately.
> The permission model implements a "seat belt" approach, which prevents trusted code from unintentionally changing files or using resources that access has not explicitly been granted to. It does not provide security guarantees in the presence of malicious code. Malicious code can bypass the permission model and execute arbitrary code without the restrictions imposed by the permission model.
Deno's permissions model is actually a very nice feature. But it is not very granular so I think you end up just allowing everything a lot of the time. I also think sandboxing is a responsibility of the OS. And lastly, a lot of use cases do not really benefit from it (e.g. server applications).
Re: Node.js needs a virtual file system
#59Yarn, pnpm, webpack all have solutions for this. Great to see this becoming a standard. I have a project that is severely handicapped due to FS. Running 13k tests takes 40 minutes where a virtual file system that Node would just work with it would cut the run time to 3 minutes. I experimented with some hacks and decided to stay with slow but native FS solution. What I really want is a way of swapping FS with VFS in a…
The way to do this today is to do it outside of node. Using an overlay fs with the overlay being a ramfs. You can even chroot into it if you can't scope the paths you need to be just downstream from some directory. Or, just use docker.
making that work cross platform is pure pain
Re: Node.js needs a virtual file system
#60Earlier quoted context omitted.
What would a Node-native database connection layer look like? What other platforms have that? Databases are third party tech, I don’t think it’s unreasonable to use a third party NPM module to connect to them.
Most obviously, Java has JDBC. I think .NET has an equivalent. Drivers are needed but they're often first party, coming directly from the DB vendor itself. Java also has a JIT compiling JS engine that can be sandboxed and given a VFS: https://www.graalvm.org/latest/security-guide/sandboxing/ N.B. there's a NodeJS compatible mode, but you can't use VFS+sandboxing and NodeJS compatibility together because the NodeJS mo…
> Most obviously, Java has JDBC. I think .NET has an equivalent. Drivers are needed but they're often first party, coming directly from the DB vendor itself.
So it's an external dependency that is not part of Java. It doesn't really matter if the code comes from the vendor or not. Especially for OpenSource databases.