Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Node.js needs a virtual file system

blog.platformatic.dev

Post reply on HN
Print view View original

151–160 of 275 posts

Re: Node.js needs a virtual file system

#151

Post by zadikian »

afavour wrote: post #150
zadikian wrote: post #102

Earlier quoted context omitted.

DBMS vendor providing the client is nice. At least if you're using pg-native in Node, that's just a wrapper around the Postgres-owned libpq, but I've run into small breaking updates before that I don't feel would've happened if Postgres maintained both.

But that’s not Node’s fault surely? Shouldn’t Postgres be providing an NPM module given the popularity of Node?

No it's not Node's fault, this isn't their job. I don't blame Postgres either, cause maintaining libpq is fair enough, just would've been extra nice to have an official Node lib too.

Re: Node.js needs a virtual file system

#152

Post by Lerc »

giancarlostoro wrote: post #136

> I pointed the AI at the tedious parts, the stuff that makes a 14k-line PR possible but no human wants to hand-write: implementing every fs method variant (sync, callback, promises), wiring up test coverage, and generating docs. This is the biggest takeaway for me for AI. It's not even that nobody wants to do these things, its that by the time you finish your tasks, you have no time to do these things, because your…

I think the insight there is that the increased productivity of AI could be used to add features where the end results are weighing the ability of the AI against the ability of an individual implementing the same thing.

The alternative is that you work on the same number of features and utilize the ability to make those features as robust as you know they could be, but you have other pressing matters to attend to. That's weighing the ability of AI against the ability of neglect.

Re: Node.js needs a virtual file system

#153

Post by zadikian »

pier25 wrote: post #54
afavour wrote: post #40

Earlier quoted context omitted.

What would a Node-native database connection layer look like? What other platforms have that? Databases are third party tech, I don’t think it’s unreasonable to use a third party NPM module to connect to them.

Bun provides native MySQL, SQlite, and Postgres drivers. I'm not saying Node should support every db in existence but the ones I listed are critical infrastructure at this point. When using Postgres in Node you either rely on the old pg which pulls 13 dependencies[1] or postgres[2] which is much better and has zero deps but mostly depends on a single guy. [1] https://npmgraph.js.org/?q=pg [2] https://github.com/porsa…

Maybe MySQL and Postgres should make official Node libs then. Bun maintaining this is ok too, but it seems odd given that it means having to keep up with new features in those DBMSes.

Re: Node.js needs a virtual file system

#154

Post by jaredklewis »

syrusakbary wrote: post #119
indutny wrote: post #29

Taking the question of whether this would be a useful addition to Node.js core or aside, it must be noted that this 19k LoC PR was mostly generated by Claude Code and manually reviewed by the submitter which in my opinion is against the spirit of the project and directly violates the terms of Developer's Certificate of Origin set in the project's CONTRIBUTING.md

Fully disagree with this take. Not allowing AI assistance on PRs will likely decimate the project in the future, as it will not allow fast iteration speeds compared to other alternatives. Note aside, OpenJS executive director mentioned it's ok to use AI assistance on Node.js contributions: I checked with legal and the foundation is fine with the DCO on AI-assisted contributions. We’ll work on getting this documented.…

AI coding is great, but iteration speed is absolutely not a desirable trait for a runtime. Stability is everything.

Speed code all your SaaS apps, but slow iteration speeds are better for a runtime because once you add something, you can basically never remove it. You can't iterate. You get literally one shot, and if you add a awkward or trappy API, everyone is now stuck with it forever. And what if this "must have" feature turns out to be kind of a dud, because everyone converged on a much more elegant solution a few years later? Congratulations, we now have to maintain this legacy feature forever and everyone has to migrate their codebase to some new solution.

Much better to let dependencies and competing platforms like bun or deno do all the innovating. Once everyone has tried and refined all the different ways of solving this particular problem, and all the kinks have been worked out, and all the different ways to structure the API have been tried, you can take just the best of the best ideas and add it into the runtime. It was late, but because of that it will be stable and not a train wreck.

But I know what you're thinking. "You can't do that. Just look at what happens to platforms that iterate slowly, like C or C++ or Java. They're toast." Oh wait, never mind, they're among the most popular platforms out there.

Re: Node.js needs a virtual file system

#155

Post by benatkin »

sauercrowd wrote: post #145
socalgal2 wrote: post #144

What's special about node.js here? Does golang, C#, python, ruby, java, etc... have a virtual file system? I get it, I've implemented things for tests, I'm just wondering if this shouldn't be solved at an OS level. --- update Let's put this another way, my code does effectively, child_process.spawn('something-that-reads-and-write-a-file') now I'm back to the same issue. To test I need a virtual file system. Node prov…

Go actually does https://pkg.go.dev/embed I do think it's more painful to distribute files when you're a distributed as a single binary vs scripts, since the latter has to figure out bundling of files anyway. But still - it does exist

Embed is read-only at runtime. This proposed vfs module for Node.js is a full virtual file system.

Re: Node.js needs a virtual file system

#156

Post by syrusakbary »

jaredklewis wrote: post #154
syrusakbary wrote: post #119

Earlier quoted context omitted.

Fully disagree with this take. Not allowing AI assistance on PRs will likely decimate the project in the future, as it will not allow fast iteration speeds compared to other alternatives. Note aside, OpenJS executive director mentioned it's ok to use AI assistance on Node.js contributions: I checked with legal and the foundation is fine with the DCO on AI-assisted contributions. We’ll work on getting this documented.…

AI coding is great, but iteration speed is absolutely not a desirable trait for a runtime. Stability is everything. Speed code all your SaaS apps, but slow iteration speeds are better for a runtime because once you add something, you can basically never remove it. You can't iterate. You get literally one shot, and if you add a awkward or trappy API, everyone is now stuck with it forever. And what if this "must have"…

Since when we accepted that we can’t go fast and offer stability at the same time?

Time is highly correlated with expertise. When you don’t have expertise, you may go fast at expense of stability because you lack the experience to make good decisions to really save speed. This doesn’t hold true for any projects where you rely on experts, good processes and tight timelines (aka: Apollo mission)

Re: Node.js needs a virtual file system

#157

Post by pocksuppet »

austin-cheney wrote: post #101
dotancohen wrote: post #94

Earlier quoted context omitted.

> I think JavaScript would benefit from an in-memory database. That database would probably look a lot like a JSON object. What are you suggesting, that a global JSON object does not solve?

Whether it is an object, array, something else, or a combination thereof is a design decision. It is not so much about the design of the structure, which should be determined by execution performance considerations, but how information is added, removed and retrieved. Gathering one or more records from a JSON object, or array index, by value of some child property somewhere in a descendant structure of the instance i…

Okay, what would your solution look like?

    from cities select state where name='Chicago'
isn't really different from

    cities.filter(x=>x.name=='Chicago').map(x=>x.state)

Re: Node.js needs a virtual file system

#158

Post by conartist6 »

indutny wrote: post #29

Taking the question of whether this would be a useful addition to Node.js core or aside, it must be noted that this 19k LoC PR was mostly generated by Claude Code and manually reviewed by the submitter which in my opinion is against the spirit of the project and directly violates the terms of Developer's Certificate of Origin set in the project's CONTRIBUTING.md

Pain is a signal. Even if the trick is not minding, it's still inadvisable to burn your hand on an open flame. The pain is there to help you not get hurt.

I do not think it is wise to brag that your solution to a problem is extremely painful but that you were impervious to all the pain. Others will still feel it. This code takes bandwidth to host and space on devices and for maintainers it permanently doubles the work associated with evolving the filesystem APIs. If someone else comes along with the same kind of thinking they might just double those doubled costs, and someone else might 8x them, all because nobody could feel the pain they were passing on to others

Re: Node.js needs a virtual file system

#159

Post by Culonavirus »

giancarlostoro wrote: post #136

> I pointed the AI at the tedious parts, the stuff that makes a 14k-line PR possible but no human wants to hand-write: implementing every fs method variant (sync, callback, promises), wiring up test coverage, and generating docs. This is the biggest takeaway for me for AI. It's not even that nobody wants to do these things, its that by the time you finish your tasks, you have no time to do these things, because your…

That's perfectly understandable. But has no business being in a large open source project, let alone world class one like Node or (god forbid) the Linux kernel. Get that shit the fuck out.

Re: Node.js needs a virtual file system

#160

Post by pier25 »

zadikian wrote: post #153
pier25 wrote: post #54

Earlier quoted context omitted.

Bun provides native MySQL, SQlite, and Postgres drivers. I'm not saying Node should support every db in existence but the ones I listed are critical infrastructure at this point. When using Postgres in Node you either rely on the old pg which pulls 13 dependencies[1] or postgres[2] which is much better and has zero deps but mostly depends on a single guy. [1] https://npmgraph.js.org/?q=pg [2] https://github.com/porsa…

Maybe MySQL and Postgres should make official Node libs then. Bun maintaining this is ok too, but it seems odd given that it means having to keep up with new features in those DBMSes.

> but it seems odd given that it means having to keep up with new features in those DBMSes

That would be more useful for the ecosystem than the Node team investing time on a virtual file system.

Post reply on HN