Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We check our node_modules folder into source control

jackfranklin.co.uk

Post reply on HN
Print view View original

201–210 of 241 posts

Re: We check our node_modules folder into source control

#201

Post by Chyzwar »

tentacleuno wrote: post #156
Chyzwar wrote: post #142

This is idiotic and could only be written by someone that have no idea about node package managements. 1. Most of CI systems have better options for caching node modules[1][2]. When you check node_modules, you add fixed cost(increasing) for every commit. When you use CI caching you add fixed cost(static) to only small number of builds. 2. Once you start upgrading packages repo size will continue to grow. At some poin…

> This is idiotic and could only be written by someone that have no idea about node package managements. Some of your points are valid, but the first sentence seems really hand-wavy and rude. > 1. Most of CI systems have better options for caching node modules[1][2]. When you check node_modules, you add fixed cost(increasing) for every commit. When you use CI caching you add fixed cost(static) to only small number of…

> Some of your points are valid, but the first sentence seems really hand-wavy and rude.

Because people here are discussing this and entertaining this idea. This creates a level of legitimacy. After this article, there might be now countless teams transitioning to this crazy idea. Then two years later, people would continue to complain about node ecosystem because they were burned badly by projects maintained using this approach.

The problem with today word is that everyone tries to be politically correct. Everyone wants to discuss things in a civilized way based on merit and logic. In many cases, we could avoid wasting time and energy by declaring things as they are. For example, If mainstream would call anti-vaxxer stupid, we would have now more people vaccinated

There are more things that people spend large amount of time discussing when there are nothing to discuss. This thread should not have 195 comments.

Re: We check our node_modules folder into source control

#202

Post by Chyzwar »

bennyp101 wrote: post #178
Chyzwar wrote: post #142

This is idiotic and could only be written by someone that have no idea about node package managements. 1. Most of CI systems have better options for caching node modules[1][2]. When you check node_modules, you add fixed cost(increasing) for every commit. When you use CI caching you add fixed cost(static) to only small number of builds. 2. Once you start upgrading packages repo size will continue to grow. At some poin…

"Because npm.com is such vital infrastructure, it unlikely that it would ever stop working." Except that is has? It's a service, like everything else on the internet, it can and will go down. The choice here is whether you can carry on, or wait until it comes back up. THAT is the main benefit I see in this. (Not everybody wants to manage a local copy/proxy/internal NPM registry etc)

We have limited amount of time, adopting this idea would impact productivity of team for long term. Making your application multi-region with backups and security is more important than protecting against a black swan event that would have limited impact. Even if npm goes down, you might have at maximum few hours of disruption for deploying in CI. This is relatively minor compared to recent us-east outage.

Re: We check our node_modules folder into source control

#203

Post by ownagefool »

somehnacct3757 wrote: post #169

Upsides from storing node_modules in repo are outweighed by the downsides. Unless of course you're Google-scale and can afford to contribute filesize fixes upstream, write fancy tooling to enforce commit-time workarounds, etc. Nobody working finger-to-feature has time for this. For your average npm shop which doesn't have infinite internet oil money, here is why the article recommendations won't work for you. Your CI…

Counter point, and ignoring download size as your typical CI probably doesn't download the entire history, when are we supposed to pretend we've reviewed our dependencies?

I'll admit I don't believe everyone always need to check every dep, but we're skating close nobody checking them ever.

Re: We check our node_modules folder into source control

#204

Post by charrondev »

matharmin wrote: post #174
DecoPerson wrote: post #113

Earlier quoted context omitted.

I might be misremembering but at runtime, the Node process loads .pnp.js, which is a kind of monolithic “compiled” modules file containing all the modules installed. No reading of the .zip files occurs at runtime. (Again, I might be misremembering or have misunderstood. Please confirm/deny this if you know.)

The pnp.js file has a list of modules, not the modules themselves. The modules are still read from zip files at runtime. Yarn doesn't do any kind of compilation itself - that would be done by build tools like webpack.

I’d be curious how “read file from a zip that I know the exact path to” performs compared to “recursively walk the node_modules directory and dynamically lookup the location of the file”.

Re: We check our node_modules folder into source control

#205

Post by ng12 »

xboxnolifes wrote: post #108
lf-non wrote: post #96

Earlier quoted context omitted.

Yes, but a lot of people prefer to have every commit in master in a buildable state. Otherwise things like git bisect become a lot more tedious.

> have every commit in master Yes, but a PR commit is not in master.

But then you lose the benefit. Hiding node_modules during code review is the easy part.

Re: We check our node_modules folder into source control

#206

Post by nickkell »

Gigachad wrote: post #106
nickkell wrote: post #102

I know pulling in lots of dependencies from various anonymous authors is a security risk. Can you be sure that all of the code has been vetted? This seems exacerbate this. You’re allowing developers to check in anything without oversight and it will be ignored just because it’s in this particular folder

That’s already exactly the same risk almost all web developers take currently. Yes it is a real threat, but it’s too hard to deal with and not often exploited.

We all hope and rely on the fact that popular OSS projects have enough eyeballs on them to make sure nothing malicious slips through. What is proposed here allows a load of changes to be made that completely bypass the normal review process

Re: We check our node_modules folder into source control

#207

Post by malkia »

DaiPlusPlus wrote: post #47
malkia wrote: post #43

Earlier quoted context omitted.

This file seems to be controlling what gets ("checked-in") - https://source.chromium.org/chromium/chromium/src/+/main:thi... In our industry (games) we often do that - checkin prebuilt code in the depots (typically "p4"). I'm not saying it's wrong/right, it's just what we do (not 100% fully, but almost, though people in IT/infra tend to do otherwise).

Ah Perforce... you have my sympathy. Does your team/company store art and asset blobs (even rendered FMVs?) in Perforce too, or do you have a separate asset-management system for that? If you do have two separate systems, how do you keep the asset-store and source-control in-sync? It's been a long time (easily 20 years now) since I dabbled in any high-end creative software (like 3ds, etc), but I remember they general…

We ... commit them. Probably some really exceptional cases stay behind some "ftp"-like service (not really ftp, something like this).

It maybe wasteful, but it's the established practice (it seems). People coming from other game companies pretty much use it too (some rare exceptions). Also the automotive/chip design industry uses them - and yes mainly for the big blobby non-diffable/mergeable assets.

Locks are terrible, and yet you gotta doit sometimes, as how else would you prevent people working on the same asset.

I'm (still) terrible with git, often screw up commits, and have to google search/stack overflow to get it right (I use it mainly for simple home projects). I can only imagine the pain and suffering a non-tech person would have with git. Also the metadata is quite lot for WFH conditions. Working remotely does not always mean working from a dumb terminal (I wish).

Re: We check our node_modules folder into source control

#208

Post by trog »

awill88 wrote: post #119
20after4 wrote: post #105

Earlier quoted context omitted.

Upgrading could introduce vulnerabilities just as easily as fixing them.

it’s possible, kind of a moot point as doing nothing induces a similar level of risk in my experience. Don’t touch your code for two weeks, I guarantee you that ‘npm audit’ will complain about some new issue.

Also worth noting the longer you wait to upgrade, the harder it can be to do so when you finally need to. If someone discovers a critical fault in the version you're running but you're several years out of date, upgrading can be a huge pain.

Re: We check our node_modules folder into source control

#210

Post by 3np »

tentacleuno wrote: post #154
3np wrote: post #56

Earlier quoted context omitted.

TFW you realize k8s isn't even used that widely inside Google.

What's your source on that? Would be interested to hear more. Aside: They seemed to like their Linux containers in 2014[0]. No idea if they're still using the same technique, or have maybe moved onto Kubernetes? [0]: https://www.theregister.com/2014/05/23/google_containerizati...

Only anecdotes from several people working there at different divisions. I think I've seen it mentioned here at times as well.

One of the lead devs - years of tenure - on one of the Android backend teams had barely heard of the name and wasn't sure what it was when I brought it up 2 years ago.

Linux containers including Docker are absolutely widespread as I understand it, though.

Post reply on HN