Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Visual Studio Code 1.7 overloaded npmjs.org, release reverted

code.visualstudio.com

Post reply on HN
Print view View original

41–50 of 94 posts

Re: Visual Studio Code 1.7 overloaded npmjs.org, release reverted

#41

Post by [deleted] »

vonklaus wrote: post #38
PudgePacket wrote: post #26

I wonder if any warning was given to npm that they would be getting this potentially huge new source of traffic. It doesn't seem to be mentioned anywhere.

Eh, NPM is a pretty core service and both sides probably should have done things a bit differently. I don't neccessarily think vscode needed to reach out to NPM to let them know they were going to be consuming their public API. Both teams appear to be in communication as a result however-- which is good. This will likely lead to more fault tolerant systems on both projects and hopefully more collaboration & features…

[deleted]

Re: Visual Studio Code 1.7 overloaded npmjs.org, release reverted

#42

Post by [deleted] »

seldo wrote: post #22
Arnavion wrote: post #14

Earlier quoted context omitted.

Can you elaborate on what the issue is and how you want it to be fixed? Is it just something like rate-limiting requests or something more fundamental? Edit: Answered at https://news.ycombinator.com/item?id=12861118

A VSCode person can (and probably will) answer in more detail, but at heart it's simple: if you want to add type-checking goodness to a library that isn't itself written in TypeScript, you can create a thing called a declaration file: https://github.com/DefinitelyTyped/DefinitelyTyped Microsoft publishes a list of known good declaration files for popular npm packages to npm, under the scope @types: https://www.npmjs.…

[deleted]

Re: Visual Studio Code 1.7 overloaded npmjs.org, release reverted

#45

Post by Aldo_MX »

jessaustin wrote: post #21
cerebellum42 wrote: post #15

Earlier quoted context omitted.

I thought that sentence sounded very trump-ish. The feature was so great that npmjs couldn't keep up with it, it was yuuuuuge!

...and they made npm Inc. pay for it!

it was a tremendous overload

Re: Visual Studio Code 1.7 overloaded npmjs.org, release reverted

#47

Post by onlydnaq »

seldo wrote: post #22
Arnavion wrote: post #14

Earlier quoted context omitted.

Can you elaborate on what the issue is and how you want it to be fixed? Is it just something like rate-limiting requests or something more fundamental? Edit: Answered at https://news.ycombinator.com/item?id=12861118

A VSCode person can (and probably will) answer in more detail, but at heart it's simple: if you want to add type-checking goodness to a library that isn't itself written in TypeScript, you can create a thing called a declaration file: https://github.com/DefinitelyTyped/DefinitelyTyped Microsoft publishes a list of known good declaration files for popular npm packages to npm, under the scope @types: https://www.npmjs.…

> At the moment there are few enough that they could fetch a list of all of them and cache it (the public registry lacks a documented API for doing that right now, but we can certainly provide one)

Might I suggest having a bloom filter containing all the existing type declaration (which would be quite small) and only querying the registry if the bloom filter reports the type declaration as a positive.

Since the filter can be really small it will probably scale a lot better than a complete list of all type-declarations, and a new filter could be downloaded by the clients every now and then.

Re: Visual Studio Code 1.7 overloaded npmjs.org, release reverted

#48

Post by kozhevnikov »

seldo wrote: post #22
Arnavion wrote: post #14

Earlier quoted context omitted.

Can you elaborate on what the issue is and how you want it to be fixed? Is it just something like rate-limiting requests or something more fundamental? Edit: Answered at https://news.ycombinator.com/item?id=12861118

A VSCode person can (and probably will) answer in more detail, but at heart it's simple: if you want to add type-checking goodness to a library that isn't itself written in TypeScript, you can create a thing called a declaration file: https://github.com/DefinitelyTyped/DefinitelyTyped Microsoft publishes a list of known good declaration files for popular npm packages to npm, under the scope @types: https://www.npmjs.…

Can they publish a package that contains a list of all their other packages under @types?

Re: Visual Studio Code 1.7 overloaded npmjs.org, release reverted

#49

Post by lmm »

ec109685 wrote: post #31
seldo wrote: post #13

I'd just like to say on behalf of npm that Microsoft's handling of this incident was A+. As soon as we alerted them to the issue they were all hands on deck and did a rollback. We've been really pleased that Microsoft chose to put their @types packages into the npm registry rather than a separate, closed system, and in general happy with Microsoft's support of node and npm. We're confident we can make the new feature…

"Many requests to the registry as the entire nation of India" per what time unit?

How does the time unit make a difference? If they're making the same number of requests/second as India then they're also making the same number of requests/hour or requests/day, no?

Re: Visual Studio Code 1.7 overloaded npmjs.org, release reverted

#50

Post by natuac »

seldo wrote: post #16
sync wrote: post #9

Shouldn't all these requests be cached by a CDN? What exactly is overloading?

CDNs don't usually cache 404s. VSCode was looking for @types packages for any and every npm package its users were using. Packages that had a type description caused no issue, but most packages don't, so we had a > 1000% spike in 404s. Our workaround before MS did the rollback was to cache 404s for @types packages specifically, and it was effective enough that the registry never really went down.

"a > 1000% spike in 404s" overloaded your servers? Such are your generation times? Can I bring the entire NPM ecosystem down from my ADSL line using some silly threaded code to make requests to randomly named packages?
Post reply on HN