Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Docker to rate limit image pulls

docker.com

Post reply on HN
Print view View original

201–210 of 274 posts

Re: Docker to rate limit image pulls

#201

Post by peterwwillis »

fnord77 wrote: post #200
bsamuels wrote: post #14

There is very little reason for a build node to need to pull 200 images in 6 hours, and here is why: When a machine issues a ``docker build`` command, the program reads the relevant dockerfile to check for any base images that need to be pulled (a la "FROM:") These base images are identified based on the image repository, image name, and image tag. The first thing docker does is it checks its local registry and tries…

If everyone in your company (plus your CI system) is behind the same firewall/IP address, that's going to be a lot more than 200 pulls.

Luckily though, companies have cash to pay for registrations, and those that won't probably have engineers who can set up Squid proxies.

Re: Docker to rate limit image pulls

#202

Post by smabie »

manigandham wrote: post #164
mm89 wrote: post #119

Earlier quoted context omitted.

Exactly. It's hard enough to save money as it is. News websites blow my mind with this - if I forked over $5 to every news outlet I occasionally like to read, I'd be spending at least $500 maybe more per year JUST to get access to some random person's biased recant of what's happening in the world. If there were a news source that did the opposite of this, and basically provided a bullet list of objective, non-biased…

I explored building exactly that but turns out there’s no money in it. Most people want the narrative with the facts, if not more so.

There's a big market for this, it's just not for individuals. For example, Bloomberg distributes factual news on its terminal. The Bloomberg terminal even highlights important words in news stories so you can absorb the information more quickly. So if there was a earthquake somewhere, it might highlight the word "earthquake," the number of people that died, and the economic cost, for example.

Also there are news wire services that do mostly what you're describing. If you just want to be entertained (most people read news for entertainment), then they don't really care about the facts. They want to hear about so and so blasting so and so or whatever. But if you're trying to make money from information (traders, journalists, etc), then you really don't want to be reading the kind of stuff the New York Times is publishing.

Re: Docker to rate limit image pulls

#203

Post by tomc1985 »

rubyn00bie wrote: post #99

I originally came here to ask how folks use that many docker images in six hours (I'm mostly a Docker n00b, and not being facetious); however, after reading the article, I clicked to see how much unlimited is and it's $5 a month. Now my question has changed to: is $5 a month really a deal breaker for folks to get unlimited? Or what are the use cases where the cost is prohibitive? Open source or community projects? In…

You should be more shocked that everything else is so expensive.

The base costs for a lot of tech stuff (like bandwidth) are so cheap. But you would never know between these bullshit "what is it worth to you" pricing models and the number of middlemen trying to stick their hand into the pot. It's disgusting.

Re: Docker to rate limit image pulls

#204

Post by geerlingguy »

jpambrun wrote: post #179

I see a lot of people mentioning the low cost, saying that it's no big deal. It's not the cost that I find annoying; it's dealing with credentials and secrets..

Yeah for ephemeral work or CI builds especially, that's the most annoying part (especially for orgs where you might not want to put your personal credentials in repo settings... but you also don't want or have the funding structure to pay the $5/month for unique credentials for that org).

Re: Docker to rate limit image pulls

#205

Post by namibj »

stingraycharles wrote: post #78
bigmanwalter wrote: post #32

Earlier quoted context omitted.

I can usually max out my connection speed with torrents. This is rarely the case with direct downloads.

The start up time of a torrent is typically so long though, by the time it has connected to all peers and is downloading I have already downloaded the iso with a direct download anyway.

Soon, when QUIC is widely supported by the swarm (just needs updating to the current version), IPFS should be better than torrents for start-up delay, and much more importantly, it allows for sharing data across "torrents" when using content-defined chunking via rabin or buzhash. This means that things like common larger binaries get shared between images that include them, which should greatly increase the average amount of seeders for the chunks that make up an image.

Re: Docker to rate limit image pulls

#206

Post by [deleted] »

ViViDboarder wrote: post #107
rubyn00bie wrote: post #99

I originally came here to ask how folks use that many docker images in six hours (I'm mostly a Docker n00b, and not being facetious); however, after reading the article, I clicked to see how much unlimited is and it's $5 a month. Now my question has changed to: is $5 a month really a deal breaker for folks to get unlimited? Or what are the use cases where the cost is prohibitive? Open source or community projects? In…

It’s not common to need that many pulls nor is it hard to build your own images. If you’re deploying to a cluster with 200 machines, you could easily hit this if you use the public registry though. However, if you’re managing that size cluster you can probably afford the fee, but more importantly, you should probably pull once to a local registry and use that to deploy to your cluster anyway.

[deleted]

Re: Docker to rate limit image pulls

#207

Post by verdverm »

forty wrote: post #130
verdverm wrote: post #20

Earlier quoted context omitted.

They have registries already which work with docker after an auth setup. They are generally private to you. Yes, docker is a struggling company which sold some lines of business and is now trying to reinvent itself again towards developers. Given other recent moves, I'm not sure the new leadership understands how to do this.

They have registries but at least AWS' one is not free at all...

They all cost money, because it's expensive to ship bytes out of the cloud. Docker is likely going the way of the Dodo because they likely funded our free access with VC money.

They also probably made a bad assumption in that they could define the only container format before standards bodies got involved. Blitz Scaling was the mantra of the time, seems to have bitten those back who took a bite of that cake.

Re: Docker to rate limit image pulls

#208

Post by judge2020 »

zxcvbn4038 wrote: post #194
AaronFriel wrote: post #137

Earlier quoted context omitted.

I don't think that's what it means. > When a pull is attempted with a tag, the Registry checks the remote Checking the remote is a manifest pull.

Difference is HEAD request or conditional GET, the server will not send a file if it matches the time and/or tag of the version you have, so you are replying with a few bytes rather then (potentially) dozens or hundreds of megabytes. Same with all CDNs.

This still counts as a manifest pull for rate limiting purposes based on what i'm seeing in this thread.

Re: Docker to rate limit image pulls

#209

Post by staticassertion »

rubyn00bie wrote: post #99

I originally came here to ask how folks use that many docker images in six hours (I'm mostly a Docker n00b, and not being facetious); however, after reading the article, I clicked to see how much unlimited is and it's $5 a month. Now my question has changed to: is $5 a month really a deal breaker for folks to get unlimited? Or what are the use cases where the cost is prohibitive? Open source or community projects? In…

Yeah, I'm just going to pay for it. I think I probably already am, but if I'm not, I will. Even the enterprise 7/user isn't that bad at all. Github enterprise is like 3x that.

Re: Docker to rate limit image pulls

#210

Post by namibj »

mirekrusin wrote: post #10
brian_herman__ wrote: post #6

Earlier quoted context omitted.

Yeah, someone should come up with an alternative that mirrors it or something.

Built in torrent and/or ipfs?

The latter has content-dependent chunking support. That would allow for cross-image sharing of common data (large binaries, etc.).
Post reply on HN