Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Scale Testing Docker Swarm to 30,000 Containers

blog.docker.com

Post reply on HN
Print view View original

11–19 of 19 posts

Re: Scale Testing Docker Swarm to 30,000 Containers

#11

Post by shykes »

IanCal wrote: post #10
nyandaber wrote: post #7

The article says the test involved 1000 nodes running 50 containers each, but the conclusion only talks about "no difference between 1st and 30000th container". So we can assume things between 30k and 50k containers didn't go as smoothly? Or did I miss something?

Possibly a copy error? The article says 30 containers on each node in the specs but 50 in the text.

It's a copy error. Initially we tested 30k containers, then later expanded the test to 50k. In both cases Swarm keeps scheduling without breaking a sweat.

Re: Scale Testing Docker Swarm to 30,000 Containers

#12

Post by fpp »

shykes wrote: post #11
IanCal wrote: post #10

Earlier quoted context omitted.

Possibly a copy error? The article says 30 containers on each node in the specs but 50 in the text.

It's a copy error. Initially we tested 30k containers, then later expanded the test to 50k. In both cases Swarm keeps scheduling without breaking a sweat.

Do understand that your target with this setup was to load test the Swarm manager and 30K is quite impressive.

Did you do / are you planning to do a test with other than the low end T2.micros to see how much of the API latency might be related to the type of nodes used.

In other words - if my intention would be to minimise the API latency - how would you approach this.

Planning to run a Swarm test (with a few 100 nodes) on Digital Ocean where we have 2 nics per machine - by this we can test latency / response time of the containers to their external work load plus to the API / Swarm manager on separate networks & nics.

Re: Scale Testing Docker Swarm to 30,000 Containers

#13

Post by lbradstreet »

kbody wrote: post #9
lbradstreet wrote: post #2

How can the 99th percentile latency be lower than the 90th? That must be a mistake or the result of multiple test runs?

Scheduling Delay is different from API response time.

The article was updated with the correct results before you saw it.

Re: Scale Testing Docker Swarm to 30,000 Containers

#15

Post by shykes »

wheaties wrote: post #14

When they say "API response time" I can't tell if that's some service API that rests in Docker itself or if it's the Docker API. Can anyone clarify? I also haven't had my coffee this morning.

It's the Swarm API call. In other words it's "container scheduling time".

Re: Scale Testing Docker Swarm to 30,000 Containers

#16

Post by justinsaccount »

I still don't quite get the use case for Swarm right now.

It seems to solve the problem of starting up a bunch of containers across a bunch of hosts, but not anything after that. Maybe the pluggable backends will enable more of the lifecycle management features like restarting containers when a host crashes or handling rolling upgrades.

Re: Scale Testing Docker Swarm to 30,000 Containers

#17

Post by ThePhysicist »

justinsaccount wrote: post #16

I still don't quite get the use case for Swarm right now. It seems to solve the problem of starting up a bunch of containers across a bunch of hosts, but not anything after that. Maybe the pluggable backends will enable more of the lifecycle management features like restarting containers when a host crashes or handling rolling upgrades.

I think Swarm transparently handles networking and volume management for containers, which can be challenging if your services require multiple containers that run on different hosts and need to talk to each other.

Re: Scale Testing Docker Swarm to 30,000 Containers

#19

Post by meteorfox »

bfirsh wrote: post #5
richardwhiuk wrote: post #3

Surely the API Response Time for the 99th percentile should be larger than that of the 90th? Percentile API Response Time Scheduling Delay 50th 150ms 230ms 90th 250ms 400ms 99th 200ms 400ms

Oops, there was a copy & paste error – sorry about that! Updated with correct figures.

Besides that, why are all these response time numbers multiple of 10, are these rounded up, or is it the precision of the timestamps used to compute these?
Post reply on HN