Blending complex systems made my latency 10x higher
srvaroa.github.io
Blending complex systems made my latency 10x higher
1–10 of 86 posts
Re: Blending complex systems made my latency 10x higher
#2> Kubernetes made my latency 10x higher
The title is a bit misleading, kub didn't cause the 10x latency - also latency was lower after they fixed their issues
TL;DR version - Migrate from EC2 to Kub; due to some default settings in Kiam & AWS Java SDK, latency of application increased, fixed after reconfiguration and kub latency lower than EC2
Re: Blending complex systems made my latency 10x higher
#3> Kubernetes made my latency 10x higher The title is a bit misleading, kub didn't cause the 10x latency - also latency was lower after they fixed their issues TL;DR version - Migrate from EC2 to Kub; due to some default settings in Kiam & AWS Java SDK, latency of application increased, fixed after reconfiguration and kub latency lower than EC2
hah.
So, title should read:
'Using default Kubernetes config settings made my latency 10x higher'
Re: Blending complex systems made my latency 10x higher
#4> Kubernetes made my latency 10x higher The title is a bit misleading, kub didn't cause the 10x latency - also latency was lower after they fixed their issues TL;DR version - Migrate from EC2 to Kub; due to some default settings in Kiam & AWS Java SDK, latency of application increased, fixed after reconfiguration and kub latency lower than EC2
hah. So, title should read: 'Using default Kubernetes config settings made my latency 10x higher'
Which isn't surprising. Kubernetes default settings are to work in all environments using overlay, not to be optimized for performance (make it work, then make it fast).
Re: Blending complex systems made my latency 10x higher
#5> Kubernetes made my latency 10x higher The title is a bit misleading, kub didn't cause the 10x latency - also latency was lower after they fixed their issues TL;DR version - Migrate from EC2 to Kub; due to some default settings in Kiam & AWS Java SDK, latency of application increased, fixed after reconfiguration and kub latency lower than EC2
It is relevant though as k8s makes everything more complicated so you have to deal with stuff like this. Also if it was a brand new app theyd maybe not notice the problem in the first place.
Re: Blending complex systems made my latency 10x higher
#6> Kubernetes made my latency 10x higher The title is a bit misleading, kub didn't cause the 10x latency - also latency was lower after they fixed their issues TL;DR version - Migrate from EC2 to Kub; due to some default settings in Kiam & AWS Java SDK, latency of application increased, fixed after reconfiguration and kub latency lower than EC2
hah. So, title should read: 'Using default Kubernetes config settings made my latency 10x higher'
[deleted]
Re: Blending complex systems made my latency 10x higher
#7> Kubernetes made my latency 10x higher The title is a bit misleading, kub didn't cause the 10x latency - also latency was lower after they fixed their issues TL;DR version - Migrate from EC2 to Kub; due to some default settings in Kiam & AWS Java SDK, latency of application increased, fixed after reconfiguration and kub latency lower than EC2
hah. So, title should read: 'Using default Kubernetes config settings made my latency 10x higher'
I might even go further - kiam is not a standard deployment supported by the kubernetes project like the api server or the scheduler or the autoscaler. See https://github.com/uswitch/kiam
That said it is a very common deployment strategy in ec2 to run kiam or kube2iam. I wish the kube core teams took over the development of an aws iam role service since issues like bad defaults would be solved much quicker. Your only other alternative is to use iam access keys and nobody likes that (security wise and it’s a pain to configure).
Re: Blending complex systems made my latency 10x higher
#8> Kubernetes made my latency 10x higher The title is a bit misleading, kub didn't cause the 10x latency - also latency was lower after they fixed their issues TL;DR version - Migrate from EC2 to Kub; due to some default settings in Kiam & AWS Java SDK, latency of application increased, fixed after reconfiguration and kub latency lower than EC2
It is relevant though as k8s makes everything more complicated so you have to deal with stuff like this. Also if it was a brand new app theyd maybe not notice the problem in the first place.
I'm not saying it's not relevant - it was actually a good technical read, they went through alot of detail with their troubleshooting.
I'm just saying the title is misleading, and clickbaity.
Re: Blending complex systems made my latency 10x higher
#9Earlier quoted context omitted.
hah. So, title should read: 'Using default Kubernetes config settings made my latency 10x higher'
I might even go further - kiam is not a standard deployment supported by the kubernetes project like the api server or the scheduler or the autoscaler. See https://github.com/uswitch/kiam That said it is a very common deployment strategy in ec2 to run kiam or kube2iam. I wish the kube core teams took over the development of an aws iam role service since issues like bad defaults would be solved much quicker. Your only…
It's built in to the AWS SDK since September
https://aws.amazon.com/blogs/opensource/introducing-fine-gra...
Re: Blending complex systems made my latency 10x higher
#10> Kubernetes made my latency 10x higher The title is a bit misleading, kub didn't cause the 10x latency - also latency was lower after they fixed their issues TL;DR version - Migrate from EC2 to Kub; due to some default settings in Kiam & AWS Java SDK, latency of application increased, fixed after reconfiguration and kub latency lower than EC2
In other words, switching to Kubernetes _did_ make their latency higher — otherwise they wouldn't have needed to reconfigure anything, would they? If you want to help k8s, try starting a pull-request to make the defaults better rather than playing spin-doctor telling people that well-documented problems don't exist.