Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

An Ubuntu kernel bug causes container crashes

lwn.net

Post reply on HN
Print view View original

21–30 of 133 posts

Re: An Ubuntu kernel bug causes container crashes

#21

Post by _y5hn »

markstos wrote: post #3

This was even worse than the headline made it sound. If you had `unattended-upgrades` running and had the "automatic reboot" option enabled, then all your Ubuntu 20.04 servers running Docker would reboot themselves and not come back up. First, the bug was in a security branch. Second, it wasn't just the containers that crashed. If you booted containers on boot via Docker, then the host OS kernel-panicked and crashed…

Wouldn't rollback of kernel be a choice in grub menu?

It's pretty standard for all distros to have that choice.

Re: An Ubuntu kernel bug causes container crashes

#22

Post by ungamedplayer »

jillesvangurp wrote: post #15
MBCook wrote: post #10

This seems like the kind of thing that automated testing should have been able to catch. It’s not like running Docker is a small use-case these days.

My thoughts exactly, the details of what this bug is about technically are interesting and fascinating but the key take away is that something went terribly wrong with Ubuntu's testing processes. This should not have shipped without more scrutiny. Somebody presumably cut some corners there and it's worrying that that is possible at all. I actually rolled out Ubuntu 22.04 to a few servers a few weeks ago. Pretty uneve…

> . One reason I've always preferred Ubuntu over Red Hat for servers is that with Red Hat/Centos essentially everything I care about is perpetually and hopelessly out of date and obsolete

This is exactly why you choose it. Lesser chance of insanity.

Re: An Ubuntu kernel bug causes container crashes

#23

Post by speedgoose »

atoav wrote: post #16

The cost of complexity showing itself. A sysadmin friend of mine is totally against docker and his reason is that he wants as little complexity as is needed on his systems. Complexity, he says, leads to emergent behavior.

I think it’s more complex to ask people to package software in a good way without software containers.

Re: An Ubuntu kernel bug causes container crashes

#25

Post by akx »

markstos wrote: post #3

This was even worse than the headline made it sound. If you had `unattended-upgrades` running and had the "automatic reboot" option enabled, then all your Ubuntu 20.04 servers running Docker would reboot themselves and not come back up. First, the bug was in a security branch. Second, it wasn't just the containers that crashed. If you booted containers on boot via Docker, then the host OS kernel-panicked and crashed…

> have to follow the procedure for restoring from backup or re-mounting the root volume on an alternate house to revert the kernel version being run.

Or add `systemd.mask=docker.service` to your boot parameters to prevent Docker from starting.

Re: An Ubuntu kernel bug causes container crashes

#26

Post by throwaway787544 »

atoav wrote: post #16

The cost of complexity showing itself. A sysadmin friend of mine is totally against docker and his reason is that he wants as little complexity as is needed on his systems. Complexity, he says, leads to emergent behavior.

Complexity is just another part of natural systems. It's not something to be avoided for it's own sake, in as much as we should avoid having eyeballs, because they are crazily complex. Yet we tend to like ours and find them worth whatever cost they incur.

The emergent behavior of containerization has had an overall positive effect, even if it has annoying costs.

Re: An Ubuntu kernel bug causes container crashes

#27

Post by atoav »

lixtra wrote: post #17
atoav wrote: post #16

The cost of complexity showing itself. A sysadmin friend of mine is totally against docker and his reason is that he wants as little complexity as is needed on his systems. Complexity, he says, leads to emergent behavior.

This is not a docker bug, it is a kernel bug. It could be triggered by other complex applications that use kernel container features.

Or it could not be triggered if you don't use containers.

Re: An Ubuntu kernel bug causes container crashes

#28

Post by scns »

jillesvangurp wrote: post #15
MBCook wrote: post #10

This seems like the kind of thing that automated testing should have been able to catch. It’s not like running Docker is a small use-case these days.

My thoughts exactly, the details of what this bug is about technically are interesting and fascinating but the key take away is that something went terribly wrong with Ubuntu's testing processes. This should not have shipped without more scrutiny. Somebody presumably cut some corners there and it's worrying that that is possible at all. I actually rolled out Ubuntu 22.04 to a few servers a few weeks ago. Pretty uneve…

For a more recent Kernel on Ubuntu i can recommend Xanmod.

https://xanmod.org

Re: An Ubuntu kernel bug causes container crashes

#29

Post by ladyanita22 »

ungamedplayer wrote: post #22
jillesvangurp wrote: post #15

Earlier quoted context omitted.

My thoughts exactly, the details of what this bug is about technically are interesting and fascinating but the key take away is that something went terribly wrong with Ubuntu's testing processes. This should not have shipped without more scrutiny. Somebody presumably cut some corners there and it's worrying that that is possible at all. I actually rolled out Ubuntu 22.04 to a few servers a few weeks ago. Pretty uneve…

> . One reason I've always preferred Ubuntu over Red Hat for servers is that with Red Hat/Centos essentially everything I care about is perpetually and hopelessly out of date and obsolete This is exactly why you choose it. Lesser chance of insanity.

Ýou can always choose Fedora Server if you want a more up-to-date server OS.

Re: An Ubuntu kernel bug causes container crashes

#30

Post by iasay »

atoav wrote: post #16

The cost of complexity showing itself. A sysadmin friend of mine is totally against docker and his reason is that he wants as little complexity as is needed on his systems. Complexity, he says, leads to emergent behavior.

I agree with this. I work on a very large platform and the cost of complexity like this is immense. It’s a not insignificant measurable loss versus not using docker and kubernetes. We’d have been better off using flat EC2 instances for everything and not incurring the packaging complexity, the repository management complexity, the pipeline complexity and the extreme staffing cost to keep multiple large kubernetes clusters running and understood.

Even container security and compliance around it is a measurable loss on its own which is trivially solved if you have bare EC2 instances and a patch cycle.

Post reply on HN