Live data from Hacker News

How Meta patches Linux at hyperscale

thenewstack.io

11–20 of 114 posts

Re: How Meta patches Linux at hyperscale

#11
post #5

Never heard of this "hyperscale" concept before. How is this any different from... scaling?

Supermarkets and super cars aren't enough. We need hypermarkets and hyper cars to be current. So now we can’t just scale, I guess that’s for trucks, so to be current you need hyperscale.

Cybertruck and cyberscale?

(I'm joking)

Re: How Meta patches Linux at hyperscale

#13
post #2

Ksplice is the original live patching technology that got bought by Oracle and was later extended to user space programs while I worked there. It's a really neat technology that isn't made obsolete by the move to cloud, since you still don't want to have to restart the whole fleet at scale.

What's the live app patcher called?

Re: How Meta patches Linux at hyperscale

#14

"Draining and un-draining hosts is hard." I'd stop right there and fix that, because that's a bullshit reason. Cycling hosts in and out of service is easy unless you're not doing things properly. The Linux kernel is simply not designed to be live patched and it's a total hack to try to do it, it will never work 100% of the time, always be a source of uncertainty, and always be expensive in terms of engineering work.…

[deleted]

Re: How Meta patches Linux at hyperscale

#16
post #9

Earlier quoted context omitted.

https://en.wikipedia.org/wiki/Hyperscale_computing

> In computing, hyperscale is the ability of an architecture to scale appropriately as increased demand is added to the system. So yeah just scaling. I agree I've never heard the word "hyperscale" before and don't think we need that extra intensifier for a well-understood idea.

1x, 10x, 1000x are all "scales", yet problems may be a little bit different at each of them

Re: How Meta patches Linux at hyperscale

#17

"Draining and un-draining hosts is hard." I'd stop right there and fix that, because that's a bullshit reason. Cycling hosts in and out of service is easy unless you're not doing things properly. The Linux kernel is simply not designed to be live patched and it's a total hack to try to do it, it will never work 100% of the time, always be a source of uncertainty, and always be expensive in terms of engineering work.…

> My guess would be that this approach is papering over organizational dysfunction.

So what? At large enough scale organization problems are harder than technical ones: if you can fix the former with the latter, that's still a win.

Re: How Meta patches Linux at hyperscale

#18

"Draining and un-draining hosts is hard." I'd stop right there and fix that, because that's a bullshit reason. Cycling hosts in and out of service is easy unless you're not doing things properly. The Linux kernel is simply not designed to be live patched and it's a total hack to try to do it, it will never work 100% of the time, always be a source of uncertainty, and always be expensive in terms of engineering work.…

It may be easy to cycle hosts in and out, but it can also be time consuming, apparently. In the article it mentions taking 45 days to patch all hosts. The article also points out that this is too long for security updates.

Nothing will work 100% of the time. If their patching mechanism is thoroughly tested and battle hardened, I think the risk would be acceptable. Once you do the initial kpatch security upgrade, you could even schedule the machine for serivce so that it's not relying on that, limiting your exposure to bugs.

Re: How Meta patches Linux at hyperscale

#19

"Draining and un-draining hosts is hard." I'd stop right there and fix that, because that's a bullshit reason. Cycling hosts in and out of service is easy unless you're not doing things properly. The Linux kernel is simply not designed to be live patched and it's a total hack to try to do it, it will never work 100% of the time, always be a source of uncertainty, and always be expensive in terms of engineering work.…

[deleted]
Post reply on HN