Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Checkpoint and restore Docker containers with CRIU

blog.circleci.com

Post reply on HN
Print view View original

11–15 of 15 posts

Re: Checkpoint and restore Docker containers with CRIU

#11

Post by vezzy-fnord »

alfonsodev wrote: post #9

What are the posible practical uses of this technology ? I was thinking that could be useful to get "snapshots" of a system when a fatal exception is raised, then download the files and practice a forensic analysis. Maybe it's too much overhead ? other uses?

Fast startup, obviating the need for any readiness notification (the point of execution can be guaranteed to correspond to readiness), time-travel debugging, process migration, rollback/rollforward of execution state, orthogonal persistence, etc.

The original motivation for checkpointing, IIRC, was not to lose the state of really long (days, weeks...) computations. But it goes much further.

Re: Checkpoint and restore Docker containers with CRIU

#12

Post by batbomb »

alfonsodev wrote: post #9

What are the posible practical uses of this technology ? I was thinking that could be useful to get "snapshots" of a system when a fatal exception is raised, then download the files and practice a forensic analysis. Maybe it's too much overhead ? other uses?

Batch processing with preemptable jobs.

Re: Checkpoint and restore Docker containers with CRIU

#13

Post by 616c »

vezzy-fnord wrote: post #4
616c wrote: post #3

A very good interview with one of the CRIU devs on FLOSS Weekly. https://twit.tv/show/floss-weekly/334 http://www.youtube.com/watch?v=w5ftqjOrpfA Some of this stuff is crazy. Moving processes is cool, but faking TCP/IP state between containers to basically transfer a TCP/IP stream from one machine to the other with client service devices noticing? INSANE. I am a Linux enthusiast and very novice *nix sysadmin. It is w…

EROS and KeyKOS accomplished the same OS-wide some 25 years ago, though they were also built on a completely different process model. Grasshopper [1] was another system from the same time period built around orthogonal persistence. Bolting checkpointing on top of a monolithic Unix is harder, though doable. DragonFly BSD has had a rather elegant and simple mechanism called sys_checkpoint(2) since 2003, but is still li…

> Bolting checkpointing on top of a monolithic Unix is harder

Indeed. In the interview this comes up, as they wanted their patches mainlined into Linux development and Linus, and unusually so given the joking about his outbursts, said they were insane or going for the impossible and passed.

Re: Checkpoint and restore Docker containers with CRIU

#14

Post by laurencerowe »

alfonsodev wrote: post #9

What are the posible practical uses of this technology ? I was thinking that could be useful to get "snapshots" of a system when a fatal exception is raised, then download the files and practice a forensic analysis. Maybe it's too much overhead ? other uses?

Being able to use AWS spot instances for long running batch processes would be great. We have some bioinformatics analysis jobs that take days to complete and this could cut their cost by ~80%.

Re: Checkpoint and restore Docker containers with CRIU

#15

Post by marcosnils »

laurencerowe wrote: post #14
alfonsodev wrote: post #9

What are the posible practical uses of this technology ? I was thinking that could be useful to get "snapshots" of a system when a fatal exception is raised, then download the files and practice a forensic analysis. Maybe it's too much overhead ? other uses?

Being able to use AWS spot instances for long running batch processes would be great. We have some bioinformatics analysis jobs that take days to complete and this could cut their cost by ~80%.

I'm interested about your use case, can you ping me at @marcosnils?

Thx!.

Post reply on HN