Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What's new in Docker 1.13: prune, secrets, checkpoints and more

cloudshare.com

Post reply on HN
Print view View original

51–60 of 62 posts

Re: What's new in Docker 1.13: prune, secrets, checkpoints and more

#51

Post by tra3 »

MrBuddyCasino wrote: post #43
scanr wrote: post #21

Earlier quoted context omitted.

Docker may be overkill to start but it's relatively low cost to implement and it will definitely pay dividends over time: * You can be sure that what you're running locally is exactly what you'll be running on the server * Your deployment experience will be the same regardless of which tech stack you're using for the web application * There are many places you can deploy docker containers (Google GCE, Amazon ECS, Ama…

> it's relatively low cost to implement Running Docker in production takes a huge amount of effort to get right and is not easily done.

I dont believe that's an accurate assessment. If the grandparent wants to run a one off container with reproducible results, something like docker-compose is perfect. If he wants to run a multi-node microservices architecture then the story gets more complicated.

Re: What's new in Docker 1.13: prune, secrets, checkpoints and more

#52

Post by djstein »

vkjv wrote: post #45
lojack wrote: post #39

Curious what methods others use for handling secrets at build time (using docker-compose). I'm currently installing (private) dependencies at runtime by mounting my secrets as a volume. I couldn't find a method that didn't seem to have some risk of inadvertently exposing them.

There are only methods that I'm aware of: - Exposing the secrets on a (http) server that the Dockerfile can use to fetch - What we use: Create a one time use secret that is destroyed after the image is built and before it is pushed.

>What we use: Create a one time use secret that is destroyed after the image is built and before it is pushed.

This approach has sparked my interest, could you post an example of any open source docker-compose file and/or associated scripts that would do this?

Re: What's new in Docker 1.13: prune, secrets, checkpoints and more

#53

Post by andmarios »

cpuguy83 wrote: post #46
andmarios wrote: post #42

Earlier quoted context omitted.

But the thing people complain most isn't because they want to delete everything but because they need docker rm, xargs and complex bash foo to delete the containers and images they don't need. For example I want to delete all old and all untagged versions of an image. I want to delete all stopped containers that use a specific image, or that were created more than two weeks ago. I want to delete all images starting w…

fyi, you do not need xargs. `docker rm $(docker ps -q --filter blash)` But agree, `prune` is currently sledge hammer and needs some refinement. It's not about not being well thought out, it's about getting something out there that can be built on top of.

Thanks, it was too long since I used filter and it wasn't that interesting. Seems much better now!

Re: What's new in Docker 1.13: prune, secrets, checkpoints and more

#54

Post by LeanderK »

tra3 wrote: post #51
MrBuddyCasino wrote: post #43

Earlier quoted context omitted.

> it's relatively low cost to implement Running Docker in production takes a huge amount of effort to get right and is not easily done.

I dont believe that's an accurate assessment. If the grandparent wants to run a one off container with reproducible results, something like docker-compose is perfect. If he wants to run a multi-node microservices architecture then the story gets more complicated.

i run a lot of small projects with docker-compose on a single host and it makes deploying my changes very easy. Maybe there is a low cost setting it up, but i think eve with a small project it pays it divides pretty fast.

Re: What's new in Docker 1.13: prune, secrets, checkpoints and more

#55

Post by sigjuice »

dandermotj wrote: post #31

I'm really looking forward to seeing the scientific community adopt docker as a way to distribute reproducible research and coursework. MIT 6.S094 has a Dockerfile[^1] that contains all the software required for taking part in the class. This is a huge boon for getting stuck into the class and its coursework. [^1]: http://selfdrivingcars.mit.edu/files/Dockerfile

How is publishing a Dockerfile even remotely reproducible? Almost every Dockerfile is a series of apt-get install, or yum install or pip install commands. How do I know what versions of packages I am downloading or whether they will even be available to download if I build from this Dockerfile, say two months from now?

IMHO, every Dockerfile has left-pad written all over it.

Re: What's new in Docker 1.13: prune, secrets, checkpoints and more

#56

Post by tonyhb »

nefasti wrote: post #48
scarlac wrote: post #8

Earlier quoted context omitted.

Docker is very well suited for local development and testing, particularly since the launch of Docker for Mac and Windows. It makes utilities like MAMP less necessary. But apart from local development, I'd say that depends on your needs. If you want more ease-of-use, and you run a single-server hosting environment with multiple projects, it may be easier to keep doing that without adding Docker. But if you want incre…

Docker for local development for us been a pain in the butt. - We've hit performance problems with the filesystem, - Problem with caching things like yarn and npm install - The need to constantly rebuild the images for changes to be picked up. - Dificulty dealing with single docker file for prod and testing, making us want to montain 2 docker files. Probably some bad setup of our part, but we've been using on product…

I don't understand how kubernetes solved your base image issue. That's a clustering system, so by default it can't help.

It sounds as though your setup doesn't work with the immutable filesystems introduced by docker. That's not an issue with docker at all - just something to learn.

I can't imagine dev or deployment without docker any more - all of my tests, yarn installs, dev workflow and prod runs through it.

Re: What's new in Docker 1.13: prune, secrets, checkpoints and more

#57

Post by lojack »

djstein wrote: post #52
vkjv wrote: post #45

Earlier quoted context omitted.

There are only methods that I'm aware of: - Exposing the secrets on a (http) server that the Dockerfile can use to fetch - What we use: Create a one time use secret that is destroyed after the image is built and before it is pushed.

>What we use: Create a one time use secret that is destroyed after the image is built and before it is pushed. This approach has sparked my interest, could you post an example of any open source docker-compose file and/or associated scripts that would do this?

I did actually encounter this solution while researching the problem, didn't love it, but you can check out the solution at: https://github.com/docker/docker/issues/13490#issuecomment-1...

As long as you add the file and remove it in the same command it doesn't get committed as an extra layer, so the container won't have any history of the secrets. You'll run into problems if you do multiple RUN's or an ADD and then RUN.

Re: What's new in Docker 1.13: prune, secrets, checkpoints and more

#58

Post by patrec »

lacampbell wrote: post #3

I'm just a guy that wants to deploy web apps. Is docker overkill for me? Basically, I want to be able to test something on my local machine under the same conditions it will be running on my server. Containerisation seems like the only way to do this that doesn't involve keeping packages and system configurations in sync in two or more systems.

If you only care about controlling the software configuration and versions, nix (nixos.org/nix/) will do this far more elegantly than docker.

Re: What's new in Docker 1.13: prune, secrets, checkpoints and more

#59

Post by wenbert »

ewang1 wrote: post #18
wenbert wrote: post #11

Does this mean the qcow2 disk space usage in Mac is fixed?

Yep, just tested it. The qcow2 disk space gets reclaimed on Docker restart.

Sweet. Now time to play. I followed that bug until i got tired of it. Took a couple of months! Good on them for fixing it. Thanks!

Re: What's new in Docker 1.13: prune, secrets, checkpoints and more

#60

Post by yeukhon »

sigjuice wrote: post #55
dandermotj wrote: post #31

I'm really looking forward to seeing the scientific community adopt docker as a way to distribute reproducible research and coursework. MIT 6.S094 has a Dockerfile[^1] that contains all the software required for taking part in the class. This is a huge boon for getting stuck into the class and its coursework. [^1]: http://selfdrivingcars.mit.edu/files/Dockerfile

How is publishing a Dockerfile even remotely reproducible? Almost every Dockerfile is a series of apt-get install, or yum install or pip install commands. How do I know what versions of packages I am downloading or whether they will even be available to download if I build from this Dockerfile, say two months from now? IMHO, every Dockerfile has left-pad written all over it.

Good question.

Reproduciblity is all about the starting point. Computers are electronic, so if your computation requires high entropy from some random source and supposed next run there is not enough entropy your experiment may fail. But that's really really really a corner case. Docker image keeps the state of the starting point (kernel, packages, history of bashrc etc) are kept version controlled. It is as if someone gave you a copy of the virtualbox image.

So how do we lock down?

1) When you start with a Dockerfile, specify the version of the packages you are installing

2) When you want to reproduce, you can rebuild an image with that Dockerdile.

3) But most people are just going to use your image which is always the same now or next year. Building image != launching a container using an image.

Post reply on HN