Billions wasted on Hadoop startups, the same will eventually be true of Docker

I agree, I think the next evolution of software deployment is definitely heading to sub-program sizes. I do want to point out that we've seen this before, it was called CGI. It's not exactly the same, and things will be better this time (more isolation, better tooling), but if we're doing functions-as-a-service now, we were doing scripts-as-a-service much earlier.

I think the great unbundling of the future is definitely coming and in fact it's already here -- it's just unevenly distributed.

> I agree that Docker doesn't have a huge and profitable future ahead of it, because it's providing commodity infrastructure. Rather I think it's interesting to think about what the next level of software deployment decomposition will be, and I'd wager that it's FaaS (ie serverless).

This was not what I meant to get across -- Docker may have a huge and profitable future ahead of it, but is only tangentially related to the near assured continuance of containerization. Docker as a company and project's goals are different, they do more than simply offer a way to run containers, and they have for a very long time -- my point was that the literal use of the word "docker" should die, because we should be just referring to containerization in the general sense (no matter which lib you're using). It's like the "tissue" vs "kleenex" debate, in a way.

docker-containers and it's related counterparts are abstractions. Useful abstractions don't necessarily equate to a new line of business.

The best attributes of containers (IMO) are packaging and distribution. What business and operators need is a repeatable, easy way to deploy applications across there infrastructure. Containers are one piece of that story.

The bigger piece, and IMO, where the business viability is, is the orchestration layer. Containers aren't very useful by themselves, you need a way to get your application online. That's where Kubernetes comes in.

You need to understand large organizations and their challenges to see what layer of the containerization stack holds the most value.

Long-term, I see 'linux containers' as we know them going away. The industry is going to move to something like [1]: lightweight, hardware-assisted VM/container hybrids. But, no matter what happens at the containerization layer, the orchestration layer is the piece that adds business value to end-users (eg, not AWS or other hosting providers).

1: https://katacontainers.io/

Yeah given how often people mistakenly call containers VMs or assume they have VM-like properties (like the ability to run a different kernel), this is tripping a lot of developers up, not just lay people.

> Isolation and sandboxing is virtualization. In a container, the applications seem to have an operating system and machine to themselves.

I don't think this is quite true, generally you can isolate and sandbox something without virtualizing anything. In this case to isolate and sandbox without a loss of functionality, we are virtualizing and swapping swap implementations out from underneath an application/process. That is not always the case though -- if you unplug a machine's ethernet cable it is isolated, but you did not virtualize anything, if you deny access to a folder on disk for a process you are isolating without virtualizing, unless you mean that any kernel interaction (like blocking an `fopen()`) is "virtualization".

> A single Unix process and its address space is also a kind of virtual machine, creating the illusion that the process has a machine all to itself. Thanks to virtual memory, other processes are not even visible; they are in a different name space of pointers. That concept breaks for multi-process applications: processes are aware of each other through manipulations of shared resources like files. Or effects like not being able to bind a networking port because some other process is tying it up. The next level of virtualization is to have namespaces for resource-related namespaces in the system beyond the address space. As far as just the filesystem space goes, we can virtualize with tools like chroot. A group of applications can have their own global /etc configuration, their own version of the C library in /lib and so on. That's the beginning of "containerization".

This is a good summary of how containerization works, and the layers of isolation provided by the kernel -- the approach here is isolation by providing a fake, but that does not mean that isolation + sandboxing = virtualization, that just means that these forms of virtualization can be used to provide isolation & sandboxing.

OK, I'd like to note that it's also the past -- CGI was (and in some dark corners still is) a thing.

> Every single person whose product is not Internet infrastructure, that wrote or configured their own unnecessary custom LXC or Docker and VM environment because 'Docker changes everything'.

Yeah but those people now have way easier to run local environments?

> If a company's product is machine learning for detecting cancer and their ops person has a custom Docker/kubernetes environment they're misusing their employer for their own technical interest

???? If your company's product is machine learning, and developers who must work on that product need to set up their environment on their local machines, docker is easier to get started with than VMs, runs faster, consumes less resources. While it might be arguable that it's easier, there are literal money savings to be had by running a docker container instead of a full VM.

Kubernetes has much more complexity and many more tradeoffs involved so I can see that being a much heavier decision.

What you're saying is that overzealous ops people who are looking to pad their own resumes should not be allowed to run amuck, and I agree with that, but docker is not the poster boy for engineering largess -- and I'd argue it never was. Companies and research groups/smaller distributions have been using containerization very productively for a long time.