Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Containers and Distributed Systems: Where They Came from and Where They’re Going

mesosphere.com

Post reply on HN
Print view View original

31–40 of 44 posts

Re: Containers and Distributed Systems: Where They Came from and Where They’re Going

#31

Post by chubot »

Just to double-click on container technology, why do you think it took so long for this technology that was built into Solaris to become mainstream with Docker?

I think VMWare deserves a mention here? And the terms OS Virtualization vs Hardware virtualization do as well (Ctrl-F doesn't find them.)

For awhile hardware virtualization (VMWare) was more prevalent, but it's more complex and has more overhead than OS virtualization (containers). That is how people solved the problem of having powerful machines and small workloads (or workloads with a lot of variance).

Although historically it might have been that hardware virtualization actually came first, in IBM mainframes. In the Unix world I guess OS virtualization came first.

Re: Containers and Distributed Systems: Where They Came from and Where They’re Going

#32

Post by godelmachine »

ChuckMcM wrote: post #2

This was a lot of fun, one of the things that doesn't get much air time is that back in the early 2000's when "clusters" and "NUMA SMP machines" were competing with each other the big argument for large SMP iron was ACID compliant SQL databases like Oracle. Now that Google has implemented an ACID compliant SQL database across clusters it puts the final nail in the argument (for me at least) that "Some things only wor…

Hi Chuck, by SMP do you mean Symmetric Multi Processing? Sincerely,

Re: Containers and Distributed Systems: Where They Came from and Where They’re Going

#33

Post by godelmachine »

ChuckMcM wrote: post #26
bogomipz wrote: post #18

Earlier quoted context omitted.

>"back in the early 2000's when "clusters" and "NUMA SMP machines" were competing with each other" Did you mean Numa vs SMP? Or something else maybe? How can a machine be a NUMA SMP? NUMA and SMP are fundamentally different architectures.

No, I was thinking non-uniform memory architecture, which is to say an SMP machine where the "speed" at which you can access RAM is dependent on the core or 'thread' from which you accessed it. Lack of memory uniformity was the compromise to achieve larger effective address spaces and "simple" programming. Today on a 24/48 core dual socket server you'll see the same sorts of thing, having a core using memory on the '…

A quick Google search after reading this comment of yours gave me this → http://www.google.com/patents/US5887146

Re: Containers and Distributed Systems: Where They Came from and Where They’re Going

#34

Post by ChuckMcM »

godelmachine wrote: post #32
ChuckMcM wrote: post #2

This was a lot of fun, one of the things that doesn't get much air time is that back in the early 2000's when "clusters" and "NUMA SMP machines" were competing with each other the big argument for large SMP iron was ACID compliant SQL databases like Oracle. Now that Google has implemented an ACID compliant SQL database across clusters it puts the final nail in the argument (for me at least) that "Some things only wor…

Hi Chuck, by SMP do you mean Symmetric Multi Processing? Sincerely,

Yes. All threads have equal computation capability and cache coherent (although not uniform) access to main memory. This was the 'big iron' of the day like the Sun, SGI, and Cray "big iron" boxes.

Re: Containers and Distributed Systems: Where They Came from and Where They’re Going

#35

Post by kecoco0207 »

There are currently possible ways for the public to access and invest in high potential private start-up companies like Mesosphere.

For example, EquityZen is a platform that lets people to invest in private pre-IPO:

https://equityzen.com/?utm_source=hackernews&utm_medium=comm...

Re: Containers and Distributed Systems: Where They Came from and Where They’re Going

#36

Post by ChuckMcM »

godelmachine wrote: post #33
ChuckMcM wrote: post #26

Earlier quoted context omitted.

No, I was thinking non-uniform memory architecture, which is to say an SMP machine where the "speed" at which you can access RAM is dependent on the core or 'thread' from which you accessed it. Lack of memory uniformity was the compromise to achieve larger effective address spaces and "simple" programming. Today on a 24/48 core dual socket server you'll see the same sorts of thing, having a core using memory on the '…

A quick Google search after reading this comment of yours gave me this → http://www.google.com/patents/US5887146

Ah yes the CCNUMA patent. This bit is the relevant part:

However, SMP systems suffer disadvantages in that system bandwidth and scalability are limited. Although multiprocessor systems may be capable of executing many millions of instructions per second, the shared memory resources and the system bus connecting the multiprocessors to the memory presents a bottleneck as complex processing loads are spread among more processors, each needing access to the global memory. As the complexity of software running on SMP's increases, resulting in a need for more processors in a system to perform complex tasks or portions thereof, the demand for memory access increases accordingly. Thus more processors does not necessarily translate into faster processing, i.e. typical SMP systems are not scalable. That is, processing performance actually decreases at some point as more processors are added to the system to process more complex tasks. The decrease in performance is due to the bottleneck created by the increased number of processors needing access to the memory and the transport mechanism, e.g. bus, to and from memory.

Alternative architectures are known which seek to relieve the bandwidth bottleneck. Computer architectures based on Cache Coherent Non-Uniform Memory Access (CCNUMA) are known in the art as an extension of SMP that supplants SMP's "shared memory architecture." CCNUMA architectures are typically characterized as having distributed global memory.

Re: Containers and Distributed Systems: Where They Came from and Where They’re Going

#37

Post by kakwa_ »

chubot wrote: post #31

Just to double-click on container technology, why do you think it took so long for this technology that was built into Solaris to become mainstream with Docker? I think VMWare deserves a mention here? And the terms OS Virtualization vs Hardware virtualization do as well (Ctrl-F doesn't find them.) For awhile hardware virtualization (VMWare) was more prevalent, but it's more complex and has more overhead than OS virtu…

I think docker succeeded not because it was a better container or virtualization solution, or because it was lighter than VMs. Jails or Solaris zones have existed for years or decades, and on Linux, we had openvz and vserver long before Docker. And I'm not even mentioning plain old chroot (with indeed some security issues).

I didn't follow the latest development, but for a while, Docker was not even that great in term of robustness and stability. I never was a big fan of the userland proxy for example. And from what I've read, upgrading from one docker version to another could be quite painful (disclaimer: I toyed with docker a little, I haven't used it in production yet).

IMHO, Docker succeeded because it brought a complete ecosystem around containers:

* Ways to generate the container image through Dockerfile

* Ways to compose an image on top of another (itself on top of another, etc...)

* Ways to share and distribute these images with Docker Registry

Re: Containers and Distributed Systems: Where They Came from and Where They’re Going

#39

Post by bogomipz »

ChuckMcM wrote: post #26
bogomipz wrote: post #18

Earlier quoted context omitted.

>"back in the early 2000's when "clusters" and "NUMA SMP machines" were competing with each other" Did you mean Numa vs SMP? Or something else maybe? How can a machine be a NUMA SMP? NUMA and SMP are fundamentally different architectures.

No, I was thinking non-uniform memory architecture, which is to say an SMP machine where the "speed" at which you can access RAM is dependent on the core or 'thread' from which you accessed it. Lack of memory uniformity was the compromise to achieve larger effective address spaces and "simple" programming. Today on a 24/48 core dual socket server you'll see the same sorts of thing, having a core using memory on the '…

>"Lack of memory uniformity was the compromise to achieve larger effective address spaces and "simple" programming."

Oh interesting. Might you have any links or suggested reading on this discussion and eventual compromise?

>"In the 2000's, before multi-core chips were a thing, there were two major camps, the 'super computer' camp, and the 'cluster' camp."

Is the cluster camp Beowulf then basically?

>"You got these very expensive fabrics from people like cray that would snoop access to memory from the cores and send coherency messages around to insure that if someone wrote something in to memory somewhere, everyone else's L1 or L2 cache got the message to invalidate what they were holding (shoot downs)."

Is this the MESI protocol then?

Thanks.

EDIT I just saw the link below about CCNUMA.

Re: Containers and Distributed Systems: Where They Came from and Where They’re Going

#40

Post by msla »

msarrel wrote: post #29
cat199 wrote: post #20

"You basically came up with Docker before Docker was around, or at least with things like Solaris zones and C groups." Except jails already existed on FreeBSD, and CP/CMS on mainfraimes in the 70s existed long before this...

The article says " It turned out that about twenty years earlier IBM did sort of solve that problem with partition isolation, but they did that on custom mainframe hardware and architecture."

Right. I remember looking at technologies for virtualization on x86 chips before actual virtualization hardware existed on those chips, and they were pretty heavyweight, including rewriting the binary to replace privileged opcodes which would fail silently or behave differently in ring 3 compared to ring 0 with sequences of opcodes which would work to maintain the isolation. It was closer to emulation than virtualization given how much work the host needed to do to maintain the illusion that the guest OSes were running on bare hardware, and it really pointed up how the x86 architecture of the time just wasn't designed for that kind of thing.

For the past decade or so, mainframe-level virtualization on x86 has been possible, at least, starting on high-end chips and working its way downwards to the commodity world. And, yes, it exists in ARM as well.

https://genode.org/documentation/articles/arm_virtualization

Post reply on HN