Containers and Distributed Systems: Where They Came from and Where They’re Going • Hacker News Board

Re: Containers and Distributed Systems: Where They Came from and Where They’re Going

#41

Post by zurn » Fri, Oct 13, 2017, 6:13 AM UTC

ChuckMcM wrote: post #26

bogomipz wrote: post #18

Earlier quoted context omitted.

>"back in the early 2000's when "clusters" and "NUMA SMP machines" were competing with each other" Did you mean Numa vs SMP? Or something else maybe? How can a machine be a NUMA SMP? NUMA and SMP are fundamentally different architectures.

No, I was thinking non-uniform memory architecture, which is to say an SMP machine where the "speed" at which you can access RAM is dependent on the core or 'thread' from which you accessed it. Lack of memory uniformity was the compromise to achieve larger effective address spaces and "simple" programming. Today on a 24/48 core dual socket server you'll see the same sorts of thing, having a core using memory on the '…

A big category of cluster-friendly HPC cluster codes works by running in lockstep on all the nodes. Apparently it works quite well for things like weather forecasting where the problem is naturally divided into a grid but there is still significant communication needed between grid tiles. So it's quite different from memcached type things.

https://www.cs.fsu.edu/~engelen/courses/HPC/Synchronous.pdf

Re: Containers and Distributed Systems: Where They Came from and Where They’re Going

#42

Post by raarts » Fri, Oct 13, 2017, 9:16 AM UTC

peterwwillis wrote: post #16

Pyxl101 wrote: post #7

Earlier quoted context omitted.

Distributed ACID transactions across multiple DBs or clusters has been possible for a while if you have been willing to pay the latency penalty for the transaction to complete, hasn't it? As far as I know, Google systems have to pay this latency cost too. From an analysis of Spanner and Calvin [1]: > The cost of two-phase commit is particularly high in Spanner because the protocol involves three forced writes to a lo…

> I suspect that Google's innovation in making these cross-region or cross-cluster transactions viable is partly in their network I think this understates it. Production distributed systems suck balls with unreliable infrastructure, and anyone who has ever tried to do realtime replication of all of their data (and lots of it) across the internet knows how completely unrealistic it is. It's like running a power cord a…

> It's like running a power cord across a highway to power a refrigerator.

??? Color me baffled.

Re: Containers and Distributed Systems: Where They Came from and Where They’re Going

#43

Post by phaemon » Fri, Oct 13, 2017, 9:50 AM UTC

kuschku wrote: post #12

phaemon wrote: post #11

Earlier quoted context omitted.

That blog post is from a year ago. And since Cockroach DB 1.0 was only released in May this year, it's a bit misleading to link to that post as though it was the current state of the sofware.

That blog post is referenced in their FAQ today , under the topic of what it can't do right now . Sorry if I misunderstood the situation, I’d appreciate any updated links. See: https://www.cockroachlabs.com/docs/stable/frequently-asked-q...

Ah, my apologies then. It's their fault for not updating the FAQ and you can hardly be blamed for quoting from it!

There was an update to that 6 months later: https://www.cockroachlabs.com/blog/better-sql-joins-in-cockr...

I'd assume they've made even more improvements since, so they really should update their docs.

Re: Containers and Distributed Systems: Where They Came from and Where They’re Going

#44

Post by mathattack » Sun, Oct 15, 2017, 5:17 AM UTC

ChuckMcM wrote: post #2

This was a lot of fun, one of the things that doesn't get much air time is that back in the early 2000's when "clusters" and "NUMA SMP machines" were competing with each other the big argument for large SMP iron was ACID compliant SQL databases like Oracle. Now that Google has implemented an ACID compliant SQL database across clusters it puts the final nail in the argument (for me at least) that "Some things only wor…

Good to see you outside the context of HN. You're an immense value to the community.