Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Interview with Eric Brewer

medium.com

Post reply on HN
Print view View original

81–90 of 142 posts

Re: Interview with Eric Brewer

#81

Post by jeswin »

nickpsecurity wrote: post #49

One thing that bothers me about the article is that it shows a recurring problem: IT not knowing what it knows. The NoSQL movement didn't notice that NonStop Architecture scaled linearly to thousands of cores with strong-consistency, five 9's, and SQL support. In the mid-80's. Instead of making a low-cost knockoff, like cluster movement did for NUMA's, they ditched consistency altogether and launched NoSQL movement.…

> The NoSQL movement didn't notice that NonStop Architecture scaled linearly to thousands of cores with strong-consistency, five 9's, and SQL support.

Also incredibly expensive. Take this case, Google took off because they were able to scale-out with off-the-shelf hardware, compared to the millions banks were pouring in for scale-up configurations which handled much less load. Scale-up can quickly hit hard limits, before it becomes exponentially expensive to continue on the path. This is true even today.

NoSQL movement, if you wanna call it that, took off because most apps (including Google, including Financial Services, even Health Care) don't need some types of consistency offered by relational databases. Many large apps are significantly de-normalized and have many foreign-key less tables, often filled up by scheduled jobs. That's fine for most apps; NoSQL architectures recognize that and users consider that in design.

For the majority of use-cases out there, NoSQL databases offer enough consistency. For the remaining use-cases, there are tools available in NoSQL databases to make them work, though it requires a bit of work.

Re: Interview with Eric Brewer

#82

Post by Artemis2 »

mark_l_watson wrote: post #7

Great Interview! I worked as a contractor at Google in 2013 and loved their infrastructure. It was amazing to fire off a Borg job that used hundreds to thousands of servers, and the web based tools for tracking the job, fantastic logging to drill into problems, etc. And, Borg was two generations ago! Even though I am very happy doing what I am now, sometimes I literally wake up in the morning thinking about Google's…

I heard from a Googler that many of other companies' public tools (databases, job schedulers, etc.) were often created by ex-Googlers who missed what they were using at Google.

Re: Interview with Eric Brewer

#83

Post by [deleted] »

otterley wrote: post #80
kayman wrote: post #61

I love the idea of using containers. Due to linux popularity and google's backing, containers will be next. But FREEBSD had jails since back in the day. What's the benefit of containers over bsd jails?

Linux has copy-on-write block devices that make it possible to efficiently layer container filesystems. FreeBSD has no such thing as far as I know; the best you can do involves hard links (correct me if I'm wrong).

[deleted]

Re: Interview with Eric Brewer

#84

Post by andyidsinga »

jeswin wrote: post #81
nickpsecurity wrote: post #49

One thing that bothers me about the article is that it shows a recurring problem: IT not knowing what it knows. The NoSQL movement didn't notice that NonStop Architecture scaled linearly to thousands of cores with strong-consistency, five 9's, and SQL support. In the mid-80's. Instead of making a low-cost knockoff, like cluster movement did for NUMA's, they ditched consistency altogether and launched NoSQL movement.…

> The NoSQL movement didn't notice that NonStop Architecture scaled linearly to thousands of cores with strong-consistency, five 9's, and SQL support. Also incredibly expensive. Take this case, Google took off because they were able to scale-out with off-the-shelf hardware, compared to the millions banks were pouring in for scale-up configurations which handled much less load. Scale-up can quickly hit hard limits, be…

yep. see also : shared nothing architecture

Re: Interview with Eric Brewer

#85

Post by rdtsc »

nickpsecurity wrote: post #49

One thing that bothers me about the article is that it shows a recurring problem: IT not knowing what it knows. The NoSQL movement didn't notice that NonStop Architecture scaled linearly to thousands of cores with strong-consistency, five 9's, and SQL support. In the mid-80's. Instead of making a low-cost knockoff, like cluster movement did for NUMA's, they ditched consistency altogether and launched NoSQL movement.…

> There is hope: Google went from a famous, NoSQL player to inventing an amazing, strong-consistency RDBMS (F1). Let's hope more follow.

The problem is at a large distributed scale forcing consistency is basically fighting with the laws of physics. Is something in Australia always consistent with something in US? Well "same time" is a funny thing in physics because it doesn't actually exist. So we have to work really hard to keep consistency going.

To illustrage, here is how Spanner (F1) works:

---

Spanner's 'TrueTime' API depends upon GPS receivers and atomic clocks that have been installed in Google's datacentres to let applications get accurate time readings locally without having to sync globally.

---

I call that a cludge in the general sense, not amazing future technology. Yes if you can afford to install GPS antennas on your datacenters you can handle it and it is ok. But it is a crutch. "Well but there is NTP one might say say". Yeah there is, and connectivity to that fails as well.

The one interesting research area though is CRDTs. These are datatypes that know how to auto-converge to a known value even if they experience temporary inconsistency. So you basically experience temporary inconsistency but it fixes itself.

Re: Interview with Eric Brewer

#86

Post by andyidsinga »

nickpsecurity wrote: post #63
zobzu wrote: post #57

Earlier quoted context omitted.

thats a pretty good comment actually. there's quite a bit of similarity. we generally tend to jump into these as "omg awesome new tech" with a very narrow view. But it also helps boosting more though-out techs (even thus it feels less efficient to go through that route first, its perhaps the only route that works with human: try, fail, try again, etc.)

Yeah we do. My only guess is it's two things: (a) our industry is horrendous at communicating previous generation's wisdom in a usable way; (b) a social phenomenon. Quick example of the first are industry pro's locking up their good advice in obscure, expensive books and cutting edge research silo'd into ACM, IEEE, etc. The other is a social thing that leads to the "network" effect. People flock to something for what…

re (a) : is not just a communication problem. a lot of solutions from previous generation are dramatically less easy than newer offerings. IMHO nosql and cloud compute offerings we're like this.

the problem with folks hating on new technology "x" is often failing to see that the new thing might only offer one significant advantage over the prior tech, or they see it, but discount it too heavily to be motivated to try it.

Re: Interview with Eric Brewer

#87

Post by bch »

otterley wrote: post #80
kayman wrote: post #61

I love the idea of using containers. Due to linux popularity and google's backing, containers will be next. But FREEBSD had jails since back in the day. What's the benefit of containers over bsd jails?

Linux has copy-on-write block devices that make it possible to efficiently layer container filesystems. FreeBSD has no such thing as far as I know; the best you can do involves hard links (correct me if I'm wrong).

https://en.wikipedia.org/wiki/UnionFS

Edit: addendum

Additionally, a hardlink solution (as you point out) really doesn't sound too outrageous to me. A trivial tool to write to manage immutable things like binaries. Or is there some trouble I'm failing to see (possible)?

Re: Interview with Eric Brewer

#88

Post by ChuckMcM »

rdtsc wrote: post #85
nickpsecurity wrote: post #49

One thing that bothers me about the article is that it shows a recurring problem: IT not knowing what it knows. The NoSQL movement didn't notice that NonStop Architecture scaled linearly to thousands of cores with strong-consistency, five 9's, and SQL support. In the mid-80's. Instead of making a low-cost knockoff, like cluster movement did for NUMA's, they ditched consistency altogether and launched NoSQL movement.…

> There is hope: Google went from a famous, NoSQL player to inventing an amazing, strong-consistency RDBMS (F1). Let's hope more follow. The problem is at a large distributed scale forcing consistency is basically fighting with the laws of physics. Is something in Australia always consistent with something in US? Well "same time" is a funny thing in physics because it doesn't actually exist. So we have to work really…

I was pleased when I read the spanner paper to see that they had gone to locally authoritative stratum 0 clocks. I long time ago in a different universe it seems, I struggled with the notions of replcated naming logs which needed to converge to a common and consistent results. It wasn't until I got to play with the combinator technology at Blekko did I feel like there was a good answer to the problem. Clouds of idempotent expressions of execution don't need clocks at all, if you have them all, you have your answer. That was a neat result.

Re: Interview with Eric Brewer

#89

Post by lallysingh »

rdtsc wrote: post #85
nickpsecurity wrote: post #49

One thing that bothers me about the article is that it shows a recurring problem: IT not knowing what it knows. The NoSQL movement didn't notice that NonStop Architecture scaled linearly to thousands of cores with strong-consistency, five 9's, and SQL support. In the mid-80's. Instead of making a low-cost knockoff, like cluster movement did for NUMA's, they ditched consistency altogether and launched NoSQL movement.…

> There is hope: Google went from a famous, NoSQL player to inventing an amazing, strong-consistency RDBMS (F1). Let's hope more follow. The problem is at a large distributed scale forcing consistency is basically fighting with the laws of physics. Is something in Australia always consistent with something in US? Well "same time" is a funny thing in physics because it doesn't actually exist. So we have to work really…

Afford to install GPS antennas? My last 4 phones had them. So does my watch.

Re: Interview with Eric Brewer

#90

Post by andyidsinga »

imo, if containers are the future ( very plausible ), then, things like Aws lambda are just as plausible if only just a bit further out.

I think this is the case due to granularity of workloads and what apears to be a continuum in the workload container from metal > vm > containers > lambdas (as first class workloads).

fun stuff

Post reply on HN