Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Interview with Eric Brewer

medium.com

Post reply on HN
Print view View original

91–100 of 142 posts

Re: Interview with Eric Brewer

#91

Post by dba7dba »

jedberg wrote: post #29

A funny and somewhat off topic story -- back in 2003, before the Google IPO, Google was doing a recruiting event at Berkeley. They brought a few of their folks with them: their founder Larry, one of their female engineers, Marissa, and some others. They did a little talk, and during the Q&A, professor Brewer told Larry that there was an opening in the PhD program and he was welcome to it. Larry politely declined. Aft…

It amuses me that Professor Brewer is now working for Larry. Pretty sure Prof Brewer isn't exactly tiptoeing around Larry. He can work ANYWHERE.

Re: Interview with Eric Brewer

#92

Post by ForHackernews »

rdtsc wrote: post #85
nickpsecurity wrote: post #49

One thing that bothers me about the article is that it shows a recurring problem: IT not knowing what it knows. The NoSQL movement didn't notice that NonStop Architecture scaled linearly to thousands of cores with strong-consistency, five 9's, and SQL support. In the mid-80's. Instead of making a low-cost knockoff, like cluster movement did for NUMA's, they ditched consistency altogether and launched NoSQL movement.…

> There is hope: Google went from a famous, NoSQL player to inventing an amazing, strong-consistency RDBMS (F1). Let's hope more follow. The problem is at a large distributed scale forcing consistency is basically fighting with the laws of physics. Is something in Australia always consistent with something in US? Well "same time" is a funny thing in physics because it doesn't actually exist. So we have to work really…

I don't see how using GPS receivers for a time index is "a kludge". It seems like an extremely sensible and practical solution to the problem. Hey, turns out we've already got this satellite constellation broadcasting an extremely accurate time signal!

It's a hack, in all the positive senses of the word.

Re: Interview with Eric Brewer

#93

Post by otterley »

bch wrote: post #87
otterley wrote: post #80

Earlier quoted context omitted.

Linux has copy-on-write block devices that make it possible to efficiently layer container filesystems. FreeBSD has no such thing as far as I know; the best you can do involves hard links (correct me if I'm wrong).

https://en.wikipedia.org/wiki/UnionFS Edit: addendum Additionally, a hardlink solution (as you point out) really doesn't sound too outrageous to me. A trivial tool to write to manage immutable things like binaries. Or is there some trouble I'm failing to see (possible)?

According to the mount_unionfs(8) man page:

> THIS FILE SYSTEM TYPE IS NOT YET FULLY SUPPORTED (READ: IT DOESN'T WORK) AND USING IT MAY, IN FACT, DESTROY DATA ON YOUR SYSTEM.

Re: Interview with Eric Brewer

#94

Post by rdtsc »

lallysingh wrote: post #89
rdtsc wrote: post #85

Earlier quoted context omitted.

> There is hope: Google went from a famous, NoSQL player to inventing an amazing, strong-consistency RDBMS (F1). Let's hope more follow. The problem is at a large distributed scale forcing consistency is basically fighting with the laws of physics. Is something in Australia always consistent with something in US? Well "same time" is a funny thing in physics because it doesn't actually exist. So we have to work really…

Afford to install GPS antennas? My last 4 phones had them. So does my watch.

Well it seems you could put Meinberg out of business ;-) Their GPS receivers seem to cost more than the phone.

https://www.meinbergglobal.com/english/products/compact-ntp-...

Can probably find some used ones on ebay for $1.5k or so.

Re: Interview with Eric Brewer

#95

Post by 15thandwhatever »

mbesto wrote: post #67
jacquesm wrote: post #52

Earlier quoted context omitted.

> why not just completely abstract the OS away and focus the attention on scaling the OS services that matter? Because it adds a layer that makes no sense unless you have very specific use cases. Though I see the point regarding people efficiency, that one makes good sense (see other comment in this sub-thread) > Why is that when I decide that I need to scale out, I need to copy every library of the OS and every line…

> See: chef, configuration management and various deployment services A chef script is basically the automation of "I need to copy every library of the OS and every line of code for the kernel and redeploy it every time I add a node?" I'm sorry if you didn't pick up on my implied remark. Two problems are then introduced when automating those actions: (1) it doesn't negate the fact that I need to store and deploy a 70…

What I do on our app servers is create a new user for each app (via configuration management).

That gives me both environment separation (A needs Ruby 1.9, B needs Ruby 2.0), resource accounting on a per-app basis, and a repeatable foundation in case I need to re-deploy the server or spin up new instances.

Re: Interview with Eric Brewer

#96

Post by rdtsc »

ForHackernews wrote: post #92
rdtsc wrote: post #85

Earlier quoted context omitted.

> There is hope: Google went from a famous, NoSQL player to inventing an amazing, strong-consistency RDBMS (F1). Let's hope more follow. The problem is at a large distributed scale forcing consistency is basically fighting with the laws of physics. Is something in Australia always consistent with something in US? Well "same time" is a funny thing in physics because it doesn't actually exist. So we have to work really…

I don't see how using GPS receivers for a time index is "a kludge". It seems like an extremely sensible and practical solution to the problem. Hey, turns out we've already got this satellite constellation broadcasting an extremely accurate time signal! It's a hack, in all the positive senses of the word.

It is a kludge because it adds the whole GPS system into a the mix of a set of software components. So now Spanner/F1 is not just a "apt-get install" way it is an apt-get install + buy server GPS receiver + make sure to install antenna on roof + make sure it is not bent or knocked down.

Re: Interview with Eric Brewer

#97

Post by jganetsk »

rdtsc wrote: post #96
ForHackernews wrote: post #92

Earlier quoted context omitted.

I don't see how using GPS receivers for a time index is "a kludge". It seems like an extremely sensible and practical solution to the problem. Hey, turns out we've already got this satellite constellation broadcasting an extremely accurate time signal! It's a hack, in all the positive senses of the word.

It is a kludge because it adds the whole GPS system into a the mix of a set of software components. So now Spanner/F1 is not just a "apt-get install" way it is an apt-get install + buy server GPS receiver + make sure to install antenna on roof + make sure it is not bent or knocked down.

The use of a GPS signal in Spanner improves performance but is not necessary for correctness.

Re: Interview with Eric Brewer

#98

Post by bch »

otterley wrote: post #93
bch wrote: post #87

Earlier quoted context omitted.

https://en.wikipedia.org/wiki/UnionFS Edit: addendum Additionally, a hardlink solution (as you point out) really doesn't sound too outrageous to me. A trivial tool to write to manage immutable things like binaries. Or is there some trouble I'm failing to see (possible)?

According to the mount_unionfs(8) man page: > THIS FILE SYSTEM TYPE IS NOT YET FULLY SUPPORTED (READ: IT DOESN'T WORK) AND USING IT MAY, IN FACT, DESTROY DATA ON YOUR SYSTEM.

You're right. My personal reference for unionfs is NetBSD, which doesn't include such a warning [0]; apparently freebsd's implementation needs some love.

[0] http://netbsd.gw.com/cgi-bin/man-cgi?mount_union++NetBSD-cur...

Re: Interview with Eric Brewer

#99

Post by jganetsk »

jacquesm wrote: post #18
DannoHung wrote: post #10

Earlier quoted context omitted.

Sorry, but what costs do containers incur? From my understanding, the resource overhead should be exceedingly minimal (disk space would ostensibly be the largest drawback, if you don't spend time cutting out the fat. Personally, I see this as a tooling issue since fat containers are completely orthogonal to how a container really executes). I understand some of the situation with IO isn't perfect yet, but I haven't h…

An OS has some pretty extensive insight into the processes that it executes, a container is an OS with a single application, so containers (assuming they take basic precautions for isolation) are not going to be able to schedule with anywhere near the efficiency that multiple processes on a single OS will. I can see some (mostly potential at this point) security advantages but that's about it (and maybe those advanta…

A container is an OS with a single application

No. A container is just a tarball of user-space code run with some isolation. The kernel is still the kernel. Run multiple containers on a machine, and the OS manages all of their processes at once.

Re: Interview with Eric Brewer

#100

Post by jganetsk »

jacquesm wrote: post #30
mixmastamyk wrote: post #24

Earlier quoted context omitted.

The linux kernel knows what is happening in the containers. There is little performance overhead, and security is not a big advantage of the approach, VMs are better.

The linux kernel knows what's happening inside the containers but it can not de-duplicate any components that are present in multiple individual containers and it can not ensure that only one version of a package is present. This will potentially lead to cache trashing and more IO than strictly speaking required for a given workload. Of course it does make it easier to package and deploy applications (and to ensure t…

Why can't it dedup? Presumably, containers share base images, which are read-only, and could thus be deduped in memory by the overlay filesystem implementation.
Post reply on HN