Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Interview with Eric Brewer

medium.com

Post reply on HN
Print view View original

101–110 of 142 posts

Re: Interview with Eric Brewer

#101

Post by ForHackernews »

rdtsc wrote: post #96
ForHackernews wrote: post #92

Earlier quoted context omitted.

I don't see how using GPS receivers for a time index is "a kludge". It seems like an extremely sensible and practical solution to the problem. Hey, turns out we've already got this satellite constellation broadcasting an extremely accurate time signal! It's a hack, in all the positive senses of the word.

It is a kludge because it adds the whole GPS system into a the mix of a set of software components. So now Spanner/F1 is not just a "apt-get install" way it is an apt-get install + buy server GPS receiver + make sure to install antenna on roof + make sure it is not bent or knocked down.

So what? Google are already managing thousands or hundreds of thousands of different hardware and software components. IMO, using GPS is a good solution to a hard problem.

Re: Interview with Eric Brewer

#102

Post by la6470 »

With all these container talks people forget Solaris zones which were pretty advanced sort of containers in Solaris. However Sun was honest about not mentioning zones as the solution for everything. The most important problem with containerization is that due to dependence of multiple containers on the same kernel of the host or VM OS , any kind of upgrade specially the security patching is virtually impossible to do without taking full downtime.

Re: Interview with Eric Brewer

#103

Post by voidr »

I have been using Amazon Web Services and other cloud platforms for over a year now, and I never really felt that VMs were the bottleneck in any way. Can someone explain to me the advantage of containers here?

I know that containers are faster because they don't virtualize the hardware, however it comes at the cost of security.

Re: Interview with Eric Brewer

#104

Post by [deleted] »

jacquesm wrote: post #47
menage wrote: post #31

Earlier quoted context omitted.

Yes, I think it's safe to say that Google operates at scale. A lot of user-facing services at Google have to be over-provisioned in order to handle the cyclical usage patterns (the daily query peak is far higher than the average for most services) and to be able to survive the loss of a datacenter or two. This results in a lot of under-utilized servers for a big fraction of the time. So by packing lots of medium and…

I think it's a fairly safe assumption to say that Google (and FB and a bunch of other extremely large web properties) run into different problems than those that are faced on a day-to-day basis by most run-of-the-mill web companies. Thank you for the insight into the number of processes inside a typical Borg container, so that was basically a kind of 'heavy process' rather than a complete application with all depende…

[deleted]

Re: Interview with Eric Brewer

#106

Post by xorcist »

menage wrote: post #69
jacquesm wrote: post #48

Earlier quoted context omitted.

I can see how there will be less duplication than with virtualization (because you share the kernel, rather than running multiple instances of the kernel) but I can't see how duplication will be less than with dedicated hardware.

It won't - for any single given service, running on a dedicated system will probably be able to squeeze out a tiny bit more performance than running on a shared system with containers, for whatever fraction of the time that you're close to maxing out that server's performance. But in practice (at Google-scale, anyway), that's dwarfed by the efficiency gains you can get by squeezing lots of things on to the same machi…

Hang on there, are you saying you can squeeze in more applications on any given server using containers, rather than just running them in the regular filesystem? That doesn't make sense.

No, you use containers despite the fact that your hardware utilization goes down (mainly because no shared pages between applications), because your huge sprawling environment is too hard to change with flag days.

Re: Interview with Eric Brewer

#107

Post by digitalzombie »

I saw a kubernete talk at a local meetup.

Google have 40 programmers dedicated to that project. It's still very beta btw all programmed in Go.

There's also mesos and I think you can use both in tandem since they're targeting a different thing.

Anyway if anybody is doing or thinking about containers check Kubernete and Mesos out. Also of course docker and rocket. Kubernete officially support docker and will be supporting rocket.

There are also article about how rump kernel are better than containers. Just fyi.

Re: Interview with Eric Brewer

#108

Post by AlisdairO »

jeswin wrote: post #81
nickpsecurity wrote: post #49

One thing that bothers me about the article is that it shows a recurring problem: IT not knowing what it knows. The NoSQL movement didn't notice that NonStop Architecture scaled linearly to thousands of cores with strong-consistency, five 9's, and SQL support. In the mid-80's. Instead of making a low-cost knockoff, like cluster movement did for NUMA's, they ditched consistency altogether and launched NoSQL movement.…

> The NoSQL movement didn't notice that NonStop Architecture scaled linearly to thousands of cores with strong-consistency, five 9's, and SQL support. Also incredibly expensive. Take this case, Google took off because they were able to scale-out with off-the-shelf hardware, compared to the millions banks were pouring in for scale-up configurations which handled much less load. Scale-up can quickly hit hard limits, be…

> For the majority of use-cases out there, NoSQL databases offer enough consistency.

For applications with a nontrivial data model, ensuring that each logical operation only does a single document update (or that multiple nontransactional updates cause no conflict in maintaining consistency in the presence of other logical operations) is actually really challenging - and it adds a substantial design overhead to every new feature added. I think you're being extremely optimistic in your assertion that NoSQL systems are that widely safely applicable. My experience has been that NoSQL-based systems stay mostly-consistent because they happen to experience very little concurrent activity, not through understanding/design on the users' part.

This is not to make light of the situations where NoSQL systems shine, but the idea that higher levels of consistency are rarely useful does not match my experience at all.

> NoSQL movement, if you wanna call it that, took off because most apps (including Google, including Financial Services, even Health Care) don't need some types of consistency offered by relational databases.

I'd say that Mongo (for example) took off because:

- They really nailed the setup experience for new users (which RDBMSs historically sucked at).

- The data model is much easier for simple apps.

- They had some fairly creative techniques for making their system look good - unacknowledged writes, and claims of support for replication which didn't really fully work.

- Most programmers don't really understand the ins-and-outs of maintaining consistency.

Re: Interview with Eric Brewer

#109

Post by ossreality »

philip1209 wrote: post #14
mark_l_watson wrote: post #7

Great Interview! I worked as a contractor at Google in 2013 and loved their infrastructure. It was amazing to fire off a Borg job that used hundreds to thousands of servers, and the web based tools for tracking the job, fantastic logging to drill into problems, etc. And, Borg was two generations ago! Even though I am very happy doing what I am now, sometimes I literally wake up in the morning thinking about Google's…

How hard was it to learn these tools as a contractor? I hear that Google is more aggressively making their tools open-source to help the portability of their engineers' skills. Specifically, friends have told me that onboarding at Google is difficult because every tool is a proprietary one built on top of more proprietary systems. In addition, people who leave Google have trouble interviewing because they rely heavil…

lol, as if Microsoft's internal tools for managing deployments is more portable? It's a rickety piece of shit and high ups keep forcing us to use it when it's vastly inferior.

And the WinMo team built, basically, Kubernetes on steroids, without containers (All the other high level concepts, abstractions, naming, labels, "pods" of associated services, shared secrets, etc)

And the fucking dumbasses bulldozed it in the reorg 18 months ago. Stupidest, stupidest thing the company has ever done, in a wave of otherwise brilliant moves.

Honestly, the truth is, Google has great tools for this that you learn, in order to be incredibly efficient. No one else has that ramp up, because they're stuck in the old mindset of managing services and move at 1% the pace.

Re: Interview with Eric Brewer

#110

Post by bcantrill »

jacquesm wrote: post #51
bcantrill wrote: post #42

Earlier quoted context omitted.

Containers have actually been around for a long, long time -- and have well-known operational efficiencies. So this isn't a "shiny new layer-of-abstraction", it's a tried-and-true abstraction that has been operating in production and at scale for a the better part of the last decade.[1] That said, the developer fascination with containers (which is to say, Docker) is new, and there is a bit of a wild west of abstract…

How do you see this delivering on the security component of the isolation? (Not that VMs are perfect in this respect but it seems to me that containers are a lot less solid)

Well, you need to get specific. Speaking for SmartOS[1], we've been running containers in production for over a decade; while security is never solved per se (that is, there is always the possibility that defects will result in future vulnerabilities), the reality is that there is a lot of experience running this system in multi-tenant, internet-facing production and that CVEs against the Solaris-based zones technology have been few and far between -- and I would imagine that the same can be said of FreeBSD Jails. These two technologies stand in sharp contrast to the Linux "container" technology (which is to say: namespaces), which is much more relatively immature and doesn't necessarily share the same design constraints as zones and jails. So if by containers you mean zones or jails, the security component of the isolation is well understood and in hand; if by containers you mean Linux namespaces, then yes, a "lot less solid" is probably phrasing it generously.

[1] https://wiki.smartos.org/display/DOC/Zones

Post reply on HN