Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Interview with Eric Brewer

medium.com

Post reply on HN
Print view View original

121–130 of 142 posts

Re: Interview with Eric Brewer

#121

Post by [deleted] »

menage wrote: post #31
jacquesm wrote: post #17

Earlier quoted context omitted.

> Primarily because dedicated servers are a lot less efficient Assuming you're operating at scale I don't see why that would be the case. And if you're not, what's the point? > The more different things you can pack on a machine while still ensuring that the high-priority/low-latency jobs get prompt access to the resources that they've reserved, the higher overall utilization you can achieve (and hence bring costs do…

Yes, I think it's safe to say that Google operates at scale. A lot of user-facing services at Google have to be over-provisioned in order to handle the cyclical usage patterns (the daily query peak is far higher than the average for most services) and to be able to survive the loss of a datacenter or two. This results in a lot of under-utilized servers for a big fraction of the time. So by packing lots of medium and…

[deleted]

Re: Interview with Eric Brewer

#122

Post by pdkl95 »

collyw wrote: post #118
liveoneggs wrote: post #66

Earlier quoted context omitted.

Because rails app 1 needs version x.y.z and rails app 2 needs version a.b.c and RVM sucks and managing gems sucks. This is, literally, the reason. You can replace rails with any similarly bad technology. I got this explanation a few weeks ago at my job: the java build process (I'm not kidding) has such a complex dependency graph that it must spin up full containers to do each build. ---- If dynamic languages supporte…

Does Ruby not have a virtualenv equivalent? (I am reading through the comments trying to work out if I should learn Docker. I already know how to use a virtualenv and I already know how to use a VM.)

While I'm not familiar with python & virtualenv, the problem of that would be solved by putting rails in a container (isolating the ruby environment in a fairly easily deployable way) was solved many years ago with Gemfiles/bundler[1] that pin your dependencies, and the various ways of deploying multiple rubies. RVM does indeed suck hard, but rbenv[2] is very simple (I've used it personally to develop and deploy a few projects).

One of the preferred ways to deploy rails is (was? I did this ~2 years ago) was to check it in with git and the name/version of the ruby environment to use is stored in the file ".rbenv-version", with dependencies managed by bundler (which you point to a local gem server). Install is then 1) use rbenv/ruby-install to install basic ruby, 2) install app with "git clone", and 3) run bundler. Many tools exist to do this in one step over ssh/etc automagically.

Even better than rbenv, you can just use chruby[3] to point to any rubies you want; just check one into your project itself (or whatever) and configure the siteruby/etc load paths to point to project directories. Really, chruby just fixes up your dev environment to point to a specific ruby; you set the actual project to be self contained with known paths, just like you would do de facto in a container.

While dependency issues were a problem back in the ruby 1.8 / rails 2.x days, this

[1] http://bundler.io/ [2] https://github.com/sstephenson/rbenv [3] https://github.com/postmodern/chruby

Re: Interview with Eric Brewer

#123

Post by rylee »

collyw wrote: post #118
liveoneggs wrote: post #66

Earlier quoted context omitted.

Because rails app 1 needs version x.y.z and rails app 2 needs version a.b.c and RVM sucks and managing gems sucks. This is, literally, the reason. You can replace rails with any similarly bad technology. I got this explanation a few weeks ago at my job: the java build process (I'm not kidding) has such a complex dependency graph that it must spin up full containers to do each build. ---- If dynamic languages supporte…

Does Ruby not have a virtualenv equivalent? (I am reading through the comments trying to work out if I should learn Docker. I already know how to use a virtualenv and I already know how to use a VM.)

There's Bundler, which allows you to install all gems to a local path (e.g. `vendor/`) and then one simply runs `bundle exec ` to do the needful.

Re: Interview with Eric Brewer

#124

Post by mirceal »

zobzu wrote: post #58
sp332 wrote: post #34

Earlier quoted context omitted.

If you have a statically-linked binary you can just run it in a container. You don't need a whole OS userland in there.

[...] since the whole userland that you need is included in the binary (worth noting) you just dont get the debugging stuff (which is okay as long as you can choose)

would argue that it's not just the userland. you only have the functionality you need and your bits. for example an http server would have the http stack + your app. there is no userland, there are no drivers, just your app as the only thing that is running on the machine. your app basically merges with the "kernel"

Re: Interview with Eric Brewer

#125

Post by TheIronYuppie »

jesstaa wrote: post #105

Containers, like virtual machines before them, aren't the future of computing. They're how we manage legacy apps. The future of computing is not this horrible kludge.

Disclaimer: I work at Google on Kubernetes

The problem containers are trying to solve is not isolation of environments (though, that's a tremendously good outcome).

The problem is how do you develop microservices that are self-contained, easily composable and inherently portable. It doesn't matter if you're running on Java 1.1 on AIX 3.0 or Node on Ubuntu 26, if I have a ball of computing providing a service in Dubai, and want to move it to Ireland to take advantage of computing space that just opened up, containers make that trivial (and about a million other scenarios).

Re: Interview with Eric Brewer

#126

Post by TheIronYuppie »

voidr wrote: post #103

I have been using Amazon Web Services and other cloud platforms for over a year now, and I never really felt that VMs were the bottleneck in any way. Can someone explain to me the advantage of containers here? I know that containers are faster because they don't virtualize the hardware, however it comes at the cost of security.

Disclaimer: I work at Google on Kubernetes & containers.

I tried to address it here: https://news.ycombinator.com/item?id=9570639

But in summary, the basic thing that containerized software offers is inherent portability and simple composition. VMs aren't going anywhere; running a container on top of them just makes everything more powerful.

Re: Interview with Eric Brewer

#127

Post by TheIronYuppie »

kayman wrote: post #61

I love the idea of using containers. Due to linux popularity and google's backing, containers will be next. But FREEBSD had jails since back in the day. What's the benefit of containers over bsd jails?

Disclaimer: I work at Google on Kubernetes & Containers

I mentioned earlier - https://news.ycombinator.com/item?id=9570639 - but the biggest thing is not isolation, it's portability. You do not care what you're running on, just that you have a certain amount of disk, CPU and network available to you, and the rest you handle yourself. That gives your organization a crazy amount of flexibility in moving that ball of stuff around.

Re: Interview with Eric Brewer

#128

Post by TheIronYuppie »

andyidsinga wrote: post #90

imo, if containers are the future ( very plausible ), then, things like Aws lambda are just as plausible if only just a bit further out. I think this is the case due to granularity of workloads and what apears to be a continuum in the workload container from metal > vm > containers > lambdas (as first class workloads). fun stuff

Disclaimer: I work at Google on Kubernetes

True, lambda is DEFINITELY an advance, but it's more like the salmon at a buffet that includes steak and chicken. Some applications will need just the ability to run code (lambda), some will need defined environments (containers) and some will need total isolation (VMs/bare metal). You'll see a mix of all of these in every mature environment - some things do not fit. For example, it's super unlikely that you'd be able to run a trading app on Lamdba (needs 10GB/sec of direct network access & memory); similarly it'd be totally unnecessary to run a thumbnail processor on bare metal (though you could, of course).

Re: Interview with Eric Brewer

#129

Post by nickpsecurity »

otterley wrote: post #79
nickpsecurity wrote: post #49

One thing that bothers me about the article is that it shows a recurring problem: IT not knowing what it knows. The NoSQL movement didn't notice that NonStop Architecture scaled linearly to thousands of cores with strong-consistency, five 9's, and SQL support. In the mid-80's. Instead of making a low-cost knockoff, like cluster movement did for NUMA's, they ditched consistency altogether and launched NoSQL movement.…

> NonStop Architecture scaled linearly to thousands of cores with strong-consistency, five 9's, and SQL support. Admittedly so, but at a very, very high price. Similarly, Sun and SGI had amazing technology in the server and workstation space (after Solaris 2.3, anyway), but over time Linux became "good enough" and we became willing to sacrifice Sun's niceties to save millions per data center. The mere existence of te…

True. I always expect this. Yet, Google doesn't fit that profile because they weren't just buying something that works: they were investing countless sums into geniuses and their projects trying to invent new things. It's at that point I expect companies to learn from the past and do better.

Re: Interview with Eric Brewer

#130

Post by otterley »

nickpsecurity wrote: post #129
otterley wrote: post #79

Earlier quoted context omitted.

> NonStop Architecture scaled linearly to thousands of cores with strong-consistency, five 9's, and SQL support. Admittedly so, but at a very, very high price. Similarly, Sun and SGI had amazing technology in the server and workstation space (after Solaris 2.3, anyway), but over time Linux became "good enough" and we became willing to sacrifice Sun's niceties to save millions per data center. The mere existence of te…

True. I always expect this. Yet, Google doesn't fit that profile because they weren't just buying something that works: they were investing countless sums into geniuses and their projects trying to invent new things. It's at that point I expect companies to learn from the past and do better.

NonStop and its ilk involve complex hardware engineering, not merely software. But software is Google's forte, not hardware. I wouldn't expect Google to reinvent legacy HA systems, if only because engineering HA systems like those isn't their business.
Post reply on HN