Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Espresso – Google’s peering edge architecture

blog.google

Post reply on HN
Print view View original

81–90 of 105 posts

Re: Espresso – Google’s peering edge architecture

#81

Post by mikecb »

zaroth wrote: post #30
EE84M3i wrote: post #20

Earlier quoted context omitted.

I found this article confusing. It doesn't really say anything about what "Espresso" actually does, let alone how it does it.

For some reason they didn't actually link to the talk, which I haven't watched, but presumably actually tries to start answering those questions. Quick search is showing the 2015 keynote that Amin gave, haven't found the 2017 one yet... [1] - 2015 ONS Keynote https://www.youtube.com/watch?v=FaAZAII2x0w

And here's 2014, which was on the Andromeda NFV stack: https://www.youtube.com/watch?v=n4gOZrUwWmc

And B4: https://www.youtube.com/watch?v=tVNlXg0iN-g

Re: Espresso – Google’s peering edge architecture

#82

Post by dspillett »

puzzle wrote: post #59
walrus01 wrote: post #55

Earlier quoted context omitted.

It's gold plated because they basically built their own ISP by acquiring either: a: dark fiber IRUs between cities/metro areas b: N x 10 and 100 Gbps wavelengths as L2 transport services from city to city, from a major carrier such as level3 or zayo c: some combination of A and B and they use that to build backbone links between their own network equipment that they have full control over. Google is its own AS and op…

Where do you get the crazy idea that Google doesn't run its links to saturation? It's crazy because it would cost an enormous amount of money. The B4 paper states multiple times that Google runs links at almost 100% saturation, versus the standard 30-40%. That's accomplished through the use of SDN technology and, even before that, through strict application of QoS. https://web.stanford.edu/class/cs244/papers/b4-sigco…

No, it would be crazy for them to run things at saturation under normal circumstances as that does not allow at all for abnormal circumstances. The opportunity cost of not using something 100% all the time is offset against the worth of increased stability/predictability in the face of changing conditions.

Though you do need to define "saturation". Are you referring to bulk bandwidth or some other measure of throughput/goodput? Saturating in terms of raw bandwidth can reduce useful throughput due to latency issues.

Re: Espresso – Google’s peering edge architecture

#84

Post by hueving »

These presentations from Google are pretty irritating at these conferences. If you're familiar with the SDN field (as most ONS attendees would be), this presentation is essentially nothing but bragging about the scale at which they operate.

There is no useful information in here to advance the state of the art, no new ideas, no publicly available implementations (closed or open source). It's just a very high-level architectural view of a large network given by people who are incentivized to present it in the most favorable light. And due to the lack of any concrete details, it's free from critical analysis.

>Espresso delivers two key pieces of innovation. First, it allows us to dynamically choose from where to serve individual users based on measurements of how end-to-end network connections are performing in real time. Second, we separate the logic and control of traffic management from the confines of individual router “boxes.”

The first has been done before at many levels of the network:

* BGP anycast

* DNS responses based on querier

* Done in load balancer

* IGP protocols to handle traffic internally while taking into account link congestion

I assume their framework gives them much nicer primitives to work with than the above, which would be an advancement in the field if we could actually see an API or something.

The second is very far from "innovation". This is the essence of SDN and this has been the hottest thing since sliced bread in the networking world since 2008 at a minimum [1] and even earlier if you look at things like the Aruba wireless controller.

1. http://archive.openflow.org/documents/openflow-wp-latest.pdf

Re: Espresso – Google’s peering edge architecture

#85

Post by dingaling »

walrus01 wrote: post #66
puzzle wrote: post #61

Earlier quoted context omitted.

70 metros translate to many more POPs (also for redundancy reasons). The metros can be seen in e.g. the 1e100.net hosts in a traceroute. They're usually the closest airport code, so e.g. lhr for London. Somebody in China reverse engineered the metro/POP naming and addressing for latency reasons. You can see that, for example, there are at least three POPs in Sydney: https://docs.google.com/spreadsheets/d/1a5HI0lkc1Ty…

yes, the practice of using IATA airport codes or similar for reverse DNS is a practice that long pre-dates google. They just adopted industry best practices when they started building consistent rDNS.

It's also short-sighted as the four-character ICAO code system encodes geographical data and encompasses a magnitude more airfields.

For example one can determine that EGLL is in Western Europe, UK without having to look up tables to determine that it is Heathrow.

Re: Espresso – Google’s peering edge architecture

#86

Post by runeks »

hueving wrote: post #84

These presentations from Google are pretty irritating at these conferences. If you're familiar with the SDN field (as most ONS attendees would be), this presentation is essentially nothing but bragging about the scale at which they operate. There is no useful information in here to advance the state of the art, no new ideas, no publicly available implementations (closed or open source). It's just a very high-level ar…

I very much agree. I was hoping Espresso would be a framework for allowing GCP user applications to leverage Google's SDN, rather than just allow Google to offer their own services using this technology. I hope that's the next step.

For example, it would be cool if it were possible to move shared-client/server-secret checking (eg. for an HTTP API) out to the edge of Google's network, such that a DDoS attack with invalid packets (secrets) never even reach the application VM/cluster. DDoS attacks, which force applications offline (by making the app scale up to an unsustainable cost level), could be prevented this way.

Re: Espresso – Google’s peering edge architecture

#87

Post by cobookman »

runeks wrote: post #86
hueving wrote: post #84

These presentations from Google are pretty irritating at these conferences. If you're familiar with the SDN field (as most ONS attendees would be), this presentation is essentially nothing but bragging about the scale at which they operate. There is no useful information in here to advance the state of the art, no new ideas, no publicly available implementations (closed or open source). It's just a very high-level ar…

I very much agree. I was hoping Espresso would be a framework for allowing GCP user applications to leverage Google's SDN, rather than just allow Google to offer their own services using this technology. I hope that's the next step. For example, it would be cool if it were possible to move shared-client/server-secret checking (eg. for an HTTP API) out to the edge of Google's network, such that a DDoS attack with inva…

You can do that by using Google clouds https load balancer.

Re: Espresso – Google’s peering edge architecture

#88

Post by lobster_johnson »

hueving wrote: post #84

These presentations from Google are pretty irritating at these conferences. If you're familiar with the SDN field (as most ONS attendees would be), this presentation is essentially nothing but bragging about the scale at which they operate. There is no useful information in here to advance the state of the art, no new ideas, no publicly available implementations (closed or open source). It's just a very high-level ar…

I don't disagree, but they have published a lot of papers about their networking research and implementation of things like like B4 and their SDN work [1]. Hopefully there's a paper on Espresso forthcoming, although the absence (as far as I can tell) of a paper on Andromeda alone means it might not be.

[1] https://research.google.com/pubs/Networking.html

Re: Espresso – Google’s peering edge architecture

#89

Post by londons_explore »

walrus01 wrote: post #78
insaneirish wrote: post #77

Earlier quoted context omitted.

> Because they are going to be expecting sub-1ms latency to your router, and not "we're taking this session and stuffing it in some sort of tunnel or encapsulation and sending it somewhere else, to where the thing that actually speaks BGP is located". Why should they care? > It's bad juju to practice deceptive peering. I don't understand applying moral judgment to a technical design choice.

I'm guessing you do not handle peering for a medium to large sized AS, so it's really hard to explain. First: they should care because the point of establishing peering in a given city is to give inter-AS traffic the absolute shortest path and shortest number of hops between two points. If I put a router in portland, OR and buy a 10 Gbps mpls tunnel to Vancouver BC and join the vanix, ask to set up with peers there,…

Just because they tunnel the BGP back to some software system to make routing decisions doesn't mean they tunnel all the user data to that same location. To do so would be silly.

In fact, a good system would have a couple of systems handling BGP, with physical location fairly irrelevant, but acting as if they are local to the peer they are talking to.

Post reply on HN