[deleted]
Even if it is under patent protection, Google is unlikely to sue you, unless you sue them first.
[deleted]
Espresso – Google’s peering edge architecture
91–100 of 105 posts
Re: Espresso – Google’s peering edge architecture
#92Re: Espresso – Google’s peering edge architecture
#93Earlier quoted context omitted.
Could you expand a bit more on you comment? I feel I'm missing some context. Specifically, what do you mean by gold plated? Why is it tempting to ignore some aspects of distributed computing? I'm missing a lot of context that you are implicitly implying so could you elaborate?
It's gold plated because they basically built their own ISP by acquiring either: a: dark fiber IRUs between cities/metro areas b: N x 10 and 100 Gbps wavelengths as L2 transport services from city to city, from a major carrier such as level3 or zayo c: some combination of A and B and they use that to build backbone links between their own network equipment that they have full control over. Google is its own AS and op…
[deleted]
Re: Espresso – Google’s peering edge architecture
#94Earlier quoted context omitted.
What I mean is that they do not run their links to saturation in the same way as an ordinary ISP. And because their traffic patterns are very different than an ordinary ISP, and much, much more geographically distributed, they can do all sorts of fun software tricks. The end result is the same: Low/no jitter and no packet loss. As contrasted with what would happen if you had a theoretical hosting operation behind 2 x…
For clarity, do you mean that Google can, for example, run to 99% saturation all the time, whereas a typical ISP might have 30-40% average, with peaks to full saturation that causes high latency/packet loss when it occurs?
Yes, that's about right. Since they control both sides of the link, they can manage the flow from higher up on the [software] stack. Basically, if the link is getting saturated, the distributed system simply throttles some requests upstream by diverting traffic from places that result in traffic over that link. (And of course this requires a very complex control plane, but doable, and with proper [secondary] controls it probably stays understandable, manageable, and doesn't go haywire when shit hits the fan.)
Re: Espresso – Google’s peering edge architecture
#95tl;dr?
Re: Espresso – Google’s peering edge architecture
#96Earlier quoted context omitted.
@puzzle: while you're not wrong, do note that B4 is not (and is not designed to be) a low-latency, low-jitter network. It's designed for massive bandwidth for inter-datacenter data transfer.
running your own internal links to near saturation (such as a theoretical 100 Gbps DWDM or MPLS circuit between two google datacenters in two different states) is a very different thing than running a BGP edge connection to saturation, such as a theoretical 100 Gbps, short reach intra building crossconnect from a huge CDN such as Limelight to a content-sink ISP such as Charter/TWTC or Comcast.
Very much so. B4 can be run near 100% because of strict admission control and optimized routing to maximize the use of all paths. It's much harder to do that on peering links where the traffic is bursty and you don't have control over the end-to-end latency and jitter. SDN isn't a magic pill for this, but it can most definitely lead to better performance and higher utilization than Ye Olde BGP Traffic Engineering.
Re: Espresso – Google’s peering edge architecture
#97Earlier quoted context omitted.
70 metros?
I wonder if they are referring to a peering Internet exchange point (IXP) when they say metro. Basically a building where networks converge and ISPs connect to each other.
A metro in Google-speak generally refers to all of the peering locations in a given city or metro area, not specifically a single IXP.
For example, if you look at the PeeringDB entry for AS15169 (https://www.peeringdb.com/asn/15169), for the London "metro" there's public peering available on LINX at 3 different POPs, and private peering available at Digital Realty, 3 different Equinix POPs, and 2 Telehouse POPs.
Re: Espresso – Google’s peering edge architecture
#98Earlier quoted context omitted.
yes, the practice of using IATA airport codes or similar for reverse DNS is a practice that long pre-dates google. They just adopted industry best practices when they started building consistent rDNS.
It's also short-sighted as the four-character ICAO code system encodes geographical data and encompasses a magnitude more airfields. For example one can determine that EGLL is in Western Europe, UK without having to look up tables to determine that it is Heathrow.
this is why some ISPs use a format that maps to two letter country code, then state or subdivision, then POP. where 01 is the first pop in the city, 02 is second, etc. The ISP's own internal documentation system will tell their NOC and neteng staff which pop number is which street address/datacenter (info that does not need to be available in rDNS). in a totally random example, nyc01 POP for an ISP might be 25 broadway, nyc02 would be 60 hudson, nyc03 would be 111 8th, and so on.
example agg router:
agg1.nyc01.ny.us.ASNUMBER.net
and then individual interfaces and subinterfaces would be defined under hierarchically under agg1.
Re: Espresso – Google’s peering edge architecture
#99Earlier quoted context omitted.
Unfortunately, no - at least not without quite a few more details. As this stands, it could be the high-level marketing overview for pretty much any SDN solution available, commercial or open.
Thanks! As someone who recently entered this field professionally, I find it amusing that most SDN solutions out there are just permutations of each other playing over marketing buzzwords. Not too different from "cloud computing" from a few years ago.
Yeah, once you get past the hype though, SDNs can be great.
Re: Espresso – Google’s peering edge architecture
#100These presentations from Google are pretty irritating at these conferences. If you're familiar with the SDN field (as most ONS attendees would be), this presentation is essentially nothing but bragging about the scale at which they operate. There is no useful information in here to advance the state of the art, no new ideas, no publicly available implementations (closed or open source). It's just a very high-level ar…
Yes. The term "innovation" applied to the two ideas bugged me too.
Could be a 'smart people problem' -- easier to 'innovate' a new wheel then visualize how existing tools can be remixed to solve.
For an example of remixing, it was apparent 15 years ago that traditional routing was actively harmful for live and VOD video streaming, so we cobbled a mix of the techniques you listed plus a couple more[1] to connect users to edges in real time based on actual real-time end-to-end conditions.
We did it consciously using these two ideas, plus content aware caching, plus one more "magic" but super trivial idea I still haven't seen elsewhere in the wild.
The low tech rethink worked well, handily outperformed proprietary solutions per industry perf and SLO metrics companies, and it's been an ongoing surprise me that it's taken so long for others to rethink or remix the same way.
We filed no patents, as to your point, this all could be argued evident to anyone 'skilled in the art'.
---
1. A couple custom bits: we also had to write an edge server shim as no media server at the time could handle sessions flapping in real time, not to mention the content awareness crucial for giant media files.