Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

New Optical Form Factors for 400 Gigabit Ethernet

lightwaveonline.com

Post reply on HN
Print view View original

61–70 of 78 posts

Re: New Optical Form Factors for 400 Gigabit Ethernet

#61

Post by kijiki »

walrus01 wrote: post #3
bjoli wrote: post #2

Does aanyone here work with consuming (not just passing it kn) these amounts of traffic? How do you manage to get anything meaningful out of it?

network engineer here: nobody connects a 200 or 400GbE interface linecard off a router to a single anything... The purpose is for ISP-to-ISP interconnections for truly huge amounts of traffic. For example direct peering in the same IX/carrier hotel between a huge source of content (youtube/google, netflix, etc) and a huge ISP (comcast, centurylink). For a big ISP, also things like 100GbE connections from a city's pai…

Cumulus (disclaimer: I co-founded it) has customers that use these: http://www.mellanox.com/page/products_dyn?product_family=260...

to dual 100G attach their servers to a pair of ToR switches. As a result, they'd really love to have something faster than 100G for their ToR to spine uplinks, but that isn't quite available yet.

Insanely (and awesomely), Mellanox already has these: http://www.mellanox.com/page/products_dyn?product_family=266... dual 200G NIC.

Re: New Optical Form Factors for 400 Gigabit Ethernet

#62

Post by pixl97 »

jacquesm wrote: post #42
Osiris wrote: post #11

I wonder when we will start seeing > 1Gpbs Ethernet in consumer products. I've read about initiatives for 5 or 10Gb consumer Ethernet. I'd love to be able to push/pull from my NAS at > 100MB/s.

> I'd love to be able to push/pull from my NAS at > 100MB/s. At a guess you are not a typical consumer. 10G will happen for the consumer, but 100G might never happen for the consumer. And I suspect it will be a while (quite a few years) before 10G will become commonplace enough that economies of scale kick in making it affordable.

Which kind of sucks now that single SSD's are commonly capable of speeds at multiple Gb/s.

One of the big problems with 10G is heat. Common 10G copper cards use 10+ watts and have considerable heat dissipation.

Re: New Optical Form Factors for 400 Gigabit Ethernet

#63

Post by vardump »

pixl97 wrote: post #62
jacquesm wrote: post #42

Earlier quoted context omitted.

> I'd love to be able to push/pull from my NAS at > 100MB/s. At a guess you are not a typical consumer. 10G will happen for the consumer, but 100G might never happen for the consumer. And I suspect it will be a while (quite a few years) before 10G will become commonplace enough that economies of scale kick in making it affordable.

Which kind of sucks now that single SSD's are commonly capable of speeds at multiple Gb/s. One of the big problems with 10G is heat. Common 10G copper cards use 10+ watts and have considerable heat dissipation.

> Which kind of sucks now that single SSD's are commonly capable of speeds at multiple Gb/s.

Single SSDs are often capable of multiple (low) tens of Gb/s. Like Samsung 960 Evo, about 25 Gbps reads (== 3.2 GB/s).

Re: New Optical Form Factors for 400 Gigabit Ethernet

#64

Post by agoodthrowaway »

djrogers wrote: post #40
francoisLabonte wrote: post #24

The real debate is between OSPF and QSFP-DD as they are the only ones with a shot at high density. QSFP-DD is backwards compatible so that you can run your 400G port as a 100G or 40G with 4 lanes at 25G/10G. OSFP will be able to do the same thing but with an adapter from OSFP to QSFP But what this article fails to mention is that OSFP is the connector of the future that will support 8 lanes of 100G to support 800G et…

I get the attraction of OSFP, but in early days backward compatibility usually trumps future potential. If I'm buying an high density switch/router tomorrow (figuratively speaking), I'd greatly prefer the flexibility to use the density it provides for some existing lower speed connections. As long as QSFP-DD 'works', as a customer I'm not really concerned with which technology is more elegant or technically 'better'.…

I see any criticism of OSFP is getting downvoted but your points are correct. OSFP is not needed for 400GE.

Re: New Optical Form Factors for 400 Gigabit Ethernet

#65

Post by mmmBacon »

francoisLabonte wrote: post #24

The real debate is between OSPF and QSFP-DD as they are the only ones with a shot at high density. QSFP-DD is backwards compatible so that you can run your 400G port as a 100G or 40G with 4 lanes at 25G/10G. OSFP will be able to do the same thing but with an adapter from OSFP to QSFP But what this article fails to mention is that OSFP is the connector of the future that will support 8 lanes of 100G to support 800G et…

There is no real debate between QSFP-DD and OSFP. OSFP will exist at Google but nearly everyone else is going to use QSFP-DD. The main reason is backwards compatibility for data center upgrades. The only reason to use OSFP might be because you want to use a 400G-ZR plug. But with the advent of 7nm DSP, vendors will be able to fit the ZR into the DD plug.

So if the above is true OSFP is not needed now.

Honestly, 800G is only incremental at best and frankly it’s not clear that An 800G pluggable is the way to go. The 800G presumes 32 radix and that breaks down power-wise in the datacenter when we get to spine switches where we need 128 radix. The reason being that we need so many power hungry chips to make a high radix switch fully non-blocking (eg Clos topology). 112G SERDES will have limited reach and hence require many retimers which eat power too. So the OSFP is in my view is A solution looking for a problem.

Re: New Optical Form Factors for 400 Gigabit Ethernet

#66

Post by Lorin »

hesdeadjim wrote: post #15
Klinky wrote: post #8

Earlier quoted context omitted.

Multiplayer gaming has more issues with latency than a lack of raw bandwidth.

Not necessarily, depending on how the game is implemented a player with bad ping can be the only one who suffers the majority of the effects of their bad connection. I am currently working with this tech: https://www.photonengine.com/en-US/Quantum And it does deliver on their promises, impressive stuff.

This looks neat, think it'd do well with upcoming projects like Star Citizen from CIG? Physics sims have to be limited dramatically atm.

Re: New Optical Form Factors for 400 Gigabit Ethernet

#67

Post by walrus01 »

wmf wrote: post #48
walrus01 wrote: post #36

Earlier quoted context omitted.

for long reach, yes, but not for ISP-to-ISP interconnection... the cost of coherent (QPSK, 8PSK, 16QAM) 100G SerDes and modulation/demodulation is considerably higher and only justified if going a great many km. For intra-peering-facility connections I don't see coherent 100G (or multiples of coherent 100G bundled by CWDM into 200G, 400G interfaces) becoming the norm anytime soon.

I was looking at this the other day; it sounds like 100G-DR1 and 400G-DR4 are not coherent and vendors are aiming to make it commodity. Coherent is more like 200-600G per lambda.

At present 100G-LR4 is the defacto standard for ISP peering interconnections (such as to an IX switch in the same building, or for PNIs between big players within the same carrier hotel). Because it is very low cost. The cheap 25 Gbps x 4 approach and low cost cwdm prism are a big part of that. Since 100G will soon move from qsfp format down to the same size as SFP and SFP+ (fitting 48 in a 1RU height, 17.5" wide line card in a chassis based Arista), I don't foresee them becoming any less popular... Not when the optics are so affordable.

Re: New Optical Form Factors for 400 Gigabit Ethernet

#68

Post by walrus01 »

kijiki wrote: post #61
walrus01 wrote: post #3

Earlier quoted context omitted.

network engineer here: nobody connects a 200 or 400GbE interface linecard off a router to a single anything... The purpose is for ISP-to-ISP interconnections for truly huge amounts of traffic. For example direct peering in the same IX/carrier hotel between a huge source of content (youtube/google, netflix, etc) and a huge ISP (comcast, centurylink). For a big ISP, also things like 100GbE connections from a city's pai…

Cumulus (disclaimer: I co-founded it) has customers that use these: http://www.mellanox.com/page/products_dyn?product_family=260... to dual 100G attach their servers to a pair of ToR switches. As a result, they'd really love to have something faster than 100G for their ToR to spine uplinks, but that isn't quite available yet. Insanely (and awesomely), Mellanox already has these: http://www.mellanox.com/page/products_…

100G NICs and dual port of the same version are a really good argument for the need for more pci-express 3.0 lanes in single socket servers. The AMD epyc is a step in the right direction. If you assume a server that might have two m.2 nvme SSD in it, those take up lanes, and then the nic will eat an x16 interface.

Re: New Optical Form Factors for 400 Gigabit Ethernet

#69

Post by walrus01 »

fulafel wrote: post #39
bjoli wrote: post #2

Does aanyone here work with consuming (not just passing it kn) these amounts of traffic? How do you manage to get anything meaningful out of it?

5G base stations cite 50 Gbps bandwidth, connect a bunch of these together in an urban area and you might want 400G for their uplink/backbone/whatever it's called.

More realistically as cellular carriers are upgrading rooftop sites to the latest LTE gear, what I'm seeing as a carrier, they're fed by either a dedicated 1G fiber transport back to an aggregation site, the same at 10G on fiber, or point to point 60GHz (1-5 Gbps) mmwave, or 71-86GHz band ptp mmwave (1 to 10Gbps).

Re: New Optical Form Factors for 400 Gigabit Ethernet

#70

Post by fulafel »

walrus01 wrote: post #69
fulafel wrote: post #39

Earlier quoted context omitted.

5G base stations cite 50 Gbps bandwidth, connect a bunch of these together in an urban area and you might want 400G for their uplink/backbone/whatever it's called.

More realistically as cellular carriers are upgrading rooftop sites to the latest LTE gear, what I'm seeing as a carrier, they're fed by either a dedicated 1G fiber transport back to an aggregation site, the same at 10G on fiber, or point to point 60GHz (1-5 Gbps) mmwave, or 71-86GHz band ptp mmwave (1 to 10Gbps).

It sure sounds like 5G deployment is "realistic". Big operators have announced deployments & limited availability this year.
Post reply on HN