Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ethernet History Deepdive – Why Do We Have Different Frame Types?

lostintransit.se

Post reply on HN
Print view View original

51–60 of 86 posts

Re: Ethernet History Deepdive – Why Do We Have Different Frame Types?

#51

Post by zamadatix »

supahfly_remix wrote: post #6
dale_glass wrote: post #4

I wish we could have another and bump the packet size. We're at the point where we can have millions of packets per second going through a network interface, and it starts to get very silly. It's at the point where even a 10G connection requires some thought to actually perform properly. I've managed to get bottlenecked on high end hardware requiring a whole detour into SR-IOV just to get back to decent speeds.

> I wish we could have another and bump the packet size. The clock precision (100s of ppm) of the NIC oscillators on either side of a network connection gives a physical upper limit on the Ethernet packet size. The space between the packets lets the slower side "catch up". See https://en.wikipedia.org/wiki/Interpacket_gap for more info. We could use more precise oscillators to have longer packets but at a more expens…

Realistically I think we would be fine to make packet size significantly larger than Ethernet would currently allow if we really wanted. E.g. Infiniband already has 1x lane speeds of 200 Gbps without relying on any interpacket gap for clock sync at all. Ethernet, on the other hand, has been consistently increasing speed while decreasing the number of bits used for the interpacket gap since it's less and less relevant for clocking. Put a few bytes back and you could probably do enormous sizes.

Re: Ethernet History Deepdive – Why Do We Have Different Frame Types?

#52

Post by p_l »

Hikikomori wrote: post #44
p_l wrote: post #33

Earlier quoted context omitted.

It was still widely used for IPv4 and v6 in 2000s. ex telco engineer told us that two major reasons was that IS-IS was simply more efficient, and that they didn't require IP communication between routers - meaning troubleshooting and management was easier than providing p2p routes for IP-based protocols.

It's still widely used, among old ISPs because why switch when it's just as good or better than the alternative and your engineers know it best. But it's not the whole stack as implied in the first reply. It was popular primarily because OSPF didn't scale well with hundreds to thousands of routers with the minimal CPU power even large core routers had.

Scalability was one part. Later the fact that you didn't need separate routing setup for v6 also was part of it.

But for a telco, the fact that you didn't have to setup IP connectivity in order for routers to see each other, was also a crucial ability.

Re: Ethernet History Deepdive – Why Do We Have Different Frame Types?

#53

Post by zamadatix »

akira2501 wrote: post #31
kstrauser wrote: post #28

Earlier quoted context omitted.

The common advice I've heard for jumbo frames is not to enable them unless you can do it for every devices on your LAN, and even then it's probably not worthwhile outside specific situations like a separate iSCSI network or such. I just now ran iperf3 from my Mac to my Synology without jumbo frames: [ ID] Interval Transfer Bitrate Retr [ 7] 0.00-10.00 sec 10.0 GBytes 8.61 Gbits/sec 0 sender [ 7] 0.00-10.00 sec 10.0 G…

It seems much more effective on plain 1000Base. It's the difference between 850Mb/s and 975Mb/s for me.

That difference is due to your devices relatively poor at handling high pps workloads and not really related to the particular speed it's running at. You can certainly get more than 850 mbps on a 1000BASE-T link with standard MTU, on the order of ~100 mbps more.

Re: Ethernet History Deepdive – Why Do We Have Different Frame Types?

#54

Post by p_l »

ninkendo wrote: post #46
londons_explore wrote: post #36

I wish layer 2 and layer 3 were 'refactored' to force all links to be point to point, which they effectively are in the modern world. When was the last time you saw ethernet frame collisions because you used a hub not a switch? We'd get rid of the idea of a broadcast domain. We'd get rid of Mac address and ARP. Switches and routers would become the same device. We'd just use ip addresses for routing, and the 'next ho…

A really interesting article covering this: “The world in which IPv6 was a good design” https://apenwarr.ca/log/20170810 It talks about how when IPv6 was being designed, they wanted to do exactly that: drop most of the layer 2 stuff, abandon the idea of a bus network, make everything point-to-point, all switches would be L3 routers, etc. Search for “What if instead the world worked like this?” for the relevant part.…

For v6-specific world, scoped addresses and scoped multicast are explicitly for that purpose. You do not need to hierarchically subnet each following router, you just need to be able to express "next hop" for the subnets you need to route towards.

You use link-local autoconfiguration, and use appropriately-scoped multicast addresses to ask "all-nodes" or "all-routers", making autoconfiguration a breeze compared to v4 world. In v4 world a similar setup is also possible, though specific details of the setup differ, and you have to setup addresses manually for each p2p link.

Re: Ethernet History Deepdive – Why Do We Have Different Frame Types?

#55

Post by zamadatix »

toast0 wrote: post #17
scifi wrote: post #10

Earlier quoted context omitted.

I'm not certain what my point is, but I wanted to mention that jumbo frames don't work over the Internet. More of a LAN thing.

My local internet exchange has a 1500 vlan and a 9000 vlan. My understanding is there are many fewer peers on the 9000 vlan, but it's not zero. If you want to use jumbo packets on the internet at large, you need to have working path MTU detection, which realistically means at least probing, but you really should have that at 1500 too, because there's still plenty of broken networks out there. My guess is you won't ha…

Separate peering VLANs for those using 1500 byte peering and 9000 byte peering about sums up how much of a PITA it is to mix things and expect PMTUD to work.

I'd be willing to bet my lunch more 10x more places have been moving down to assuming 1280 byte connections (since IPv6 guarantees it) than have been peering on the internet at >1500 (not counting 1504 for VLAN tags and whathaveyou).

Re: Ethernet History Deepdive – Why Do We Have Different Frame Types?

#56

Post by [deleted] »

namibj wrote: post #16
monocasa wrote: post #12

Earlier quoted context omitted.

You don't need that as much on modern protocols. The point of 8b/10b or 64b/66b is that it guarantees enough edges to allow receivers to be self clocking from the incoming bits being more or less thrown directly into a PLL.

That's a separate concern. The previously mentioned issue is that to never buffer packets in a reclocking repeater on a link, you _need_ the incoming packet rate to never be higher than the rate at which you can send them back out, or else you'd fill up/buffer. If your repeaters are actually switches, this manifests as whether you occasionally drop packets on a full link with uncongested switching fabric. Think two s…

[deleted]

Re: Ethernet History Deepdive – Why Do We Have Different Frame Types?

#57

Post by rjsw »

fanf2 wrote: post #29
rjsw wrote: post #13

Earlier quoted context omitted.

OSI was usable as a wide area network before TCP/IP was.

[citation needed]

Personal experience.

Was developing OSI applications in 1989, you could order an X.25 circuit from your PTT and run OSI over that using the ISODE [1] toolkit.

The earliest ISP in my country didn't start until 1992.

[1] https://en.wikipedia.org/wiki/ISO_Development_Environment

Re: Ethernet History Deepdive – Why Do We Have Different Frame Types?

#58

Post by ninkendo »

p_l wrote: post #54
ninkendo wrote: post #46

Earlier quoted context omitted.

A really interesting article covering this: “The world in which IPv6 was a good design” https://apenwarr.ca/log/20170810 It talks about how when IPv6 was being designed, they wanted to do exactly that: drop most of the layer 2 stuff, abandon the idea of a bus network, make everything point-to-point, all switches would be L3 routers, etc. Search for “What if instead the world worked like this?” for the relevant part.…

For v6-specific world, scoped addresses and scoped multicast are explicitly for that purpose. You do not need to hierarchically subnet each following router, you just need to be able to express "next hop" for the subnets you need to route towards. You use link-local autoconfiguration, and use appropriately-scoped multicast addresses to ask "all-nodes" or "all-routers", making autoconfiguration a breeze compared to v4…

I mean sure, you’d definitely use a scoped address to talk to the next hop, but it still doesn’t solve how the router/switch knows which port to send the packet to for its next destination.

Say I have a global unicast address on my desktop, 2 hops from my router, and I want to allow traffic to it. My router gets a packet sent to it over its link-local address, with a destination header of my desktop’s IP. Say it has 4 ports (each going to another router/switch, each with its own link-local address.)

How would it know which port to use as the next hop? It would need a routing table, and that would need to be configured automatically if we want to work as well as switches do today. What would be the protocol for this auto configuration? BGP or something like it? How do the routers know the available address space? Or are we just stipulating that we’d invent a protocol for this, if it had ever happened?

In ethernet we have the Spanning Tree Protocol for this, to discover the topology of an Ethernet network and know which links to use for which MAC addresses (including the ability to detect cycles.) I feel like something like spanning tree would still need to exist in an all-unicast, no-ethernet, L3-only world. Does such a thing exist already, or would we need to invent it in this counterfactual universe?

Re: Ethernet History Deepdive – Why Do We Have Different Frame Types?

#59

Post by fsckboy »

FuriouslyAdrift wrote: post #41
londons_explore wrote: post #36

I wish layer 2 and layer 3 were 'refactored' to force all links to be point to point, which they effectively are in the modern world. When was the last time you saw ethernet frame collisions because you used a hub not a switch? We'd get rid of the idea of a broadcast domain. We'd get rid of Mac address and ARP. Switches and routers would become the same device. We'd just use ip addresses for routing, and the 'next ho…

All wifi is a giant collision domain. Also, each segment of a wired network is a collision domain. What you are describing is more in line with MPLS or Infiniband. I agree with you frustration. I prefer to design networks that start routing right at the access port or even using an agent, virtual network port, or VPN endpoint at the client or application (like QUIC), but that is very expensive from a resource standpo…

>Also, each segment of a wired network is a collision domain

huh? where "segment" means where you are using a hub not a switch? cuz that was a long time ago

Re: Ethernet History Deepdive – Why Do We Have Different Frame Types?

#60

Post by db48x »

londons_explore wrote: post #36

I wish layer 2 and layer 3 were 'refactored' to force all links to be point to point, which they effectively are in the modern world. When was the last time you saw ethernet frame collisions because you used a hub not a switch? We'd get rid of the idea of a broadcast domain. We'd get rid of Mac address and ARP. Switches and routers would become the same device. We'd just use ip addresses for routing, and the 'next ho…

Actually, I know of one large system that heavily relies on having racks and racks of servers all located in the same broadcast domain. It makes the networking a bit more complicated, but in turn the software is a lot less complicated. It’s a decent trade–off.
Post reply on HN