IPv6 address space layout best practices

We've rolled out about 12 million nodes on IPv6, over 300 customer /48 networks, and about 6,000 individual /64 subnetworks. 99% of those have been RFC 4193 Unique Local IPv6 Unicast addresses.

IPv6 is like a breath of fresh air when it comes to setting up address spaces. For the past 10 years as a network engineer, three overriding issues have existed when setting up new networks. (1) Begging ARIN for a /19 or /20 when setting up a major new network, (2) Constantly having to have a CIDR calculator running in my head when trying to figure out where the usable hosts/broadcast/network/gw live for the various /29s, /28s, /27s/ /26s, (3) Dreading having to communicate with a new partner or customer also in RFC 1918 space (because of the odds of an address conflict), and/or adding NATing so that we can do a hand off in public space.

With IPV6 - all of those issues are gone. Every subnetwork is always a /64 - CIDR is basically gone. Also, because RFC 4193 is so large (FC::/7, though, for local assignment, it's always FD::/8) - we for all practical purposes never have to worry about negotiating with partners.

Our "Addressing Strategy" is as follows:

For a new customer, Randomly generate, following the guidance of RFC 4193, a /48. Then, for all of their subnetworks, just increment by one. I.E. A customer might get FDC2:D343:107F::/48, and their first subnetwork in that range would be, FDC2:D343:107F:1::/64

For our own IPv6 space, we occasionally play stupid aggregation games, just to keep our routing tables a little more dense - aggregate on ::/56 boundaries - I.E. If we have a new data center with a bunch of subnetworks in our FD1B:D2C5:6AB3::/48 space, we might assign it to be FD1B:D2C5:6AB3:0100::/56, and then we have 255 subnetworks, FD1B:D2C5:6AB3:0100::/64 - FD1B:D2C5:6AB3:01FF::/64) that we work with.

About the only problem with IPv6, is that they make it difficult for you to get your own provider independent globally routable address space that you can advertise out via BGP. So, for instance, a company can't just (easily) get some address space and advertise it out through the internet - the powers that be want you to get a subnetwork out of your ISPs /32. As a result, we ended up just doing everything in RFC 4193, and, using a relative handful of "globally routable" IPv6 hosts for communicating on the internet from our ISPs /32 space. Our ISP gave us a globally routable /48, and we use about 12 hosts addresses from its possible 2^80 possible addresses. Talk about sparse utilization!

Eventually, as more IPv6 hosts land on the internet, and we want to communicate to them from inside our network, we'll probably just add a IPv6-to-IPv6 Network Prefix Translation (NPTv6), or, more likely, an IPv6 NAT system.

NAT has become so embedded in the Enterprise, and provider independent globally routable addresses assigned to companies so difficult to get, that, the globally routable space will ironically be used for fewer IPv6 hosts in many cases, than it was for IPv4 hosts in many cases.

I realize that this is very controversial, and contrary to what every IPv6 IETF network designer/architect/RFC writer wants to hear, but, in talking with the dozen or so colleagues that I have who are also deploying IPv6, it's seems to be the consensus. NAT has been used for so long, by so many network engineers, that non-routability internal space has become our "goto" strategy for the second layer of defense. (With stateful firewalls, of course, always, always being the first layer)

Indeed, when we tried to move a bunch of our applications to Amazon, they indicated that they were only supporting IPv6 for limited interaction on the internet, and that they weren't supporting IPv6 for their internal networks - really embedding the Proxy/NAT concept for AWS applications. Which is a shame, because we'd love to move a lot of our IPv6 applications into AWS.

There are certain RFCs that become, "Common Parlance". In IPv4, RFC 1918 is very well known to be the 10/8, 172.16/12, and 192.168/16 private IP space. Likewise, in IPv6, RFC 4193 is the new "RFC 1918" - and, in common conversation, usually rolls off the (at least my tongue) more easily than, "Unique Local IPv6 Unicast Addresses" - which, often gets confused with (long deprecated) "Site-local unicast" addresses (FECO::/10, RFC 3513) anyways. By calling out the particularly RFC, we eliminate all the confusion, and save a lot of space.

I really did try my best to keep the number soup down (Notice I didn't call out RFC 6296 when mentioning IPv6-to-IPv6 Network Prefix Translation)- I only used two, (And also used the name when I introduced RFC 4193) - but RFC 4193 is so important, that I hope every IPv6 network engineer has it memorized as much as RFC 1918 is to every IPv4 network engineer.

With a technical audience (Which HN has its share of), though - Use of the RFC numbers is critical, as they point to specific implementations of the text description, and may contain nuance, or, indeed, very different implementation constraints, than the name alone suggests.

I'm certain there are many companies that are doing precisely that - but, your introduction sentence was important, "Owned three /16s" - your company had already developed practices and procedures that assumed everyone had a globally routable IP address, and so it was much more likely that when they transitioned to IPv6, that they would likewise use globally routable IPv6 addresses. I bet there is a good chance that they went to the effort of getting their own provider independent IPv6 space - which eliminates a lot (all?) of the hassle associated with using a /48 chunk from their ISPs /32. I got started late in the game (1998), so I've never worked for a company that had a lot of globally routable space (a /18 at most). We always started out inside in RFC 1918, and saved our globally routable space for internet-facing hosts.

The nice things about RFC 4193, is that while the routes aren't floating around on the internet, the addresses you assign are highly, highly unlikely to conflict with someone else's network, and, in the very rare situation that they do during a merger/acquisition - you are no worse off than if you had changed ISPs and had to renumber your internal network anyways. Per RFC 4193 (Section 3.2.3), if you connected with 100 other networks, the probability of collision is 4.54*10^-09 - so the vast majority of the time you won't have a renumbering exercise to worry about.

Note, though - I'm not suggesting that anybody else adopt this strategy of using RFC 4193, and I actually wish more of my colleagues would start using globally routable IPv6 for their internal networks - particularly so I can learn from them to find out what they do when they change ISPs.

Fascinating... it seems easy enough, basically in that address space is abundant enough to not have to be concerned with much of anything in a local space.

But it reads juuuuust slightly more complex than IPv4 does... it's just that side of difficult to comprehend. I think this is the true human factor behind the lack of adoption.

It's huge, like looking at all the stars in the sky, but bigger. Just the switch from numbers between 1-255 to hex codes and the more of them... it's that much more of a barrier to use and adoption.

Any ideas to simplify my own understanding of IPv6? I took the HE.net certification courses, which are excellent, but it's hard to make it stick. How do we make this intelligible so that everyone can be comfortable using it—and ideally, more comfortable than IPv4, so that it becomes preferable? That's the challenge.

I think it is just the prejudice about how "easy" things that we already know seem. But I do remember, and have seen many people, struggling to get a grasp of IPv4 subnetting, routing, security, etc. But you practice and over time it becomes second nature. If you put in half the amount of effort into IPv6 that you did when you learned IPv4 you'd probably be set.