Live data from Hacker News

Migrating From AWS to FB

instagram-engineering.tumblr.com

21–30 of 94 posts

Re: Migrating From AWS to FB

#21
post #17

"Facebook’s private IP space conflicts with that of EC2" ^ That wouldn't've happened in GCE (i.e., they should have been acquired by Google).

Are you sure? This document suggests GCE instances use the 10.x.x.x address space (just as AWS instances in EC2 Classic do):

https://developers.google.com/compute/docs/instances-and-net...

Re: Migrating From AWS to FB

#22
post #18

Earlier quoted context omitted.

Agree in principle, but it's unlikely for Amazon to ever force customers to allocate addresses from their own IPv6 block to begin with. Like many other startups, Instagram started with the bare-minimum setup outside of VPC, and scaled from there, not even knowing the benefits of VPC before it was too late for a simple cutover.

Amazon doesn't have to force customers to use their own IPv6 blocks. EC2 instances, whether inside or outside VPC, would be assigned unique IPv6 addresses from Amazon's address space (which would be extensive). If this were the norm when Instagram started out, it would have been just as easy to use, and there would have been no clash of addresses when migrating to Facebook.

Are you saying that Amazon would divide up it's own IPv6 address space and provide a subnet for each customer? That's really the only way it would work for our situation. I'm not sold that Amazon would actually have any motivation to go through the trouble to provide this as it would probably only impact 1% of their customers at best. The most rational path for them is to just assign addresses from a big pool and not bother with the all the fancy subnetting unless the customer asks for it (VPC).

Re: Migrating From AWS to FB

#23
post #18

Earlier quoted context omitted.

Amazon doesn't have to force customers to use their own IPv6 blocks. EC2 instances, whether inside or outside VPC, would be assigned unique IPv6 addresses from Amazon's address space (which would be extensive). If this were the norm when Instagram started out, it would have been just as easy to use, and there would have been no clash of addresses when migrating to Facebook.

Are you saying that Amazon would divide up it's own IPv6 address space and provide a subnet for each customer? That's really the only way it would work for our situation. I'm not sold that Amazon would actually have any motivation to go through the trouble to provide this as it would probably only impact 1% of their customers at best. The most rational path for them is to just assign addresses from a big pool and not…

>The most rational path for them is to just assign addresses from a big pool and not bother with the all the fancy subnetting.

That's what he's suggesting, though (I think). Except because it's IPv6, Amazon's big pool of addresses would never conflict with Facebook's big pool of addresses.

Re: Migrating From AWS to FB

#24
post #23

Earlier quoted context omitted.

Are you saying that Amazon would divide up it's own IPv6 address space and provide a subnet for each customer? That's really the only way it would work for our situation. I'm not sold that Amazon would actually have any motivation to go through the trouble to provide this as it would probably only impact 1% of their customers at best. The most rational path for them is to just assign addresses from a big pool and not…

>The most rational path for them is to just assign addresses from a big pool and not bother with the all the fancy subnetting. That's what he's suggesting, though (I think). Except because it's IPv6, Amazon's big pool of addresses would never conflict with Facebook's big pool of addresses.

Exactly.

Re: Migrating From AWS to FB

#25
post #19

Called it, five years ago: http://www.web2expo.com/webexny2009/public/schedule/detail/9... Run in multiple clouds from day one. Take the pain. It gives you flexibility. Basic vendor management 101.

Maybe it's worth the pain if every startup became the size of Instagram and got bought by Facebook, but that's only true for a tiny, tiny fraction of a % of all startups.

Re: Migrating From AWS to FB

#26
These engineering feats are truly impressive and worth writing about.

And yet every time I read about this kind of stuff I think, how glad am I that we are building a DISTRIBUTED social network and will never have to solve problems on this massive scale! We won't have to move millions of other people's photos here or there if everything is distributed from day 1. People will be able to move their own stuff easily wherever they want.

Re: Migrating From AWS to FB

#28
post #11

Earlier quoted context omitted.

Facebook supports ipv6, but maybe it is external only? I know Google is ipv6 internally too. Although as EC2 has no ipv6 maybe that was the blocking issue.

Facebook makes extensive use of IPv6 internally. As you suggested, EC2's lack of support is the issue. Edit: here are the slides from Facebook's presentation to the IPv6 World Congress in March about their internal IPv6 use. If IPv6 interests you, they're definitely worth a read: http://www.internetsociety.org/deploy360/wp-content/uploads/...

Thumbs up for the interesting read. It's amazing how Amazon doesn't support IPv6 yet, though

Re: Migrating From AWS to FB

#29
post #9
post #8

Earlier quoted context omitted.

I think it has more to do with the fact that there are a standard series of private IP blocks.

Right, but if everyone used IPv6 there would be no need to use non-routeable private IPs for anything, you could just use non-conflicting IPv6 addresses and not route them.

I kind of like having standard private subnets. My router is always 192.168.1.1, or sometimes 10.0.1.1, and so is my friend's, my parent's, and my grandparent's.

Re: Migrating From AWS to FB

#30
post #9

Earlier quoted context omitted.

Right, but if everyone used IPv6 there would be no need to use non-routeable private IPs for anything, you could just use non-conflicting IPv6 addresses and not route them.

I kind of like having standard private subnets. My router is always 192.168.1.1, or sometimes 10.0.1.1, and so is my friend's, my parent's, and my grandparent's.

You can have that in IPv6 too, with link-local addresses (fe80::/64). In fact, it's becoming quite common to assign fe80::1 to a subnet's default gateway.
Post reply on HN