Live data from Hacker News

Migrating From AWS to FB

instagram-engineering.tumblr.com

61–70 of 94 posts

Re: Migrating From AWS to FB

#61
post #9
post #8

Earlier quoted context omitted.

I think it has more to do with the fact that there are a standard series of private IP blocks.

Right, but if everyone used IPv6 there would be no need to use non-routeable private IPs for anything, you could just use non-conflicting IPv6 addresses and not route them.

The range of non-conflicting IPv4 private addresses is not really small. Everyone just happens to use the same two or three blocks, the easiest to remember.

Re: Migrating From AWS to FB

#62
post #25
post #19

Called it, five years ago: http://www.web2expo.com/webexny2009/public/schedule/detail/9... Run in multiple clouds from day one. Take the pain. It gives you flexibility. Basic vendor management 101.

Maybe it's worth the pain if every startup became the size of Instagram and got bought by Facebook, but that's only true for a tiny, tiny fraction of a % of all startups.

Yes - truth be told, I am not sure if there's much to take away from this article if you're not really aiming for something huge. And even then, maybe you should spend time on what'll get you traction rather than trying to design an architecture for what you hope to become. Instagram is very much an outlier.

Re: Migrating From AWS to FB

#63
post #9

Earlier quoted context omitted.

Right, but if everyone used IPv6 there would be no need to use non-routeable private IPs for anything, you could just use non-conflicting IPv6 addresses and not route them.

I kind of like having standard private subnets. My router is always 192.168.1.1, or sometimes 10.0.1.1, and so is my friend's, my parent's, and my grandparent's.

After spending some time as a contractor doing systems work for a few companies I've stopped ever assigning an internal network to 192.168.0.0, 192.168.1.0, or 10.0.0.0.

The number of times I found myself attempting to VPN into a clients network, only to find it conflicted either with my home network, or whatever coffee shop I was sitting in, was ridiculous. Depending how many hosts you need to run on your network there are huge numbers of possible subnets you could use for an internal network - do yourself a favour and keep off the ones set up be default on every router sold.

Re: Migrating From AWS to FB

#64
post #9

Earlier quoted context omitted.

Right, but if everyone used IPv6 there would be no need to use non-routeable private IPs for anything, you could just use non-conflicting IPv6 addresses and not route them.

But I can recall ip4 addresses en mass in my head. I can't memorize ip6 addresses easily. Plus, I'd like to keep pockets of private IPs that are never accessible as routable targets. Can you explain why I wouldn't to do this or why I should evolve my understanding of ip6 better?

There's no real need to memorise IP addresses, that's what DNS was made for. If your servers are on the internet at large then they probably have DNS already, and if its a local network then most operating systems will now automatically work out where machines on the .local domain are (I'll be honest, I don't fully understand how that works).

Re: Migrating From AWS to FB

#65

Earlier quoted context omitted.

But I can recall ip4 addresses en mass in my head. I can't memorize ip6 addresses easily. Plus, I'd like to keep pockets of private IPs that are never accessible as routable targets. Can you explain why I wouldn't to do this or why I should evolve my understanding of ip6 better?

There's no real need to memorise IP addresses, that's what DNS was made for. If your servers are on the internet at large then they probably have DNS already, and if its a local network then most operating systems will now automatically work out where machines on the .local domain are (I'll be honest, I don't fully understand how that works).

I think it is a safe assumption that Sam knows what the DNS was made for.

Re: Migrating From AWS to FB

#66
post #9

Earlier quoted context omitted.

Right, but if everyone used IPv6 there would be no need to use non-routeable private IPs for anything, you could just use non-conflicting IPv6 addresses and not route them.

But I can recall ip4 addresses en mass in my head. I can't memorize ip6 addresses easily. Plus, I'd like to keep pockets of private IPs that are never accessible as routable targets. Can you explain why I wouldn't to do this or why I should evolve my understanding of ip6 better?

Why does it matter if an address is public or not? Whether the address is publicly routable has nothing to do with whether the host is accessible... Or don't you have firewalls?

Re: Migrating From AWS to FB

#67

Earlier quoted context omitted.

But I can recall ip4 addresses en mass in my head. I can't memorize ip6 addresses easily. Plus, I'd like to keep pockets of private IPs that are never accessible as routable targets. Can you explain why I wouldn't to do this or why I should evolve my understanding of ip6 better?

There's no real need to memorise IP addresses, that's what DNS was made for. If your servers are on the internet at large then they probably have DNS already, and if its a local network then most operating systems will now automatically work out where machines on the .local domain are (I'll be honest, I don't fully understand how that works).

I've had only a very limited exposure to IPv6, but it seemed to me that the slogan "DNS solves it for you!" doesn't really pan out. It solves it if you're on a well-set-up network and have your DNS up and running happily, but with the ad-hoc networks my [limited] experience has seen, it hasn't been trivial. Essentially, it means you have to run an interpreter service (the DNS) to understand the network - one more bit of software to configure and troubleshoot... though to be fair, IPv4 was also quite confusing when I first started playing with it.

Re: Migrating From AWS to FB

#68
post #18

Earlier quoted context omitted.

Amazon doesn't have to force customers to use their own IPv6 blocks. EC2 instances, whether inside or outside VPC, would be assigned unique IPv6 addresses from Amazon's address space (which would be extensive). If this were the norm when Instagram started out, it would have been just as easy to use, and there would have been no clash of addresses when migrating to Facebook.

Are you saying that Amazon would divide up it's own IPv6 address space and provide a subnet for each customer? That's really the only way it would work for our situation. I'm not sold that Amazon would actually have any motivation to go through the trouble to provide this as it would probably only impact 1% of their customers at best. The most rational path for them is to just assign addresses from a big pool and not…

At AWS's scale, 1% of customers is quite a lot of people and quite a lot of money.

Re: Migrating From AWS to FB

#69

Earlier quoted context omitted.

I kind of like having standard private subnets. My router is always 192.168.1.1, or sometimes 10.0.1.1, and so is my friend's, my parent's, and my grandparent's.

After spending some time as a contractor doing systems work for a few companies I've stopped ever assigning an internal network to 192.168.0.0, 192.168.1.0, or 10.0.0.0. The number of times I found myself attempting to VPN into a clients network, only to find it conflicted either with my home network, or whatever coffee shop I was sitting in, was ridiculous. Depending how many hosts you need to run on your network th…

This is also important advice if you're thinking about setting up VPN access to your home network: Do not pick the most common/default subnets, i.e. 192.168.0.0/24, 192.168.0.1/24, etc. Picking a somewhat-random subnet as suggested would mitigate the problem and it's what I did for my home network.

Re: Migrating From AWS to FB

#70
post #67

Earlier quoted context omitted.

There's no real need to memorise IP addresses, that's what DNS was made for. If your servers are on the internet at large then they probably have DNS already, and if its a local network then most operating systems will now automatically work out where machines on the .local domain are (I'll be honest, I don't fully understand how that works).

I've had only a very limited exposure to IPv6, but it seemed to me that the slogan "DNS solves it for you!" doesn't really pan out. It solves it if you're on a well-set-up network and have your DNS up and running happily, but with the ad-hoc networks my [limited] experience has seen, it hasn't been trivial. Essentially, it means you have to run an interpreter service (the DNS) to understand the network - one more bit…

Won't mdns / zeroconf / avahi / however it's called this week work for ad-hoc networks? It surely does the trick in LAN.

It won't work properly across routers, at least not out of the box (tried that when configuring Tinc VPN), but maybe this would be a good direction?

Post reply on HN