Migrating From AWS to FB • Hacker News Board

Re: Migrating From AWS to FB

#71

Post by purephase » Fri, Jun 27, 2014, 11:23 AM UTC

jmccree wrote: post #46

NathanKP wrote: post #34

I wonder why Instagram wasn't using VPC in the first place. I've been using AWS for a startup for a few years now and I had our instances running in VPC from about the second month onward. It's been one of the best architecture decisions I've ever made. At this point we only use one public IP address. (If direct access to a machine is needed then you can connect via VPN running on the one bastion host with the public…

If you assume they had no pressing need for any VPC specific functionality, you can get similar security by locking your security group/s down to only ELB for public service ports and having one instance in another security group with ssh/vpn allowed (to specific ips) as a jump box/vpn. Spending weeks of multiple teams engineering time to move to VPC without a pressing need would seem to me to make little business se…

Agreed. This is the route I use and it works fine. I can see how it could quickly get out of hand with a lot of security groups, and I would love some sort of security group inheritance, but for -100 instances, it is not the hard to keep the public access to ELB.

Re: Migrating From AWS to FB

#72

Post by rb2k_ » Fri, Jun 27, 2014, 11:41 AM UTC

themartorana wrote: post #54

blackaspen wrote: post #45

Earlier quoted context omitted.

Hindsight is 20/20. I think you guys did an exceptional job to tackling a really difficult problem (I've been in the same position, migrating EC2 to Datacenters) and we determined that EC2 -> VPC -> Datacenters is really the only way, and Neti solves it surprisingly well. Going forward, hope that acquired companies opened their AWS accounts late enough that Amazon forced them to use VPC.

We're small, comparatively - 20-30 servers max - and we need to get in to VPC for a new cluster that requires static internal IPs. (Reboot an EC2 Classic instance and you may get a different 10.x address.) In any case, the migration is daunting even at our size, although our devops team size is 1. I do wish they had VPC when we started.

You could also just attach EIPs and use those, right?

Re: Migrating From AWS to FB

#73

Post by ceejayoz » Fri, Jun 27, 2014, 12:16 PM UTC

SushiMon wrote: post #58

I'm wondering if they got nailed by out-migration charges and how much that was. I assume a bunch of their images were in S3. Amazon charges a pretty penny to take things out.

I'm confused by this. S3 GET requests are the cheapest request type, and getting the images out would just cost you the bandwidth involved.

Maybe you're mixing things up with Glacier?

Re: Migrating From AWS to FB

#74

Post by ceejayoz » Fri, Jun 27, 2014, 12:19 PM UTC

sumbry wrote: post #53

serge2k wrote: post #20

Earlier quoted context omitted.

This article mentions nothing about latency.

We're undertaking a similar project and the latency is almost negligible. In fact the latency is lower bridging between classic and vpc in the same availability zone than between two classic availability zones.

That's not multiple clouds, though.

Re: Migrating From AWS to FB

#75

Post by [deleted] » Fri, Jun 27, 2014, 12:34 PM UTC

0x006A wrote: post #2

why not just create a vpn between the nodes with another private IP space and send your data through that?

[deleted]

Re: Migrating From AWS to FB

#76

Post by yeukhon » Fri, Jun 27, 2014, 12:50 PM UTC

Contributors? What contributors? People from within the company or open source contributors?

Re: Migrating From AWS to FB

#77

Post by agwa » Fri, Jun 27, 2014, 2:20 PM UTC

hrktb wrote: post #61

rdl wrote: post #9

Earlier quoted context omitted.

Right, but if everyone used IPv6 there would be no need to use non-routeable private IPs for anything, you could just use non-conflicting IPv6 addresses and not route them.

The range of non-conflicting IPv4 private addresses is not really small. Everyone just happens to use the same two or three blocks, the easiest to remember.

Actually, when you're working at Facebook scale, the range is quite small. One of the reasons Facebook started moving to IPv6 was that they were running out of RFC1918 addresses.

Re: Migrating From AWS to FB

#78

Post by evan_ » Fri, Jun 27, 2014, 3:34 PM UTC

SushiMon wrote: post #58

I'm wondering if they got nailed by out-migration charges and how much that was. I assume a bunch of their images were in S3. Amazon charges a pretty penny to take things out.

They could have used the export service:

http://aws.amazon.com/importexport/

Re: Migrating From AWS to FB

#79

Post by lmm » Fri, Jun 27, 2014, 3:45 PM UTC

sliverstorm wrote: post #29

rdl wrote: post #9

Earlier quoted context omitted.

Right, but if everyone used IPv6 there would be no need to use non-routeable private IPs for anything, you could just use non-conflicting IPv6 addresses and not route them.

I kind of like having standard private subnets. My router is always 192.168.1.1, or sometimes 10.0.1.1, and so is my friend's, my parent's, and my grandparent's.

Which is great until you and your friend want to play games with each other and need to set up a VPN.

Re: Migrating From AWS to FB

#80

Post by [deleted] » Fri, Jun 27, 2014, 3:45 PM UTC

sliverstorm wrote: post #29

rdl wrote: post #9

Earlier quoted context omitted.

Right, but if everyone used IPv6 there would be no need to use non-routeable private IPs for anything, you could just use non-conflicting IPv6 addresses and not route them.

I kind of like having standard private subnets. My router is always 192.168.1.1, or sometimes 10.0.1.1, and so is my friend's, my parent's, and my grandparent's.

[deleted]