Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why Loggly Chose AWS Route 53 Over Elastic Load Balancing

loggly.com

Post reply on HN
Print view View original

21–30 of 50 posts

Re: Why Loggly Chose AWS Route 53 Over Elastic Load Balancing

#21

Post by hijinks »

former_loggly wrote: post #17

Former Loggly employee here. Loggly is at CTO #3 or 4 in about 3 years. The CEO, marketing guy with black turtle neck, "runs" engineering. It is NOT an engineering company and they are on their way to outsourcing all development to India. Formally they had all of their EC2 instances configured to run without swap and didn't use EBS such that instances would crash 1-3 times a day and lose all data which would require…

I interviewed there for a devops/sys admin role and ran far away after that process when I learned about what is going on and problems the ops group has to solve.

Then reading this fluff piece made me glad I never even thought about working there after that phone interview. Whoever claims a DNS round robin is a good way to handle fail over doesn't really know what they are talking about. I have dug into the how something like rsyslog handles a dns request. My guess is it just passes it off to the OS.

But what I got from this is loggly is ok with losing customer data

Re: Why Loggly Chose AWS Route 53 Over Elastic Load Balancing

#22

Post by hijinks »

hatred wrote: post #15
korzun wrote: post #14

Earlier quoted context omitted.

Yeah I don't get their approach. There is no way this will cause 100% delivery if one server fails within that rotation. That chances of failure go up dramatically if 2+ hosts behind round robin fail, etc. Not to mention once hosts resolve this to an IP they will re-use the route. This approach is not balanced. I don't want to be /that/ guy but if they can't scale with ELB they should invest in a dedicated load balan…

I am more worried about the DNS caching issue. The 2+ hosts failing should not be much of a problem if you have a separate health checker host which does nothing except gathering heart-beats from all the hosts in your fleet and updating the DNS periodically.

not to mention people that use something like a local caching daemon like nscd might cache a bad ip for a whole day

Re: Why Loggly Chose AWS Route 53 Over Elastic Load Balancing

#23

Post by latch »

mrucci wrote: post #18

Interesting points. Here is a few things you'll miss choosing Route 53 over ELB: * HTTPS termination. * Autoscaling group management. By connecting an ELB to an autoscaling group, the logic of registration and deregistration is fully managed behind the scenes. With route53, you have to implement it yourself. * Minimum autoscaling group size. If you enable ELB health checks, you can rely on the ELB to maintain a group…

Is https termination really worth mentioning? They'll still be running some type of web server (nginx, apache, whatever) and enabling https termination there is probably easier than going through the elb wizard.

Re: Why Loggly Chose AWS Route 53 Over Elastic Load Balancing

#25

Post by mey »

Goopplesoft wrote: post #13
mey wrote: post #2

Why would you allow your clients to transmit potentially sensitive data to you as clear text over the internet?

I'm sure it's because rsyslog supports it (many mentions in the article to staying compliant with rsyslog).

Thats what I don't get, how is it responsible to stay compliant when that means insecure?

Re: Why Loggly Chose AWS Route 53 Over Elastic Load Balancing

#26

Post by spamizbad »

mrucci wrote: post #18

Interesting points. Here is a few things you'll miss choosing Route 53 over ELB: * HTTPS termination. * Autoscaling group management. By connecting an ELB to an autoscaling group, the logic of registration and deregistration is fully managed behind the scenes. With route53, you have to implement it yourself. * Minimum autoscaling group size. If you enable ELB health checks, you can rely on the ELB to maintain a group…

ELB's HTTPS termination is mediocre and, last I checked, doesn't offer the best ciphers. A year ago It was impossible to get an A+ on ssltest https://www.ssllabs.com/ssltest/ using ELB to terminate SSL.

Not to mention it still needlessly includes a ton of dangerously insecure ciphers just begging to be misclicked.

Re: Why Loggly Chose AWS Route 53 Over Elastic Load Balancing

#27

Post by fubu »

Serious question: Are people upvoting this to poke fun like some kind of daily wtf?

A logging platform that lists 1 of their 2 major requirements as "To not drop any data, ever" is using round robin DNS for fault tolerance? I can't see too many people on HN upvoting this for being insightful or impressive.

Edit: I just can't help myself. How are you going to send syslog when any server fails and not "drop any data, ever"? Even over TCP the in transit messages are lost when the connection is broken. So like, their business is basically syslog and they don't know that?

Re: Why Loggly Chose AWS Route 53 Over Elastic Load Balancing

#28

Post by [deleted] »

fubu wrote: post #27

Serious question: Are people upvoting this to poke fun like some kind of daily wtf? A logging platform that lists 1 of their 2 major requirements as "To not drop any data, ever" is using round robin DNS for fault tolerance? I can't see too many people on HN upvoting this for being insightful or impressive. Edit: I just can't help myself. How are you going to send syslog when any server fails and not "drop any data, e…

[deleted]

Re: Why Loggly Chose AWS Route 53 Over Elastic Load Balancing

#29

Post by latch »

fubu wrote: post #27

Serious question: Are people upvoting this to poke fun like some kind of daily wtf? A logging platform that lists 1 of their 2 major requirements as "To not drop any data, ever" is using round robin DNS for fault tolerance? I can't see too many people on HN upvoting this for being insightful or impressive. Edit: I just can't help myself. How are you going to send syslog when any server fails and not "drop any data, e…

I upvoted it in the hopes that someone would provide the missing piece. Like "oh, we forgot to mention that the DNS is pointing to our own haproxy servers that all have redundant power/network/whatever) or something.

Re: Why Loggly Chose AWS Route 53 Over Elastic Load Balancing

#30

Post by shackledtodesk »

mrucci wrote: post #18

Interesting points. Here is a few things you'll miss choosing Route 53 over ELB: * HTTPS termination. * Autoscaling group management. By connecting an ELB to an autoscaling group, the logic of registration and deregistration is fully managed behind the scenes. With route53, you have to implement it yourself. * Minimum autoscaling group size. If you enable ELB health checks, you can rely on the ELB to maintain a group…

ELB scales horrible and can not scale to even tens of thousands of connections per second, let alone handling spikes of 100k/sec simultaneous connections. Even if you get AWS to prewarm to ELB at a higher peak rate, if you spike over those limits you will drop new incoming connections. HTTPS termination is trivial compared to a requirement to be able to actually handle hundreds of thousands to millions of simultaneous connections per second.
Post reply on HN