Earlier quoted context omitted.
I'm a one man operation keeping a master-slave setup with a manual failover and it's been pretty smooth sailing once I got it setup. Don't know how much more complex master-master would be.
Same here, and as long as you understand how mysql replication works it's not too much effort to deal with. Performing the initial sync without downtime is a bit tricky, but can be done with a well-designed database and some thought. Basically you need to at least temporarily make the bulk of your data read-only, so that you can do most of the data transfer while things are running, and then only briefly lock tables…
“We have been experiencing a catastrophic DDoS attack”
111–120 of 151 posts
Re: “We have been experiencing a catastrophic DDoS attack”
#112DigitalOcean has had a few DDoS attacks targeting their SFO1 datacenter over the past few months, but fortunately each one seemed to disappear in under half an hour.
Re: “We have been experiencing a catastrophic DDoS attack”
#113Godaddy has been getting attacked a lot recently as well. Who is likely behind attacking servers, whether other server companies or governments?
Re: “We have been experiencing a catastrophic DDoS attack”
#114Re: “We have been experiencing a catastrophic DDoS attack”
#115I've done some googling before asking here: Can anybody explain why Linode is so often targeted like this? We moved Cronitor off Linode in spring 2015. During the christmas holiday when they suffered a 2 week DDOS I thought of the family time I'd be missing that year as we did a crash migration to AWS had we not migrated when we did. I have to imagine this has been horrible for their business. I would use Linode if I…
I am also surprised as this is not the first time I am reading about it on HN. Linode seems to be highly vulnerable to certain attacks as we could see in the past. I hope they will fix it and provide a permanent solution as I was hoping to use them as a part of my network, but I see more and more signals they can't handle serious traffic. Hopefully, they will redesign their infrastructure to handle it. I am with all…
My understanding is that Atlanta in particular has some poor upstreams and is making our job pretty difficult there. Notice that it's almost always Atlanta that's getting hit. I would suggest just using other datacenters or making sure your high availability model includes several DCs (which I would suggest at any hosting provider, really).
Re: “We have been experiencing a catastrophic DDoS attack”
#116I disagree with people saying these types of attacks can't be prevented if you switched hosts. I'm sure Google+cloudflare[0] would keep your website online. AWS also if you had the cash. The amount of distributed traffic happening right now against linode would probably only represent a 5% increase in traffic to a popular Google product. At least you know they have the expertise. Nothing against the very smart and ta…
Google and AWS probably have sophisticated DDOS mitigation (can anyone comment on this?) and you can scale up pretty quickly. But if your service is the direct target of the attack (as opposed to the whole provider) and your servers are getting hammered... Even if you architecture allows quick horizontal scaling, you still face a tough decision. The attack could go on for days, and the hosting costs can go really hig…
Google/AWS probably have 100 times the capacity (and redundancy and architecture reliability and failover and awesomeness) of linode. That means that, first, they can't be put down easily, second, a DDoS is limited to a small subset of the infrastructure and doesn't bleed to every customers and services.
As for traditional hosting companies (OVH and the likes) When you're being DDoSed, they'll null-route your IP space. (i.e. they advertise your IPs as dont-exist-on-the-internet-anymore). The traffic is dropped while in transit on the internet because it can't go anywhere. It doesn't reach the hosting company anymore.
Note: being null-routed means your site and all your services are off the internet and thus effectively dead.
As for CloudFlare. They have many locations all around the world and they can absorb a lot of traffic, to the point they themselves cannot be DDoS. They have active monitoring and mitigation against common attacks and known malicious sources, which may prevent the attack without even you knowing about it.
When you're under attack, you can block subnet/AS/countries in cloudflare settings, or request a challenge/capcha from every visitors. Cloudflare will reject visitors (with or without challenging them) at their edge location before any traffic can get to you. It is very effective from my experience.
Generally speaking. The only way to stop a DDoS is to do it before it reaches your datacenters so you need help from your ISP/provider/CDN.
Edit: The attack that put down linode last christmas was against linode itself and not a specific customer. Part of the mitigation included linode moving its critical services behind cloudflare :D
Re: “We have been experiencing a catastrophic DDoS attack”
#117Earlier quoted context omitted.
It's pretty much the same. You almost never want writes on both sides (now in a failover plan anyway), so as long as you have a switch for which side receives the writes, it's simple.
Site A, insure any writes use odd numbers in unique field. Site B, insure any writes use even numbers for unique field. This avoids conflict.
If I'm updating existing records, and the MySQL master at Site A gets updated, then goes down before Site B is updated.. I've got an inconsistent setup.
Been thinking about Master-Master MySQL replication recently as we have a system that's duplicated and taken offline each summer (to run a summer camp), and looking for a way to sync changes in it back to the main 'live' MySQL database.
Re: “We have been experiencing a catastrophic DDoS attack”
#118Earlier quoted context omitted.
One difference between Linode and providers like AWS is that the typical deployment architecture on Linode still exposes customer VPSes to direct L3 internet traffic whereas on a best-practices AWS deployment that is almost never the case. I'd imagine it's easier to filter out bogus L3 traffic when the vast majority of your target IP space comes with explicit configuration as to what sort of L7 application traffic is…
You can't really compare aws to linode. Aws have hundreds of Gb of transit bandwidth so they can easily absorbe big attacks. They also have a backbone network which allows them to increase the surface area of attacks which increases the available bandwidth. It's actually not that easy to Filter "bogus" traffic. In the hosting world, especially cloud, you have thousands of customers doing whatever they want. Who knows…
Put an ELB in front of the services, the ELB only listening to port 80/443, roll out the ELB publicly, roll out new instances only accessible privately, kill old instances being DDoSed => Problem solved => Repeat for all other services, they shouldn't be publicly accessible in the first place.
Ain't saying it's easy but there are some options to help mitigate the attack.
Re: “We have been experiencing a catastrophic DDoS attack”
#119Earlier quoted context omitted.
We stuck with Linode after last Christmas too, but this new attack looks like the final straw. How have you found Vultr?
All's well so far! Happy to recommend them. They seem to be the next biggest name in VPSes after Linode and DigitalOcean, with broadly similar price and quality.
Re: “We have been experiencing a catastrophic DDoS attack”
#120Going cheap with one of the non-big 3 public cloud providers will cost you.