Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Jerks on the Internet: what my first DDoS taught me

sergiomattei.com

Post reply on HN
Print view View original

31–40 of 95 posts

Re: Jerks on the Internet: what my first DDoS taught me

#31

Post by baolongtrann »

fxfan wrote: post #29

Does anybody have experience in getting DDOS'd? All i see are 3 ip addresses (offending) in the screenshot and it makes me wonder how many is typical? I have never been ddos's and all I ever receive are failed ssh attempts with simple passwords. Pretty much the easiest thing to tackle. But I'd love to know from DDOS'd people how their attacks looked? From cloudflare logs all I see is a single IP address being blocked…

By definition, DDoS is Distributed. From my experience working on a Layer4 DDoS Protection solution, a typical case often range from 1000 to 100k flows.

Re: Jerks on the Internet: what my first DDoS taught me

#33

Post by jeffwass »

fxfan wrote: post #29

Does anybody have experience in getting DDOS'd? All i see are 3 ip addresses (offending) in the screenshot and it makes me wonder how many is typical? I have never been ddos's and all I ever receive are failed ssh attempts with simple passwords. Pretty much the easiest thing to tackle. But I'd love to know from DDOS'd people how their attacks looked? From cloudflare logs all I see is a single IP address being blocked…

Anybody know why blocking the offending IP didn’t work?

Re: Jerks on the Internet: what my first DDoS taught me

#34

Post by kqr »

fxfan wrote: post #29

Does anybody have experience in getting DDOS'd? All i see are 3 ip addresses (offending) in the screenshot and it makes me wonder how many is typical? I have never been ddos's and all I ever receive are failed ssh attempts with simple passwords. Pretty much the easiest thing to tackle. But I'd love to know from DDOS'd people how their attacks looked? From cloudflare logs all I see is a single IP address being blocked…

I don't know where the line goes between DOS and DDOS, but in my youth I had a virtual server attacked by an acquaintance for a few days. They still won't admit to doing it, but all available evidence at the time pointed to them.

It started as a TCP SYN flood, which I had never seen before -- but since it originated from only two addresses, I could block them. Once I did, more addresses joined in on the attack. I figured out what was happening and how to prevent the connection table from filling up completely under the SYN flood, and then the attack changed shape into a UDP flood, from yet more addresses.

(Many of the addresses corresponded to free shell hosts and managed webhosts with easily exploitable PHP scripts. I tried to inform the people running those hosts that they were being used for an attack, but the vast majority of them seemed to ignore that. A couple of people responded and we could figure out kinda what had happened and what scripts were involved. Remember, kids, that connecting a machine to the internet is a great responsibility.)

Re: Jerks on the Internet: what my first DDoS taught me

#35

Post by adventured »

megablast wrote: post #27

> Therefore, when requesting the endpoint, a massive SQL request would be made, freezing the server while the items were fetched + serialized into JSON (a Django REST Framework performance weak point). No caching?

I was surprised to not see that in the things I will do / fix list. Caching those JSON API endpoints would have dramatically changed the ability to absorb that DDoS. If merely adding pagination brought it back to functioning (from 100% CPU to ~60%), it wasn't a very large attack and caching would have trivially handled it. Either way, even with Cloudflare and pagination, they should prioritize adding caching on the API at some point in the near-term. The relief on the database will be considerable and it'll buy a lot of API usage growth runway at almost no cost.

Since they're already using Nginx, if they don't want to bother with learning anything else, it's a couple of hours of research to learn how to set up rock solid basic caching using Nginx. It'll quickly get you 85% of the way on caching, until you need something better. Set Nginx loose to do one of the things it's very good at.

Re: Jerks on the Internet: what my first DDoS taught me

#36

Post by Jeraz0l »

jeffwass wrote: post #33
fxfan wrote: post #29

Does anybody have experience in getting DDOS'd? All i see are 3 ip addresses (offending) in the screenshot and it makes me wonder how many is typical? I have never been ddos's and all I ever receive are failed ssh attempts with simple passwords. Pretty much the easiest thing to tackle. But I'd love to know from DDOS'd people how their attacks looked? From cloudflare logs all I see is a single IP address being blocked…

Anybody know why blocking the offending IP didn’t work?

Most likely because he was trying to block the IP at a point where it was forwarded by some other service. At that point, the offending IP would only exist in the x-http-forwarded-for http header and not as the source address of the request.

Re: Jerks on the Internet: what my first DDoS taught me

#37

Post by jacobolus »

sergiomattei wrote: post #24
RHSeeger wrote: post #19

Why is the text for this article 2.5" wide in a normal browser? It makes it annoying to read :(

Thanks for the feedback! Will modify and enlarge the font a little bit.

The font size is fine. The width of the text block is absolutely ridiculous.

http://webtypography.net/2.1.2

https://practicaltypography.com/line-length.html

Re: Jerks on the Internet: what my first DDoS taught me

#38

Post by sergiomattei »

Jeraz0l wrote: post #36
jeffwass wrote: post #33

Earlier quoted context omitted.

Anybody know why blocking the offending IP didn’t work?

Most likely because he was trying to block the IP at a point where it was forwarded by some other service. At that point, the offending IP would only exist in the x-http-forwarded-for http header and not as the source address of the request.

Yup! This was it.

Re: Jerks on the Internet: what my first DDoS taught me

#39

Post by dwd »

fxfan wrote: post #29

Does anybody have experience in getting DDOS'd? All i see are 3 ip addresses (offending) in the screenshot and it makes me wonder how many is typical? I have never been ddos's and all I ever receive are failed ssh attempts with simple passwords. Pretty much the easiest thing to tackle. But I'd love to know from DDOS'd people how their attacks looked? From cloudflare logs all I see is a single IP address being blocked…

They tend to look like that in your Apache logs when they target at the web application level.

Once you're flooded and your process load spikes, you can't see what your web server has run out of workers to handle.

Netstat will give you a better picture as far as IPs/connections once you get the process load down so you can actually run anything.

Re: Jerks on the Internet: what my first DDoS taught me

#40

Post by jmchuster »

Hope 10% of this might be useful to you.

1) the three most important metrics for any endpoint are error rate, lantency, throughput. So i hope you've learned to not be surprised that abnormal throughput (either via ddos attacks or friendly n+1 queries) is a common error condition.

2) banning ip addresses is useless and often counter productive. If possible, short-circuit requests from an ip so you can isolate them and then determine how much damaging knowledge they know (while letting them think they're still hitting your real service)

3) feature development is a great goal, but don't forget that the most important feature is availability. Spending an extra 30 minutes to consider things like pagination and so on, end up being worth it if you think you might get attacked more than 0 times per year.

Post reply on HN