Ask HN: Anonymous person sent proof of SSH access to our production server • Hacker News Board

Re: Ask HN: Anonymous person sent proof of SSH access to our production server

#161

Post by HelloImDumb » Wed, Jul 27, 2016, 6:43 PM UTC

JasuM wrote: post #83

calgoo wrote: post #75

Earlier quoted context omitted.

Also, to get rid of 60% of the script kiddies, change the port to something high 1000+ (or even better 10000+). Its not a great option, but it does lower the amount of automated attacks that you get.

I'd be very cautious before moving SSH to a non-privileged port (over 1024). Any user on the server might start their own SSH server on the port assuming the real SSH server is dead. While this is hard to exploit (needs access to normal user, needs to kill real SSH server, need to get around SSH server key checking), it still is at least a theoretical reduction in security.

Besides local iptables you can forward the port at the firewall level. Many people (myself included) have observed failed ssh logins going from many thousands daily to on average zero just by changing the port on a netfacing server. Of course a determined hacker who is after you can trivially portscan. But why not block all that noise and a huge percentage of shotgun attacks? If someone is out just to find servers to root with a new zero day they're liable to spray the net searching on the target port rather than portscanning all IPv4 space. Why not buy yourself some free time?

It is basically zero inconvenience to add an extra argument or shell setting.

Re: Ask HN: Anonymous person sent proof of SSH access to our production server

#162

Post by einrealist » Wed, Jul 27, 2016, 6:44 PM UTC

einrealist wrote: post #113

1) Isolate that server. Treat this server as your ground zero, but assume that other systems might be compromised (including servers and devices of employees etc). 2) Rotate / delete SSH keys on all other servers, that have the same keys installed as the compromised server. Private SSH keys may have been compromised in your company. Inform all employees who had a public key on that server that their access is revoked…

One addition in regard to the perpetrator: Do not trust this person - these persons (always assume its more than one person). Don't commit to anything. Since they cannot be trusted, don't waste time and money on them.

Re: Ask HN: Anonymous person sent proof of SSH access to our production server

#163

Post by CiPHPerCoder » Wed, Jul 27, 2016, 6:50 PM UTC

web007 wrote: post #130

CiPHPerCoder wrote: post #74

Earlier quoted context omitted.

> or Mitnick Security Please no. Not Mitnick. I'd rather funnel clients towards my competitors than Kevin Mitnick. He's a skilled social engineer, and his greatest social engineering success was manipulating the media into believing he speaks for hackers in general. He is not a programmer, his opinions on cryptography aren't insighful, etc. His only skill is deception.

Regardless of your opinion of The Man Himself, the company employees people that are good at things beyond social engineering. I've seen two separate engagements with them (one as a 3rd party and one as technical contact), and both found significant non-trivial vulnerabilities that needed to be patched.

I can't speak towards the efficacy of his staff, but that testimonial is generally true of any engagement with most security teams.

Re: Ask HN: Anonymous person sent proof of SSH access to our production server

#164

Post by _kst_ » Wed, Jul 27, 2016, 6:51 PM UTC

purerandomness wrote: post #139

_kst_ wrote: post #115

Earlier quoted context omitted.

If you're going to offer a reward, I suggest making it conditional on revealing how they got in. (This also assumes you're able to contact the person who sent the e-mail.)

> making it conditional on revealing how they got in Do you mean obligatory ? If you really mean conditional, could you elaborate why?

I mean don't pay someone for breaking into your system and not telling you how they did it. Pay them only if they provide you with some useful information that can help you fix the problem.

Re: Ask HN: Anonymous person sent proof of SSH access to our production server

#165

Post by Practicality » Wed, Jul 27, 2016, 6:58 PM UTC

You should consider what information was available on that server. Did the code contain any passwords for other systems (such as an internal DB, or another production machine?). If so, those systems should be considered compromised now too.

On the other hand, if you are just getting started it may be that production doesn't have much on it yet, and you can just nuke the thing and start over.

I understand recommending security experts is easier, so that if you're wrong, you can just blame the expert, but you may be able to make the decision yourself if you are aware of everything that is on that machine.

Re: Ask HN: Anonymous person sent proof of SSH access to our production server

#166

Post by alpb » Wed, Jul 27, 2016, 7:05 PM UTC

I think this is what happens your service does not have a responsible disclosure and reward policy. Every fairly important service should have a security page and you should think of what you should be rewarding for each scenario in advance. Because it will happen and you better give people good reason to stay ethical.

Re: Ask HN: Anonymous person sent proof of SSH access to our production server

#167

Post by nitrogen » Wed, Jul 27, 2016, 7:10 PM UTC

Don't forget the legal obligations you may have to your customers, depending on what kind of data transits through or is stored on your servers. Different states have different requirements, so you'll want professional legal advice.

Re: Ask HN: Anonymous person sent proof of SSH access to our production server

#168

Post by drzaiusapelord » Wed, Jul 27, 2016, 7:13 PM UTC

danesparza wrote: post #3

This has been covered elsewhere (like on serverfault: http://serverfault.com/a/107346/2557 ) But it comes down to: - Take existing server down immediately. I'm assuming it is not on an isolated network -- so this should really be a priority. - Prep a new patched server (with a smaller attack surface and updated security credentials) - Postmortem the old box on an isolated network. Try to understand how the attacker g…

Great advice, I'd also add a couple more things about checking the desktop environment because its much more likely a windows or osx box was compromised and keys stolen or keystrokes captured than someone brute-forcing sshd or finding some sshd zero-day.

The few times we've been in this position, its always been someone's desktop that's been compromised. If you don't have a strong desktop security policy, a UTM, traffic inspection, key encryption policy, etc and you store keys or passwords locally, then this becomes a risk.

Re: Ask HN: Anonymous person sent proof of SSH access to our production server

#169

Post by MichaelGG » Wed, Jul 27, 2016, 7:13 PM UTC

laumars wrote: post #81

calgoo wrote: post #75

Earlier quoted context omitted.

Also, to get rid of 60% of the script kiddies, change the port to something high 1000+ (or even better 10000+). Its not a great option, but it does lower the amount of automated attacks that you get.

I'm not a fan of that option because it's causes an inconvenience for legitimate usage without offering any real security benefits. If you've hardened sshd then the biggest benefit you get from changing the port number is reducing the size of your log files. If you've not hardened sshd then you're just one nmap (et al) scan away from being in the same boat as you were previously. If you really want to hide SSH from s…

How is port knocking not, I dunno, at least 10x more inconvenient for users?

FWIW, I've never seen a brute force attack on SSH other than on port 22. Most likely, a targeted attacker would realise if you change the port, you're probably not going to have a trivial password.

Re: Ask HN: Anonymous person sent proof of SSH access to our production server

#170

Post by devishard » Wed, Jul 27, 2016, 7:20 PM UTC

laumars wrote: post #160

devishard wrote: post #154

Earlier quoted context omitted.

> My point was correct to the specific attack you broadly described. But you didn't correct it, you proposed a solution that didn't address the attack I described. > Your example required a web server attack that allowed arbitrary code execution and privladge elevation; which is a hugely specific attack and it's pretty fair to say it's unlikely (in the case of gaining root access and then choosing to enable SSH, whic…

> The XSS attack is just an example of a vulnerability that wouldn't be addressed by hardening SSH. There are plenty of other vulnerabilities that wouldn't be addressed by hardening SSH. Well durr! We could be here all night listing things that SSH hardening wouldn't secure against. I never suggested it was a silver bullet to fix all security needs (which seems to be the faux argument you're accusing me off). I'm lov…

> Well durr! We could be here all night listing things that SSH hardening wouldn't secure against. I never suggested it was a silver bullet to fix all security needs (which seems to be the faux argument you're accusing me off).

That's not what I accused you of--people can read our previous discussion and see both what I actually accused you of saying, and also that you said what I accused you of saying.

> You cannot compete on an intellectual level so you make baseless accusations about my professional capabilities instead.

I did call your course of action a mistake, but that's not an attack on your professional capabilities. Everyone makes mistakes. I'm sure you're reasonably skilled at your job.

I think you'd enjoy this conversation a lot more if you didn't take my disagreement with your strategy as a personal attack. But if that's what you want to do I can't stop you.