Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Anonymous person sent proof of SSH access to our production server

news.ycombinator.com

Post reply on HN
Print view View original

211–220 of 246 posts

Re: Ask HN: Anonymous person sent proof of SSH access to our production server

#211

Post by Daneel_ »

I am a forensic investigator and security consultant working for a well established organisation.

I'd recommend engaging a forensic consultant from a reputable company. The mish-mash of advice here is mildly useful if you know what you're doing, but since you don't the only way to be somewhat confident that you're no longer compromised is to perform proper scoping and investigation.

If you've already wiped/rolled/overwritten logs then this instantly becomes more difficult. I would want to see a saved copy of your firewall configuration too, for analysis in Nipper or similar.

I'd recommend the free Redline forensic tool from Mandiant if you're unwilling to hire a consultant.

Re: Ask HN: Anonymous person sent proof of SSH access to our production server

#212

Post by SolarNet »

tedunangst wrote: post #124
kbenson wrote: post #91

In addition to the wonderful technical advice already here for how to deal with the server, there is the question for how to deal with the anonymous person. If the proof contained the method of exploit I suggest something along the lines of: "Thank you for bringing this problem to our attention! We are taking steps to resolve the problem now, but would like to reward you for your work. If you let us know how you woul…

Didn't you read the LastPass thread? If the hacker can take down your company, you have to offer them a reward equal to the value of your company.

You are forgetting some externalities re: value of being legitimate vs. criminal (e.g. contracts vs. ransom, legit money vs. tainted money, morality, fame, etc).

Re: Ask HN: Anonymous person sent proof of SSH access to our production server

#213

Post by gbog »

Many people tell you to hire an "expert". Be careful, many such security experts are experts in fud and taking the money of clueless frightened people. For a lot of money they'll run their attack bot on your servers and send you you a twenty pages auto generated report which you will need another expert to read and understand. And security is not a one off task, you need to either not care at all, or make sure everybody cares all the time.

While you doing yourself the emergency backup and logs evacuation task, and the password flips, I would suggest to mission your best two hackers on the task of quick learning the basics and making sure there's no obvious hole in the wall.

Re: Ask HN: Anonymous person sent proof of SSH access to our production server

#214

Post by morninj »

scoot wrote: post #204
emcrazyone wrote: post #158

Earlier quoted context omitted.

100% agree. Disclaimer I own a data center and have dealt with customer collocated equipment breaches. In addition to the above steps: -> disable root being able to login inside your sshd_config file. Make sure PermitRootLogin no -> rename the root account too so if they are using an exploit based on user authentication then perhaps they won't be able to elevate to root. -> disable password based logins and go to cer…

> Disclaimer I own a data center I'm curious why you think owning a DC makes you less qualified to respond? Presumably because someone that senior is less in touch with day-to-day security operations?

The "disclaimer" prologue is often humble bragging. It's often less about flagging a conflict of interest and more about claiming to have authority or status.

Re: Ask HN: Anonymous person sent proof of SSH access to our production server

#215

Post by aaronmdjones »

more_corn wrote: post #185
MichaelGG wrote: post #169

Earlier quoted context omitted.

How is port knocking not, I dunno, at least 10x more inconvenient for users? FWIW, I've never seen a brute force attack on SSH other than on port 22. Most likely, a targeted attacker would realise if you change the port, you're probably not going to have a trivial password.

I agree, I set up port knocking and it's a huge pain in the butt (so much so that I never moved it out of testing). It also lacks cryptographic verification. It's a lot of trouble for a not quite ideal solution.

The Single Packet Authorization variant of port knocking has built-in authentication.

Re: Ask HN: Anonymous person sent proof of SSH access to our production server

#216

Post by siffer »

Consider hiring a Security Engineer for future issues. As for the current issue, are you a target for any known reason. Is their anything of value on the server?

If not, it's highly possible someone in your group who already has access could be involved.

Otherwise, maybe the Security Engineer serendipitously found you.

Re: Ask HN: Anonymous person sent proof of SSH access to our production server

#217

Post by mercanator »

I think your best bet is to follow the advice here to offer a bounty, but also start setting up honeypot servers for each of your public network facing services. This way you can do surveillance as this hacker (or others) are gaining access to your system. Good honeypot monitoring software should tell you where they got in from, what directories they accessed, and keystrokes being used.

Re: Ask HN: Anonymous person sent proof of SSH access to our production server

#218

Post by Spooky23 »

laumars wrote: post #43

In terms of hardening against SSH attacks, the principles are quite simple. Your business case might mean that some of the following cannot be applied, but there's plenty of measures below that you can use to harden SSH. 1. firewall - only allow SSH connections from trusted static IPs 2. Use SSH keys then disable password logins. Lots of guides online to create keys, so I'll just cover the 2nd point: as root or sudo,…

Better yet, don't expose SSH to untreated networks. Setup a VPN/jumpbox and only allow connectivity from that jump box.

Re: Ask HN: Anonymous person sent proof of SSH access to our production server

#220

Post by theaustinseven »

They got SSH access? Steps I would take are:

1. Isolate the machine.

2. Rotate your keys.

3. Set up ssh via ssh keys and remove ssh passwords.

4. Now do whatever you want with the isolated machine.

If you already are using ssh via ssh keys then either one of your employees has been hacked or "the call is coming from inside the house".

I would really suggest, as others have, trying to get the mysterious individual to tell you how they did what they did.

Post reply on HN