Live data from Hacker News

Ask HN: Anonymous person sent proof of SSH access to our production server

news.ycombinator.com

31–40 of 246 posts

Re: Ask HN: Anonymous person sent proof of SSH access to our production server

#31
post #12
post #2

>We are a small company and don't have any security experts, etc. Find one.

Or contact the person if you can and ask them how they got in maybe even offer him a financial reward. Since he contact you Anonymously and is not trying to extort you he's just trying to point the issue out so there's no point in over reacting.

Sure, getting information on the particular vulnerability and its fix is useful.

However, doing anything less than clean reinstall of the tainted system and implementing the fix there would be underreacting. Verifying if that system was/wasn't backdoored takes ten (if not hundred) times more effort than nuking it from orbit and reconfiguring a new one.

Re: Ask HN: Anonymous person sent proof of SSH access to our production server

#33
Step 1) Image the system. Don't log into it, don't pull anything off of it. Take a snapshot of it. If your hosting provider doesn't provide you a direct way to do that, contact them and get them to do it. You want a clean image of it for investigating later.

Step 2) Hire a security expert / forensics company. Give them the image, ask them how to proceed.

Things to keep in mind:

- You don't know now much you can trust the person who has contacted you. It's possible they think they're a good samaritan, though logging into a system as a proof of concept is pretty far into grey-hat.

- Anything you say to them may one day be public record, attached to your company forever.

- It's possible they've compromised far deeper than this, and they just haven't said so

- If they've gotten in, it's possible that they aren't the only ones, so even if they cooperate and help you close the hole, you still want to do steps 1 and 2 above.

Re: Ask HN: Anonymous person sent proof of SSH access to our production server

#34
post #23

Earlier quoted context omitted.

This isn't the most insightful or actionable comment, but it is the correct solution for both the short term problem and many long term problems.

There's not much insightful advice to be given, chances are this guy could've easily traversed to every box on their network and set up a plethora of backdoors. Without a "security expert" there really is nothing they can do.

> There's not much insightful advice to be given

Except some semblance of an idea of where to find a security expert. Even if it's a slight remix of "where to find ryanlol", that's plenty more insight and guidance. :)

(It probably seems obvious to you, but most people don't know where to find one.)

Re: Ask HN: Anonymous person sent proof of SSH access to our production server

#35
post #32
post #13

Give the person a small reward for pointing out the vulnerability and offer another small reward for suggesting how to fix it.

why small ? the reward should be in accordance with the criticity IMHO.

So a startup with 100k total funding got messaged a major company crushing bug. What should they offer for such a big find, 50k? Half their runway?

It's not that rewards should be small for big finds, but if you are legit poor, you have limits on what you can do.

Re: Ask HN: Anonymous person sent proof of SSH access to our production server

#36
Step 1: Try to get in contact with the person and see if he/she is willing to help you share details on entering your systems. Thank this person and see if you can provide a reward.

Step 2: Next step is setting up new systems, and start from scratch. Install the systems, start with basic system hardening and up-to-date software packages. Use https://github.com/CISOfy/lynis to validate your configuration.

Do not have any interaction or data exchange with the old (compromised) systems.

Step 3: Save all running systems to learn from the event. See if you can find the main cause why this happened.

Step 4: Learn about security, hire someone on your team with security knowledge.

Step 5: Do regular (technical) audits.

Re: Ask HN: Anonymous person sent proof of SSH access to our production server

#37
Why would you have and production system exposing SSH to the public?

If you must, at least do these steps:

- Disable password SSH login - Install root kit scanner, like rkhunter and check if your networked systems are infected. s/he might gained access to other instances in your infra. - Use port scanning on all your instances and check if there is any suspecious rpc port is open that you are not familiar wtih. - Enable unattended security upgrades. - Check for the vulnerabilities listings for your internet facing services, like nginx, apache, HAproxy, etc.. - forward all your syslog logs to remote system so the attacker can't cleanup her/her traces after establishing the attack. - enable automatic blockers like fail2ban.

Re: Ask HN: Anonymous person sent proof of SSH access to our production server

#38
post #30
post #21

Earlier quoted context omitted.

Also, wipe clean the existing server after the Postmortem. Even if fairly sure that the server can be cleaned it's never safe to re-use a box which was compromised.

Use one of these: http://www.wnd.com/files/2015/08/hillary-clinton-secret-serv...

"You mean like, with a cloth or something?"

Re: Ask HN: Anonymous person sent proof of SSH access to our production server

#39
post #36

Step 1: Try to get in contact with the person and see if he/she is willing to help you share details on entering your systems. Thank this person and see if you can provide a reward. Step 2: Next step is setting up new systems, and start from scratch. Install the systems, start with basic system hardening and up-to-date software packages. Use https://github.com/CISOfy/lynis to validate your configuration. Do not have…

"Thank this person and see if you can provide a reward"

This should be:

"Thank this person and provide a reward"

Looking at all the other steps you'll have to go through to remedy the situation, this is the least of your costs. (Provided they cooperate and are not malicious)

Post reply on HN