Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Former Hostgator employee arrested, charged with rooting 2,700 servers

arstechnica.com

Post reply on HN
Print view View original

31–40 of 61 posts

Re: Former Hostgator employee arrested, charged with rooting 2,700 servers

#31

Post by andrewljohnson »

lawnchair_larry wrote: post #29
ghshephard wrote: post #2

This all seemed like a pretty run of the mill story about an insider violating company trust, and then getting caught - until the final sentence: "Among other things, a desktop monitoring system that took screenshots of employee workstations in one-minute increments helped Hostgator officials quickly zero in on Gisse." Not something I'd want on my personal system, but it's exactly the sort of thing that I think every…

I'll never work for any employer who does that, so I'd have trouble calling it a reasonable thing to do.

Any powerful IT guy should be monitored and have his power checked. IT personnel have almost unmatched power in an organization to cause damage without detection.

Anything with lots of confidential information, or anything financial, and you are going to want to monitor all the people with access constantly. You may not want to snoop real-time, but you are going to want to be able to find and fix breaches after the fact, and do root-cause analysis.

It's not a matter of trust in the IT people, it's a matter of people go crazy sometimes, and people make bad hiring decisions sometimes.

Re: Former Hostgator employee arrested, charged with rooting 2,700 servers

#32

Post by ghshephard »

lawnchair_larry wrote: post #29
ghshephard wrote: post #2

This all seemed like a pretty run of the mill story about an insider violating company trust, and then getting caught - until the final sentence: "Among other things, a desktop monitoring system that took screenshots of employee workstations in one-minute increments helped Hostgator officials quickly zero in on Gisse." Not something I'd want on my personal system, but it's exactly the sort of thing that I think every…

I'll never work for any employer who does that, so I'd have trouble calling it a reasonable thing to do.

The areas where it would be important are on security/NOC systems, where there is almost unlimited power given the proper credentials.

The Runbooks that NOC teams have, quite often have them connecting to a lot of systems with greatly heightened privileges - It's not unusual for a NOC employee to have expansive sudo privileges on many of the unix hosts they manage. They are also often on privileged VLANs, with direct IP routing to a lot of hosts that normally wouldn't be reachable.

Most of our NOC guys have their own personal laptops, and they can hop onto the (unprivileged) wireless system and do their own thing when they aren't working an incident.

I'd have no problem having my screen captured once a minute when I was working in that type of environment.

Re: Former Hostgator employee arrested, charged with rooting 2,700 servers

#33

Post by stoic »

hfsktr wrote: post #14
gav wrote: post #9

Earlier quoted context omitted.

A previous employeer used Spector 360[1] on the majority of workstations. It would monitor everything including taking a screengrab every 5 seconds that you could then watch later. They'd sit down employees and playback fast-forwarded video showing how much time was wasted on Facebook, personal email, shopping, etc. It's horribly invasive but it meant everyone was too scared to use work computers for personal things.…

My old job used spector. I'm pretty sure everyone knew it but people would still be on FB playing games when you walked past. I believe IT was the last to get it and for about a week after they installed it on our machines it didn't work because Microsoft Essentials disabled it as malware. I don't think they ever looked at it unless they wanted to fire someone and didn't want to pay unemployment and needed proof that…

This is pretty much why they had this at HG

Re: Former Hostgator employee arrested, charged with rooting 2,700 servers

#34

Post by lawnchair_larry »

ttrreeww wrote: post #26
ultimoo wrote: post #12

Wow, he actually patched 'netstat' and 'ps'. It must have been to hide certain processes and port numbers from showing up. I wonder how may one go about understanding if they are using a 'hacked' version of something as non-trivial and comprehensive as 'ps' and 'netstat'. I mean, if I'm not suspicious, I wouldn't give it a second though and trust the output of these commands.

Amateur hour, quite a few rootkits out there are almost completely undetectable...

Really? Can you name one?

Re: Former Hostgator employee arrested, charged with rooting 2,700 servers

#35

Post by incision »

ghshephard wrote: post #2

This all seemed like a pretty run of the mill story about an insider violating company trust, and then getting caught - until the final sentence: "Among other things, a desktop monitoring system that took screenshots of employee workstations in one-minute increments helped Hostgator officials quickly zero in on Gisse." Not something I'd want on my personal system, but it's exactly the sort of thing that I think every…

>Not something I'd want on my personal system, but it's exactly the sort of thing that I think every NOC/Secure environment should have for post-mortem assessments.

Tools of that sort [1][2] are pretty standard in call center environments.

I'm not a fan.

First, the software tends to be incredibly expensive. Second, in my experience, it's primarily used by managers looking for reasons to bludgeon their $30K, entry-level call takers over trivial infractions.

1: http://www.nice.com/contact-center-interaction-recording

2: http://www.callcopy.com/products/screen-capture

Re: Former Hostgator employee arrested, charged with rooting 2,700 servers

#36

Post by stevenrace »

lawnchair_larry wrote: post #34
ttrreeww wrote: post #26

Earlier quoted context omitted.

Amateur hour, quite a few rootkits out there are almost completely undetectable...

Really? Can you name one?

Anything that lives outside of userspace.

If one has root and patience - flash BIOS and wait for a coldboot. One can even get an IP stack to pull down new firmware between boots. The user sees a normal post screen and your hypervisor sees normal hardware adapters.

See Jonathan Brossard's 'prior work' slide from his Defcon talk on his work [1] for more details on the state of X86 backdooring.

'Trusted computing' and all that.

[1] http://www.youtube.com/watch?v=yRxDvkKBMTc

https://media.defcon.org/dc-20/presentations/Brossard/DEFCON...

Re: Former Hostgator employee arrested, charged with rooting 2,700 servers

#37

Post by MertsA »

ultimoo wrote: post #12

Wow, he actually patched 'netstat' and 'ps'. It must have been to hide certain processes and port numbers from showing up. I wonder how may one go about understanding if they are using a 'hacked' version of something as non-trivial and comprehensive as 'ps' and 'netstat'. I mean, if I'm not suspicious, I wouldn't give it a second though and trust the output of these commands.

If it's a Redhat based server just run rpm -Va and that will verify the integrity of every rpm package on the system. If you're in a situation where you can't trust the current computer you can run it in a chroot from a known good copy.

Re: Former Hostgator employee arrested, charged with rooting 2,700 servers

#38

Post by saurik »

saurik wrote: post #15
ultimoo wrote: post #12

Wow, he actually patched 'netstat' and 'ps'. It must have been to hide certain processes and port numbers from showing up. I wonder how may one go about understanding if they are using a 'hacked' version of something as non-trivial and comprehensive as 'ps' and 'netstat'. I mean, if I'm not suspicious, I wouldn't give it a second though and trust the output of these commands.

While this isn't guaranteed (all tools, including the compiler, may be patched), you can use checks and balances: verify /proc doesn't contain phantom processes, compile your own copy of ps, try more-obscure tools like top. If by "understand" you just mean "notice"... well, you don't, until one day you accidentally stumble across one of the above and start digging. (Maybe, for example, you install some kind of server…

(When I said "more obscure tools like top" I meant to say "pstree" but edited the statement one too many times before posting.)

Re: Former Hostgator employee arrested, charged with rooting 2,700 servers

#39

Post by shuzchen »

drzaiusapelord wrote: post #25
ultimoo wrote: post #12

Wow, he actually patched 'netstat' and 'ps'. It must have been to hide certain processes and port numbers from showing up. I wonder how may one go about understanding if they are using a 'hacked' version of something as non-trivial and comprehensive as 'ps' and 'netstat'. I mean, if I'm not suspicious, I wouldn't give it a second though and trust the output of these commands.

Ideally, you should be running some kind of tripwire scan. If the hash for common utilities changes and you didn't update those binaries, then something bad has happened. Hacked versions of common utilities is a common payload for rootkits.

That would be helpful for rootkits coming from outsiders, but would only serve to slow down (not stop) an insider. An insider knows what protections are in place (probably implemented them too) and can defeat the hash check if she knew how the hash was calculated, or can ship the binary alongside a regular update.

Re: Former Hostgator employee arrested, charged with rooting 2,700 servers

#40

Post by lawnchair_larry »

stevenrace wrote: post #36
lawnchair_larry wrote: post #34

Earlier quoted context omitted.

Really? Can you name one?

Anything that lives outside of userspace. If one has root and patience - flash BIOS and wait for a coldboot. One can even get an IP stack to pull down new firmware between boots. The user sees a normal post screen and your hypervisor sees normal hardware adapters. See Jonathan Brossard's 'prior work' slide from his Defcon talk on his work [1] for more details on the state of X86 backdooring. 'Trusted computing' and a…

I know how they work, I'm asking GP to name one that he could have used. There isn't actually much "out there" that is undetectable. You'd likely have to write your own, which is highly non-trivial.
Post reply on HN