When should I not kill -9 a process? • Hacker News Board

Re: When should I not kill -9 a process?

#81

Post by kator » Sat, May 24, 2014, 9:54 PM UTC

jacquesm wrote: post #62

kator wrote: post #35

I think caution is still key, if you don't know what's going on Slow is Fast here. Several years ago while I was on an airplane flying to spend a nice vacation break with my family my admin partner tried to shutdown a MySQL db the "right way". He logged in and ran a mysqladmin shutdown and waited for a while. Not sure how long he waited but he claimed it was a "long time". Since it felt like there was no response to…

You never were more than a power failure away from that disaster anyway by the looks of it. It wouldn't matter much whether or not the shutdown had started. In cases like these you're essentially playing Russian Roulette with your data, only you're using 5 bullets instead of 1.

Well in all fairness we did have replicas and sadly something was broken there.. Not pointing fingers but I checked it all before going on "vacation".. :-)

Re: When should I not kill -9 a process?

#82

Post by xxs » Sat, May 24, 2014, 11:13 PM UTC

kator wrote: post #78

na85 wrote: post #73

Earlier quoted context omitted.

That seems like a strange anecdote to me. The only reason I can think of to shutdown a production SQL database would be to do maintenance. In which case any sane admin should be taking a backup in advance. That should be the first step. If the database is not a production server, does it matter if data was lost? If the production database crashed/bugged out and therefore needed to be halted, was it really ready for p…

No offense taken, the reason he went this path is the web sites involved seemed to be non-responsive. The core issue was actually a deadlock but he wasn't trained in detecting that and in the past just restarting the db "fixed it". Because of course all the sessions dropped and dead locks were resolved. This was very much a high visibility production environment and yes we all make mistakes sometimes.. How else can w…

resolving deadlocks, other locks: log on via mysql, show processlist and kill the few long standing queries. It's an ugly solution and should not be used freely, yet a lot better than database restart.

Re: When should I not kill -9 a process?

#83

Post by dfox » Sun, May 25, 2014, 1:09 AM UTC

voltagex_ wrote: post #61

dfox wrote: post #23

Earlier quoted context omitted.

It's probably better to use SIGABRT (-6) for that, as it also dumps core and is less likely to be handled by the process (there are valid reasons to handle SIGSEGV and continue in execution)

I never thought of handling SIGSEGV. What would you use it for?

It's mostly about removing checks for rare occurrences:

* catching of NULL pointer references (ie. transforming SEGV into catchable userspace exception) * testing whether some in memory data have been changed/accessed (ie. mprotect(), wait for SEGV, set flag, un-mprotect(), return) * paging in userspace (eg. across network)

All that seems obscure (and is mainly useful for virtual machines and such), but it is not so rare to find process that actually does this (usually because of some library, or because it is some kind of virtual machine)

Re: When should I not kill -9 a process?

#84

Post by kator » Sun, May 25, 2014, 10:05 AM UTC

xxs wrote: post #82

kator wrote: post #78

Earlier quoted context omitted.

No offense taken, the reason he went this path is the web sites involved seemed to be non-responsive. The core issue was actually a deadlock but he wasn't trained in detecting that and in the past just restarting the db "fixed it". Because of course all the sessions dropped and dead locks were resolved. This was very much a high visibility production environment and yes we all make mistakes sometimes.. How else can w…

resolving deadlocks, other locks: log on via mysql, show processlist and kill the few long standing queries. It's an ugly solution and should not be used freely, yet a lot better than database restart.

Exactly, but my partner wasn't the most patient person and in all fairness it was typical to 3,000 or more connections active to the database. With that many queries outstanding and the dead locks it was always a bit hairy trying to find them.. Meanwhile every teenager on the planet was hitting their refresh button trying to figure out why the site was down.. :-)

Re: When should I not kill -9 a process?

#85

Post by marcosdumay » Mon, May 26, 2014, 3:55 PM UTC

kator wrote: post #74

marcosdumay wrote: post #66

Earlier quoted context omitted.

I'm aware it may be an irritating question to answer, and I'd understand if you don't bother, but I have to ask it because I did never understand why people do such things... Was the extra performance in any way worth it?

This was a core cluster db for about 150 web sites that had high profiles and user traffic. It was a long time ago and the client didn't want to pay for 20 more boxes to do something more durable. These things are always a balance between durability and speed. In this case the client wanted cheap speed. Sadly it worked very well except when people used a big hammer on it. We had replication slaves but in this inciden…

Thanks to the answer. Looks like Murphy's law was in action that day.

So, that was by the client's demand. You got to save 20 machines with it, I'm impressed, and now I understand it better why you did it.