Fun that a bug for my team happens to be the thing linked in this article. :)
Hey Daniel,
Funny to see your name there!
Out of control Java processes when dealing with a leap second?
21–30 of 43 posts
Re: Out of control Java processes when dealing with a leap second?
#22I was able to restore my java app boxen (and all other boxen) by only setting the date to the current date. No restarts necessary.
Re: Out of control Java processes when dealing with a leap second?
#23Would love to read a root cause analysis of the relevant source code. Anyone have a link?
Re: Out of control Java processes when dealing with a leap second?
#24Would love to read a root cause analysis of the relevant source code. Anyone have a link?
Re: Out of control Java processes when dealing with a leap second?
#25Leap second + Java + Linux fix (without reboot): /etc/init.d/ntp stop; date; date `date +"%m%d%H%M%C%y.%S"`; date; -- then restart java
fixed it for Twitpic too
Re: Out of control Java processes when dealing with a leap second?
#26Would love to read a root cause analysis of the relevant source code. Anyone have a link?
I, too, am a little curious. Anyone seen thread dumps from any of these floating around?
Re: Out of control Java processes when dealing with a leap second?
#27Earlier quoted context omitted.
Yeah this is what is slaying linkedin at the moment.
MySQL is also apparently affected, according to the #mysql IRC channel.
Yes, this happened on my development MySQL instance and also one of my production instances of MySQL -- but not another one that I have. I'm not sure why it only affected one, maybe different versions of MySQL?
Re: Out of control Java processes when dealing with a leap second?
#28Leap second + Java + Linux fix (without reboot): /etc/init.d/ntp stop; date; date `date +"%m%d%H%M%C%y.%S"`; date; -- then restart java
A lot simpler:
date --set "`date`"
Re: Out of control Java processes when dealing with a leap second?
#29its the futex() call that is bugged, so therefore things that are highly multi-threaded... for example mysql and java!
Many unix things are not as multithreaded as those examples.
Re: Out of control Java processes when dealing with a leap second?
#30Even if you're not running java in your stack, it's worth a quick check on your servers. Our SoftLayer servers use an Adaptec RAID card and the monitoring software uses the JVM and will suck up all of your CPU. All of our MySQL/Cassandra/Redis/Kestrel servers needed to be bounced because they had RAID cards. Frustrating.
Even if you don't have java installed but you're running on shared infrastructure that might be running MySQL/Java elsewhere it's a crazy day. My inbox is full of performance-below-acceptable-levels/high-load alerts.