Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Kraken down 24 hours on system upgrade

status.kraken.com

Post reply on HN
Print view View original

11–20 of 61 posts

Re: Kraken down 24 hours on system upgrade

#11

Post by eks »

donquichotte wrote: post #9
JorgeGT wrote: post #6

They erased this glorious status update: > I fell asleep at my desk. In a dream, the ghost of a butterfly from the future whispered to me "the dollar hit 0 bitcoin". I wake up, people around me dabbing the blood from their eyes. Progress has been made but not enough. I crack my whip -- "we aren't getting to zero like this!". The flogging will continue until morale improves. Expect another update after I make my round…

It appears that we are at the mercy of mentally unstable amateurs.

Just because the social media guy likes to play social media'ish doesn't mean the rest of the team is like that.

But definitely not a good moment to make a joke.

Re: Kraken down 24 hours on system upgrade

#12

Post by dx034 »

mnx wrote: post #5

At least they are communicating. Hopefully there will be a postmortem.

Not really, just saying that it takes longer than expected. Certainly better than giving no update at all but that would likely cause panic for anyone with money on kraken.

Re: Kraken down 24 hours on system upgrade

#13

Post by mrweasel »

eks wrote: post #7
miguelrochefort wrote: post #3

They said it would take 2 hours. It's been more than 24 hours.

It's scary. Why not just revert to the old engine when things started to fell out of place? Having the exchange down for that long is bound to have a big backlash when it comes back online. They should have put a Kraken 2.0 trade engine alongside the first one, and moved people gradually there. It doesn't matter how confident they were with the upgrade before it happened, it's crypto, everything is new. A few lines o…

I notice the "upgrade is coming" notice on their site last week, but I don't have a Kraken account, I was just interested in learning more about crypto-currency trading. Now they made me curious about their platform, for a purely technical perspective.

As you say, why didn't they just revert? Are they not able to? What are the steps in their system upgrade? Are they moving to new hardware? What's their setup like? What software are they running (custom written surely, but what language, which database technologies?)

Incidents like this make me curious, and I would love to read the post mortem on something like this.

Re: Kraken down 24 hours on system upgrade

#14

Post by stp-ip »

eks wrote: post #7
miguelrochefort wrote: post #3

They said it would take 2 hours. It's been more than 24 hours.

It's scary. Why not just revert to the old engine when things started to fell out of place? Having the exchange down for that long is bound to have a big backlash when it comes back online. They should have put a Kraken 2.0 trade engine alongside the first one, and moved people gradually there. It doesn't matter how confident they were with the upgrade before it happened, it's crypto, everything is new. A few lines o…

Their downtime and non usable site was so bad already that downtime is almost as inconvenient as the working site was before. Now at least they are fixing it. They held back the update for months due to testing. They have to take the jump and after x hours of downtime the damage is done, so fixing it once and for all instead of rolling back might actually be the better solution.

Re: Kraken down 24 hours on system upgrade

#15

Post by stp-ip »

mrweasel wrote: post #13
eks wrote: post #7

Earlier quoted context omitted.

It's scary. Why not just revert to the old engine when things started to fell out of place? Having the exchange down for that long is bound to have a big backlash when it comes back online. They should have put a Kraken 2.0 trade engine alongside the first one, and moved people gradually there. It doesn't matter how confident they were with the upgrade before it happened, it's crypto, everything is new. A few lines o…

I notice the "upgrade is coming" notice on their site last week, but I don't have a Kraken account, I was just interested in learning more about crypto-currency trading. Now they made me curious about their platform, for a purely technical perspective. As you say, why didn't they just revert? Are they not able to? What are the steps in their system upgrade? Are they moving to new hardware? What's their setup like? Wh…

Second the notion of a post mortem, but I assume due to their focus on security they value security by obscurity as an additional factor.

Re: Kraken down 24 hours on system upgrade

#16

Post by ll931110 »

donquichotte wrote: post #4

This is really bad. I hope they can handle everyone withdrawing their funds as soon as they are back online. Edit: current status: https://status.kraken.com/incidents/nswthr1lyx72

"Withdrawals in all currencies will be offline for an additional 2-3 hours after other systems come online. If you submit a withdrawal request while withdrawals are offline, the withdrawal will be sent after withdrawals come online again."

That may help stabilizing a bit.

Re: Kraken down 24 hours on system upgrade

#19

Post by bhouston »

donquichotte wrote: post #4

This is really bad. I hope they can handle everyone withdrawing their funds as soon as they are back online. Edit: current status: https://status.kraken.com/incidents/nswthr1lyx72

They are cancelling all pending withdrawal and liquidation requests as well according to their status updates.

So basically a software upgrade/bug can now cause a bank run? Interesting times.

Post reply on HN