Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Robinhood reports outage as markets rebound

axios.com

Post reply on HN
Print view View original

111–120 of 164 posts

Re: Robinhood reports outage as markets rebound

#111

Post by rurp »

yebyen wrote: post #103
EpicEng wrote: post #64

Earlier quoted context omitted.

These guys just took $323 million in a series E. They're valued at 7.6B (well, they were...) Can't afford better engineers?

If they are down for one full day out of every 365, that's still a potential ceiling of 99.726% uptime for the year. My home internet was down 14% of the afternoon on Sunday this week, and on through the evening. On a month-to-month basis, I can't even seem to crack 99% reliability. (Geez, I should really call them on it, or perhaps get a new modem...) Point is, how high is Comcast's valuation again exactly?

ISP are pretty much the poster children for the corruption and incompetence that comes from well entrenched oligopolies. Industries with healthy competition, such as retail investing, generally have much higher standards. Comparing RH to Comcast is not a good look for RH.

Re: Robinhood reports outage as markets rebound

#112

Post by yushuf »

nodesocket wrote: post #102
halotrope wrote: post #73

Apparently leap day is the cause: https://mobile.twitter.com/jtech63/status/123460004578739404... Also rumor has it they “lost user data and the backups are not working as expected” Edit: This is the tweet. Not sure if it is legit: https://mobile.twitter.com/classrobinhood/status/12345774264...

As somebody who does DevOps consulting I was baffled how they could be down all day if it was a pure operations issue. That tweet is just speculation though. Hopefully more facts and details come out. My bet is still on it being a database issue. If they indeed are using AWS RDS, I wouldn’t be surprised if they have half of AWS support engineers working on it. Once again, if I had to bet, I say they rolled their own…

I recently interviewed there and they said they're still using Postgres master-slave setup most likely on some cloud provider. Could be catastrophic failure with multiple shards going down at the same time. This might also explain why they mentioned their "backups" not working

Re: Robinhood reports outage as markets rebound

#113

Post by manigandham »

grandridge wrote: post #98
manigandham wrote: post #81

Earlier quoted context omitted.

It's probably one of the most suspenseful market opens in history. The tug of war between bulls and bears across the entire sub has been incredibly entertaining. Regarding leverage: buying options is a fixed loss against the cash in your account. RH also only allows selling secured options. There's no way to lose more than you started with (at least with the current app, they had problems before). Retail investors us…

Haha, you clearly weren't around for bear Stern's and lehman

I watched all that live and saw my own holdings tumble. There wasn't that much suspense though, nobody thought the market would suddenly go up.

Re: Robinhood reports outage as markets rebound

#114

Post by nodesocket »

yushuf wrote: post #112
nodesocket wrote: post #102

Earlier quoted context omitted.

As somebody who does DevOps consulting I was baffled how they could be down all day if it was a pure operations issue. That tweet is just speculation though. Hopefully more facts and details come out. My bet is still on it being a database issue. If they indeed are using AWS RDS, I wouldn’t be surprised if they have half of AWS support engineers working on it. Once again, if I had to bet, I say they rolled their own…

I recently interviewed there and they said they're still using Postgres master-slave setup most likely on some cloud provider. Could be catastrophic failure with multiple shards going down at the same time. This might also explain why they mentioned their "backups" not working

I believe they are on AWS, so maybe RDS PostgreSQL. If so they should be getting unlimited support from top tier AWS engineers.

Re: Robinhood reports outage as markets rebound

#116

Post by EpicEng »

yebyen wrote: post #103
EpicEng wrote: post #64

Earlier quoted context omitted.

These guys just took $323 million in a series E. They're valued at 7.6B (well, they were...) Can't afford better engineers?

If they are down for one full day out of every 365, that's still a potential ceiling of 99.726% uptime for the year. My home internet was down 14% of the afternoon on Sunday this week, and on through the evening. On a month-to-month basis, I can't even seem to crack 99% reliability. (Geez, I should really call them on it, or perhaps get a new modem...) Point is, how high is Comcast's valuation again exactly?

Ah yes, ISP's; the most beloved of corporate entities, and not in any way aided by regional duopolies.

RH isn't an ISP, and being down for an entire day of trading is a big deal. If I don't like my ISP... welp, I'm pretty much screwed. If I don't like RH I have plenty of options.

Re: Robinhood reports outage as markets rebound

#117

Post by manigandham »

halotrope wrote: post #73

Apparently leap day is the cause: https://mobile.twitter.com/jtech63/status/123460004578739404... Also rumor has it they “lost user data and the backups are not working as expected” Edit: This is the tweet. Not sure if it is legit: https://mobile.twitter.com/classrobinhood/status/12345774264...

This person is probing an API which is already down so of course there are errors.

Robinhood would be using a proper date library anyway to handle timezones. The system was fine on the 29th, 1st and even this morning pre-market. Seems more like a capacity issue that cascaded into failure.

Re: Robinhood reports outage as markets rebound

#119

Post by [deleted] »

halotrope wrote: post #97
vsareto wrote: post #92

Earlier quoted context omitted.

If this is true and nothing happens to this company, everyone should feel free to use bugs to cover up any and all IT negligence.

I doubt it will be possible to just shake it off as there is likely millions of dollars lost for customers. Not in some abstract way like when github goes down but literally because people cannot close out positions and options expire. If it is really just a leap year bug It should be gross negligence. Not sure if their TOS will hold then.

[deleted]

Re: Robinhood reports outage as markets rebound

#120

Post by ecnahc515 »

halotrope wrote: post #73

Apparently leap day is the cause: https://mobile.twitter.com/jtech63/status/123460004578739404... Also rumor has it they “lost user data and the backups are not working as expected” Edit: This is the tweet. Not sure if it is legit: https://mobile.twitter.com/classrobinhood/status/12345774264...

My favorite part is this comment: https://twitter.com/holman/status/1234620062398464002 https://www.reddit.com/r/RobinHood/comments/48mep4/robinhood...

Additionally, if you’re a conspiracy theorist — and who isn't, in this wild 2020 ride — Robinhood was down exactly four years ago today, too, lol

Post reply on HN