Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

An Update from Robinhood’s Founders

blog.robinhood.com

Post reply on HN
Print view View original

161–170 of 295 posts

Re: An Update from Robinhood’s Founders

#161

Post by SifJar »

rolltiide wrote: post #130
jennyyang wrote: post #34

I know quite a few people that were personally affected by this and lost money due to the two outages and they are all pulling their money from Robinhood. The fact that they can't offer any compensation might be a big problem for them, since they already have zero trading fees, which is what most brokerages offer as compensation. Personally it doesn't pass the smell test for me. The load was much higher the previous…

> The fact that they can't offer any compensation might be a big problem for them, since they already have zero trading fees Robinhood makes the most money than any known firm on Wall Street by getting paid specifically to leak user's trades to other traders. SEC requires a periodic report on that which shows compensation. Can't believe people are still buying Robinhood's pitch of misdirection.

I think you may misunderstand the concept of "Payment for Order Flow"

Re: An Update from Robinhood’s Founders

#162

Post by LaserToy »

malux85 wrote: post #150
cheschire wrote: post #141

Earlier quoted context omitted.

If this were an outage directly caused by a natural disaster, I could understand. This outage was an availability problem. This clearly points to some prioritization problems within the leadership layers if robust and resilient infrastructure was not emphasized. The prioritization problems may not be due to ignorance or malice though, and may be justifiable if there are other fires that are burning brighter. It's sti…

Sometimes you just have to cut them some slack. Have you engineered a highly available cluster before? I'm not talking about the hot-standby postgres master that gets called on once every 2 years, but I'm talking about a 180 node Cassandra cluster thats doing 15,000 writes a second 24/7 and peaking at 60,000 writes a second every day, and you have to do node replacements every week or two because of the high load. Or…

It is not about scale, it is about the fact that people lost real money. If you can’t make it work you should not be in that business, and I don’t really care how hard they work.

I’m taking my account off their platform.

Re: An Update from Robinhood’s Founders

#163

Post by malux85 »

LaserToy wrote: post #162
malux85 wrote: post #150

Earlier quoted context omitted.

Sometimes you just have to cut them some slack. Have you engineered a highly available cluster before? I'm not talking about the hot-standby postgres master that gets called on once every 2 years, but I'm talking about a 180 node Cassandra cluster thats doing 15,000 writes a second 24/7 and peaking at 60,000 writes a second every day, and you have to do node replacements every week or two because of the high load. Or…

It is not about scale, it is about the fact that people lost real money. If you can’t make it work you should not be in that business, and I don’t really care how hard they work. I’m taking my account off their platform.

Is this your first day of trading or something?

People lose money in trading all the time, for hundreds of reasons and some of those reasons are infrastructure downtime.

If your risk profile doesn't reflect that, maybe you should take your money out of trading altogether.

Re: An Update from Robinhood’s Founders

#164

Post by luckylion »

malux85 wrote: post #150
cheschire wrote: post #141

Earlier quoted context omitted.

If this were an outage directly caused by a natural disaster, I could understand. This outage was an availability problem. This clearly points to some prioritization problems within the leadership layers if robust and resilient infrastructure was not emphasized. The prioritization problems may not be due to ignorance or malice though, and may be justifiable if there are other fires that are burning brighter. It's sti…

Sometimes you just have to cut them some slack. Have you engineered a highly available cluster before? I'm not talking about the hot-standby postgres master that gets called on once every 2 years, but I'm talking about a 180 node Cassandra cluster thats doing 15,000 writes a second 24/7 and peaking at 60,000 writes a second every day, and you have to do node replacements every week or two because of the high load. Or…

On the other hand, they've had plenty of time and resources to do just that in a reliable fashion, it's not like it's one guy in his bedroom (I hope!). It's not like they are volunteers doing this open source for the community, they are getting paid (very well, I assume) to run the system. And Management is getting paid (even better, I assume) to make sure the priorities are right and correct decisions are taken. "Who could've known there might be a lot more traffic" sounds like somebody failed in Management, and engineering might have failed by not foreseeing the issue and/or informing Management.

Sure, don't burn people at the stake, but "hey, it's hard, don't blame them, they are doing their best" doesn't cut it for me. I'm sure they're expecting to be paid and not for someone to "do their best" to pay them.

Re: An Update from Robinhood’s Founders

#165

Post by floatingatoll »

dilly_li wrote: post #20

Start from the email notification. They have been asking themselves the easy questions. Just look at the top questions in their email: * Are the funds in my account safe? Yes, your funds are safe. * Was my personal information affected? No, your personal information was not affected. * Can I use my Robinhood debit card? Yes. If you have a debit card, you should have been—and should still be able to—use your card, but…

If they compensate for missed trades due to service outages, then an attacker could take a position, repeatedly DDOS Robinhood until the position is favorable during a DDOS, and then demand reimbursement since they "would have" cashed out that favorable position.

It follows that Robinhood must never reimburse for outages.

Re: An Update from Robinhood’s Founders

#166

Post by franga2000 »

cheez wrote: post #128
UncleMeat wrote: post #96

Earlier quoted context omitted.

Everything has outages. Is this the new narrative now that we've moved on from the leap year thing? That RobinHood is just a bunch of shitty engineers? There are no public details about the root cause. I think RH is bad for people in general, but this pile-on is outrageous.

There are a couple of situations where outages are not normal or acceptable: 1. Dealing with other people's money 2. Monitoring/managing other people's health

From what I've heard, the "teams" maintaining most of these aren't paid half as much as a mid-level FAANG team.

Luckily for everyone, those industries are so old, they have accidental redundancy built in (paper records for old doctors who can't be arsed to use a computer, etc.).

Re: An Update from Robinhood’s Founders

#168

Post by speedplane »

cheez wrote: post #128
UncleMeat wrote: post #96

Earlier quoted context omitted.

Everything has outages. Is this the new narrative now that we've moved on from the leap year thing? That RobinHood is just a bunch of shitty engineers? There are no public details about the root cause. I think RH is bad for people in general, but this pile-on is outrageous.

There are a couple of situations where outages are not normal or acceptable: 1. Dealing with other people's money 2. Monitoring/managing other people's health

> There are a couple of situations where outages are not normal or acceptable: 1. Dealing with other people's money 2. Monitoring/managing other people's health

Generally true, but there is a couple of exceptions to this rule: if everyone knows that the company is brand new and does not have an established reputation, then using that app requires a general acceptance of risk.

Robinhood was brand new, and outages should have been expected. The problem with Robinhood isn't the outage, it's that it was marketed to college students gambling with their parent's money, who know just enough about the stock market to be dangerous, but not enough to invest properly.

Re: An Update from Robinhood’s Founders

#169

Post by malux85 »

luckylion wrote: post #164
malux85 wrote: post #150

Earlier quoted context omitted.

Sometimes you just have to cut them some slack. Have you engineered a highly available cluster before? I'm not talking about the hot-standby postgres master that gets called on once every 2 years, but I'm talking about a 180 node Cassandra cluster thats doing 15,000 writes a second 24/7 and peaking at 60,000 writes a second every day, and you have to do node replacements every week or two because of the high load. Or…

On the other hand, they've had plenty of time and resources to do just that in a reliable fashion, it's not like it's one guy in his bedroom (I hope!). It's not like they are volunteers doing this open source for the community, they are getting paid (very well, I assume) to run the system. And Management is getting paid (even better, I assume) to make sure the priorities are right and correct decisions are taken. "Wh…

Can you give me a concrete example of a massive distributed system that has zero downtime?

Because the largest distributed system I have seen and worked on was at Apple (or maybe DFP at Google) - and even though they had some of the smartest people in the world and literally billions of dollars behind them, there were still an endless list of problems and downtime events.

Spoiler alert: It doesn't exist.

Re: An Update from Robinhood’s Founders

#170

Post by sitzkrieg »

toomuchtodo wrote: post #55
0xy wrote: post #46

Earlier quoted context omitted.

The difference is regulation. There are very few regulations and oversight on cloud compute providers, whereas an average person cannot just spin up an app and begin selling securities in a month as you can being a cloud provider. While RH's ToS does theoretically absolve them of technical issues, they are obligated to comply with 'best execution' securities mandates, no? Separately, it'd be extremely bad for busines…

Robinhood will have to deal with a flood of FINRA and SEC complaints from these outages. I'm unsure how much longer FINRA will allow them their broker dealer license with a copious amount of failure in the rear view mirror. Arbitration is forced, but Robinhood is on the hook for the fees for everyone who decides to arbitrate. Robinhood users might not get anything, but they can still cause pain.

if that were true thinkorswim would have been gone years ago. heavy futures volatility killed it for days at a time
Post reply on HN