An Update from Robinhood’s Founders • Hacker News Board

Re: An Update from Robinhood’s Founders

#281

Post by aeyes » Thu, Mar 05, 2020, 3:58 AM UTC

manigandham wrote: post #59

tbrock wrote: post #7

Earlier quoted context omitted.

This happened to us at Hustle years ago. Basically if you run on AWS there’s a DNS server provided inside each VPC that usually works fine but which has no observable load metrics etc... so you don’t really know you are slamming it and are about to have a problem unless you audit your entire codebase. Why? Well that tiny DNS server has certain capacity constraints and if you don’t cache DNS lookups by using a http/ht…

What scenarios cause this many DNS lookups though? Connections should be kept-alive after the IP translation, so if it's really new connections being setup constantly then wouldn't that show up as a major bottleneck first?

Running on Kubernetes this is easy, it's one of the first issues you hit.

Every DNS request for external domains turns into 10 if you don't explicitly configure FQDNs (dot at the end). This is because in the default configuration the resolver runs with ndots 5 to search all the possible internal Kubernetes and cloud-provider names. Then you have lookups for IPv4 and IPv6 in parallel. So for every external name you look up, you storm the upstream DNS with 10 requests for non existing domains.

Furthermore, the current default DNS service in Kubernetes doesn't have any kind of caching for these kinds of lookups (especially not NXDOMAIN) enabled.

But like I said, this is one of the first issues you hit running Kubernetes on Amazon. It is widely known and can easily be fixed by scaling up some more instances, changing ndots settings, using FQDNs or configuring caching. There is no way that this was the issue, it is plastered all over the internet, the logs are clear and the fixes can be implemented in minutes.

It also doesn't go down completely, the rate-limiter is packets/s on the interface.

Re: An Update from Robinhood’s Founders

#282

Post by bayonetz » Thu, Mar 05, 2020, 6:29 AM UTC

pensatoio wrote: post #146

minimaxir wrote: post #125

Earlier quoted context omitted.

If you actually work at Square, it's poor form to advertise in this manner.

Or they’re just taking pride in their work?

Indeed!

Re: An Update from Robinhood’s Founders

#283

Post by bayonetz » Thu, Mar 05, 2020, 6:34 AM UTC

minimaxir wrote: post #125

bayonetz wrote: post #109

Just use Square’s Cash App! Free stock trades AND you can buy fractional shares AND a bunch of other stuff like P2P payments and bitcoin. I work there and so can say with some authority that we can handle more volume without going down than RH can.

If you actually work at Square, it's poor form to advertise in this manner.

Disagree. I’m suggesting a better alternative at a contextually relevant time based on personally earned experience.

Re: An Update from Robinhood’s Founders

#284

Post by C1sc0cat » Thu, Mar 05, 2020, 9:14 AM UTC

kube-system wrote: post #264

C1sc0cat wrote: post #262

Earlier quoted context omitted.

Fix your stock exchanges to have stock splits then like the LSE does.

The fees I am referencing were imposed by brokers, not exchanges. Our exchanges have stock splits, but that still doesn't make a $10 fee on a single $50 share very palatable to the small-time investor.

A 50$ share should be split which was my point and tbh if your only investing $50 you should not be investing in individual shares.

Re: An Update from Robinhood’s Founders

#285

Post by conanbatt » Thu, Mar 05, 2020, 10:19 AM UTC

alkonaut wrote: post #202

conanbatt wrote: post #41

Earlier quoted context omitted.

You couldn't execute or cancel orders.

That means there are no orders mishandled either. If no one has an SLA then just switching the servers off without thinking about whether customers were planning on trading seems fully in their right. This is terrible for their reputation, but that does't mean they are going to start handing out money because people argue they could have avoided losses if the servers had been up. It's going to be extremely difficult…

I doubt that you need an SLA to enforce a penalty for keeping other people's money.

Re: An Update from Robinhood’s Founders

#286

Post by alkonaut » Thu, Mar 05, 2020, 11:00 AM UTC

conanbatt wrote: post #285

alkonaut wrote: post #202

Earlier quoted context omitted.

That means there are no orders mishandled either. If no one has an SLA then just switching the servers off without thinking about whether customers were planning on trading seems fully in their right. This is terrible for their reputation, but that does't mean they are going to start handing out money because people argue they could have avoided losses if the servers had been up. It's going to be extremely difficult…

I doubt that you need an SLA to enforce a penalty for keeping other people's money.

They can probably be fined by some authority, but that penalty isn’t the same as being liable for losses people claim they made because the site was inaccessible. The fine wouldn’t be paid to customers.

Re: An Update from Robinhood’s Founders

#287

Post by SifJar » Thu, Mar 05, 2020, 1:23 PM UTC

rolltiide wrote: post #256

SifJar wrote: post #161

Earlier quoted context omitted.

I think you may misunderstand the concept of "Payment for Order Flow"

In what way? How would you describe it in a way everyone can understand in as few words?

It is not "leak user's trades to other traders", it is accepting payment from a third party in order to execute the trade via that third party.

Re: An Update from Robinhood’s Founders

#288

Post by kube-system » Thu, Mar 05, 2020, 1:45 PM UTC

C1sc0cat wrote: post #284

kube-system wrote: post #264

Earlier quoted context omitted.

The fees I am referencing were imposed by brokers, not exchanges. Our exchanges have stock splits, but that still doesn't make a $10 fee on a single $50 share very palatable to the small-time investor.

A 50$ share should be split which was my point and tbh if your only investing $50 you should not be investing in individual shares.

Whether or not you think that market for trading should exist, it does.

Re: An Update from Robinhood’s Founders

#289

Post by dehrmann » Thu, Mar 05, 2020, 4:01 PM UTC

throwaway9d0291 wrote: post #190

dehrmann wrote: post #138

Earlier quoted context omitted.

I've always been told trading, but especially trading options, is asking the big boys to take your money.

It's not at all so straightforward. There's definitely money to be made trading. In particular, there's plenty of opportunity to find a successful strategy that just doesn't scale. It won't make you rich but it will make you money.

> there's plenty of opportunity to find a successful strategy that just doesn't scale.

True.

Re: An Update from Robinhood’s Founders

#290

Post by tbrock » Thu, Mar 05, 2020, 5:02 PM UTC

JdeBP wrote: post #225

jcheng wrote: post #40

Earlier quoted context omitted.

I read this and thought, “surely there’s an OS-level DNS cache?” Apparently not on Linux! https://stackoverflow.com/questions/11020027/dns-caching-in-...

That's misleading. The way that this has worked for decades on Linux-based operating systems and on Unices is that one installs a local caching DNS proxy, choosing one of the many available: ISC's BIND, Bernstein's dnscache, unbound, dnsmasq, PowerDNS, MaraDNS, and so forth. Every Unix system having a local caching DNS proxy was and is as much a norm as every Unix system having a local MTS. A quarter of a century ago…

I didn’t say they discourage usage of a dns cache at all.