Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

If Carpenters Were Hired Like Programmers

jasonbock.substack.com

Post reply on HN
Print view View original

61–69 of 69 posts

Re: If Carpenters Were Hired Like Programmers

#61

Post by gymbeaux »

dsr_ wrote: post #54
gymbeaux wrote: post #42

Earlier quoted context omitted.

The Robinhood API is of course the API for stock broker Robinhood. Their API, last I used it, probably could have been optimized to provide data in fewer API requests, but aside from that, there are many pieces of data that change constantly, like the current price of a stock. There are pieces of data that change occasionally, like the next earnings report date for a company. Then there are pieces of data that should…

> Then there are pieces of data that should never change, like a company's CUSIP (essentially the primary key in a database of stocks). I hate to break it to you, but tickers and CUSIPs do change. The world of 'corporate actions' turns out to be a catchbasin requiring human intervention.

Yeah fair enough. I see a ticker change somewhat often (eg ETH to ETD, FB to META), CUSIPs change much less often, but unlike tickers CUSIPs aren’t reused. A company’s CUSIP will always be their CUSIP, but it might not be their current CUSIP.

Re: If Carpenters Were Hired Like Programmers

#62

Post by gymbeaux »

justinclift wrote: post #59
gymbeaux wrote: post #28

Earlier quoted context omitted.

It's not "security" in the hacking sense. Perhaps that was a poor word choice. So when you hit an API, the server can reject your request for any reason, typically a malformed request. Maybe a header it's expecting is missing, or the API key is invalid, something like that. With Robinhood's API, you can't simply provide your bearer token and hit the endpoint. There are MANY headers it is expecting, and if any are mis…

Hopefully you explained the unique weirdness with Robinhood's API parser during the presentation, such that it was necessary? Otherwise it sounds more like a case of "Here's how to get around the rate limiting of Company Foo's API and abuse those limits anyway". Without the explanation, that'd get you an immediate No at lots of places just from the implied legal exposure/liability. Though probably not all. ;)

I’m not sure I agree with that. If you’re a technical person, like say a “Senior Software Engineer”, you should pretty much know that’s not what’s happening with something like this. Rate limiting is server-side. “Getting around it” means making fewer requests than what would trigger the rate limiting/temporary blacklisting.

You could distribute the requests amongst multiple IP addresses via say a proxy, but that isn’t necessarily going to work, like if the rate limiting is done by bearer token or API key rather than IP address (they should be doing it by the token rather than the IP address), but I made no mention of that so I don’t think anybody would think that’s what I was doing.

Re: If Carpenters Were Hired Like Programmers

#63

Post by gymbeaux »

D13Fd wrote: post #56
smashed wrote: post #25

Earlier quoted context omitted.

> For those curious, #4, my presentation was about how I reverse engineered the Robinhood API, spoofed requests to get around their “security” This was probably against their TOS and even if not, trying to bypass security measures, no matter how weak, is unethical behavior, and possibly illegal. That would be a red flag in an hiring scenario since you were essentially bragging about breaking the law. Unless Robinhood…

Yeah. This sounds like the presentation requirement (ridiculous as it is) worked to filter someone based on unethical behavior.

To quote another commenter, “If an interviewer whined about Wall Street TOS, I'd pull my application on the spot.” But yes, that might be why I didn’t make it past that step. My internet connection also dropped in the middle of it (AT&T Fiber, connected via Ethernet), and I had to give the presentation again, that might have been why- though they explicitly said that wasn’t why, but… you never know.

Re: If Carpenters Were Hired Like Programmers

#64

Post by gymbeaux »

u32480932048 wrote: post #55
smashed wrote: post #25

Earlier quoted context omitted.

> For those curious, #4, my presentation was about how I reverse engineered the Robinhood API, spoofed requests to get around their “security” This was probably against their TOS and even if not, trying to bypass security measures, no matter how weak, is unethical behavior, and possibly illegal. That would be a red flag in an hiring scenario since you were essentially bragging about breaking the law. Unless Robinhood…

If an interviewer whined about Wall Street TOS, I'd pull my application on the spot.

I think some companies just want to look for a reason to disqualify a candidate to make their decision process easier. If you have two roughly equal candidates, pick the one whose side project did NOT violate some random company’s TOS. I kind of get it. It’s incredibly lazy and arguably just as unethical as violating some random company’s TOS, but I get it.

Re: If Carpenters Were Hired Like Programmers

#66

Post by justinclift »

gymbeaux wrote: post #62
justinclift wrote: post #59

Earlier quoted context omitted.

Hopefully you explained the unique weirdness with Robinhood's API parser during the presentation, such that it was necessary? Otherwise it sounds more like a case of "Here's how to get around the rate limiting of Company Foo's API and abuse those limits anyway". Without the explanation, that'd get you an immediate No at lots of places just from the implied legal exposure/liability. Though probably not all. ;)

I’m not sure I agree with that. If you’re a technical person, like say a “Senior Software Engineer”, you should pretty much know that’s not what’s happening with something like this. Rate limiting is server-side. “Getting around it” means making fewer requests than what would trigger the rate limiting/temporary blacklisting. You could distribute the requests amongst multiple IP addresses via say a proxy, but that isn…

It depends on how it's pitched. :)

If you went with something like "using caching to avoid hitting API limits" it sounds legit.

Whereas if you went with something like "getting around Company Foo's API rate limits" it sounds like you're exploiting a bug in their system.

If you did present the system as spoofing requests to subvert a rate limit / blacklist, as per:

  #4, my presentation was about how I reverse engineered the Robinhood API, spoofed
  requests to get around their “security”, and used two levels of caching (Redis and a
  typical relational DB) to subvert rate limiting/IP blacklisting
... then that doesn't sound like the legit use case scenario. ;)

Your further follow up discussion makes it sound like you were just using a self-developed caching system though, and presented it terribly for the target audience.

Re: If Carpenters Were Hired Like Programmers

#67

Post by gymbeaux »

justinclift wrote: post #66
gymbeaux wrote: post #62

Earlier quoted context omitted.

I’m not sure I agree with that. If you’re a technical person, like say a “Senior Software Engineer”, you should pretty much know that’s not what’s happening with something like this. Rate limiting is server-side. “Getting around it” means making fewer requests than what would trigger the rate limiting/temporary blacklisting. You could distribute the requests amongst multiple IP addresses via say a proxy, but that isn…

It depends on how it's pitched. :) If you went with something like "using caching to avoid hitting API limits" it sounds legit. Whereas if you went with something like "getting around Company Foo's API rate limits" it sounds like you're exploiting a bug in their system. If you did present the system as spoofing requests to subvert a rate limit / blacklist, as per: #4, my presentation was about how I reverse engineere…

I don't know that there's a way you can pitch your reasoning to me where I would agree. The two statements sound equivalent to me. API rate limits are just that- rate limits. This is not akin to hacking and a software engineer I doubt would ever confuse it with hacking of any kind. What would be an example of "exploiting a rate limiting bug"? Either the server limits/throttles requests, or it doesn't.

Re: If Carpenters Were Hired Like Programmers

#68

Post by justinclift »

gymbeaux wrote: post #67
justinclift wrote: post #66

Earlier quoted context omitted.

It depends on how it's pitched. :) If you went with something like "using caching to avoid hitting API limits" it sounds legit. Whereas if you went with something like "getting around Company Foo's API rate limits" it sounds like you're exploiting a bug in their system. If you did present the system as spoofing requests to subvert a rate limit / blacklist, as per: #4, my presentation was about how I reverse engineere…

I don't know that there's a way you can pitch your reasoning to me where I would agree. The two statements sound equivalent to me. API rate limits are just that- rate limits. This is not akin to hacking and a software engineer I doubt would ever confuse it with hacking of any kind. What would be an example of "exploiting a rate limiting bug"? Either the server limits/throttles requests, or it doesn't.

> What would be an example of "exploiting a rate limiting bug"?

All sorts of possibilities spring to mind for that. :)

Here's a hypothetical example. Lets say you discovered a logic bug in the API limit code along these lines: "So, it turns out if you have an `X-Forwarded-For` header in the request, then the request is processed properly except it isn't counted towards the limit. Effectively a bypass."

Exploiting this one would of course be trivial, as you'd just need to set that header for all your requests.

Re: If Carpenters Were Hired Like Programmers

#69

Post by Turing_Machine »

throwawaaarrgh wrote: post #8

The irony here is carpenters aren't generic. You don't hire "a carpenter", you hire a cabinetmaker, a framer, a fine woodworker, a cooper. It would be ridiculous to hire a framer for cooperage. Most people doing tech hiring are idiots. But most people getting hired are also idiots. So I think it all shakes out about even. If the results actually mattered, we would have some kind of industry wide standards, people wou…

> Most people doing tech hiring are idiots.

Didn't Guido van Rossum once get rejected for a Python job for having insufficient experience?

When the hiring team demands X+5 years of experience for a technology that only came out X years ago, it's no surprise when the applicants start faking experience.

Post reply on HN