Interesting. I had it in my head that Netflix was a pure Java shop. Was I wrong in the first place, or has something changed?
Mainly a Java shop. They published Node and even some C stuff back then in their blog posts.
Python at Netflix
151–160 of 250 posts
Re: Python at Netflix
#152Does anyone have more info about this or know of similar projects? We lean on the many of the statistical and mathematical libraries (numpy, scipy, ruptures, pandas) to help automate the analysis of 1000s of related signals when our alerting systems indicate problems. We’ve developed a time series correlation system...
There's https://prometheus.io/ which you're probably familiar with
Re: Python at Netflix
#153> The ability to drop into a bpython shell and improvise has saved the day more than once. what do you really do here - connect to the flask instance and route requests manually ?
Also who uses anything but ptpython!
Re: Python at Netflix
#154I fail to understand the use of python in a distributed environment while the language has such poor concurrency support (on top of the lack of a type system). You can make your application HA, but they are obviously not trying to squeeze out every CPU cycle.
Re: Type system -- at Gigantum we very aggressively enforce all classes and methods in the core libraries must be fully typed using mypy. The depth and expressiveness of mypy rivals that of any other strongly typed language.
Re: Python at Netflix
#155Earlier quoted context omitted.
Your country of citizenship or birth plays zero role in terms of getting an H-1B visa. It neither helps nor hurts. And it's certainly not "pretty easy" to get an H-1B in any sense. For example, right now, if you wanted to work in the U.S. the earliest date you could start working would be October 1, 2020 . (That's because this year's lottery is already over, you'll have to try your shot with the April 2020 lottery, a…
This is not how it works, if you have a master's degree and want to work for Google you will most likely get your H1-B visa. https://www.immi-usa.com/h1b-masters-quota/ If you don't what they do is they ask you to work for them elsewhere for 1 year ( London / Switzerland ) then move you back to the US with an L1 visa.
> if you have a master's degree ... you will most likely get your H1-B visa
It's "most likely", if your definition of "most likely" is 51%.
Let's calculate the probability:
• In 2018, there were 95,885 applicants with a Masters degree or higher[1]. Out of 190,098 total applicants.
• There are 20,000 spots available for U.S. Master's degree holders, and 65,000 spots for everyone. (Ignoring the fact that there's a reservation of 1,400 for Chile and 5,400 for Singapore.)
• Your probability of rejection in the masters lottery is 1-(20000/95885) = 0.7914
• Your probability of rejection in the general lottery is 1-(65000/(190098-20000)) = 0.6179
• These are independent events; the probability of being rejected in both is: 0.6179 x 0.7914 = 0.489
• If you flip that, your probability of being selected in the lottery with a U.S. Master's degree is 0.51, ie. 51%.
[1] https://redbus2us.com/h1b-historical-data-lottery-vs-85k-quo...
Re: Python at Netflix
#156I fail to understand the use of python in a distributed environment while the language has such poor concurrency support (on top of the lack of a type system). You can make your application HA, but they are obviously not trying to squeeze out every CPU cycle.
> I fail to understand the use of python in a distributed environment while the language has such poor concurrency support Because it's a distributed environment probably is exactly why. Python has (arguably) great concurrency support apart from Multi-threading. https://www.youtube.com/watch?v=MCs5OvhV9S4 So if you need concurrency in the context of a single thread, then Python's GIL is a non-starter. But a distribut…
In fact it's precisely Python's deficiency in multithreading that lead to it having one of the best ecosystems for every other form of concurrency, like green threads and multiprocess applications.
Re: Python at Netflix
#157Earlier quoted context omitted.
The only way to get hired at competitive companies is via referral - if you live in the Bay Area you probably know someone or know someone who knows someone at Netflix. Trying any other way is a lottery - and if you didn’t go to MIT/Stanford/CalTech/Harvard then your odds are pretty bad. The other way is already working at a famous company and having another famous company send you a recruitment email via linked in,…
I was not hired via referral. They reached out because they were using a node module I had written. Not sure how common that is, but tbh I think any general characterization of the hiring practices of such a large org is bound to be wrong in many ways.
It was more a comment on the lack of feedback rather than hiring practices, which as mentioned I have no exposure to.
Re: Python at Netflix
#158I fail to understand the use of python in a distributed environment while the language has such poor concurrency support (on top of the lack of a type system). You can make your application HA, but they are obviously not trying to squeeze out every CPU cycle.
> I fail to understand the use of python in a distributed environment while the language has such poor concurrency support Because it's a distributed environment probably is exactly why. Python has (arguably) great concurrency support apart from Multi-threading. https://www.youtube.com/watch?v=MCs5OvhV9S4 So if you need concurrency in the context of a single thread, then Python's GIL is a non-starter. But a distribut…
If you’re doing (data analysis|simulations|Image processing) you can offload computation to numpy, which releases the GIL. This allows nice multicore speedups with python and threading.
The same holds for various CPU intensive standard library functions implemented in C.
The GIL issue is real, but posts like this one confused me for years. Please, don’t exaggerate GIL issues.
Re: Python at Netflix
#159I had actually looked at Flask-RESTPlus for a relatively recent project before deciding to use FastAPI.
Surprised they use Flask. I'm working on my first larger, industrial-strength REST API in Python and I've found the Django Rest Framework to be more suited once you get to that level of complexity.
Django works great if you're buying into the Django ecosystem. But if you're standing up a simple REST API, I've found Flask is much more flexible. Either approach will get you to where you're going, but I find Flask gets out of your way and lets you do your thing, while you have to find the Django Way of doing what you need to do.
This may be biased (I've been a Flask user since 0.7), and perhaps I'm stuck in my ways, but I've found I either need to find a supported and blessed Django plugin for the thing I need, or build something much more complex than I'd need in Flask.
Re: Python at Netflix
#160I fail to understand the use of python in a distributed environment while the language has such poor concurrency support (on top of the lack of a type system). You can make your application HA, but they are obviously not trying to squeeze out every CPU cycle.
> I fail to understand the use of python in a distributed environment while the language has such poor concurrency support (on top of the lack of a type system).
Well, without knowing the exact issue people trying to solve, of course you won't understand the motivation behind.
Reading their post, it seems that they are not using Python for serving, mainly for long running daemon processes. Concurrency is probably not something people care about in such situation, nor squeeze CPU perf. In fact such workload probably wants to maintain a low-key finger print.