Live data from Hacker News

Python at Netflix

techblog.netflix.com

51–60 of 78 posts

Re: Python at Netflix

#51
post #27

Re: Web applications in python - I am curious about the overall feeling for them - Do you write web apps in python because rapid development makes up for any performance problems and if you run into problems you would rewrite it in something faster, or do you feel like a python web app can be just as good as alternatives when it comes to speed? Put another way - when you think of python for web applications / service…

I like Python for the speed of development. Also, at least for these internal apps, speed is generally of secondary concern, since these aren't typically high load applications. At reddit we solved the problem by writing the most often called parts in C and using c extensions. There is definitely more work to do there, but overall I think Python gets a bad rap as far as speed is concerned. It's certainly not the quic…

> At reddit we solved the problem by writing the most often called parts in C and using c extensions.

I'm wondering if anyone has solved these sorts of problems with Cython in the real world. It looks like a great solution for a large set of problems in terms of Python performance, since it becomes a CPython extension, but I never hear anyone talk about it. Can anyone chime in?

Re: Python at Netflix

#53
post #31

Earlier quoted context omitted.

I have to say that after doing a python project I totally don't get why people love it so much. The syntax is slightly cleaner, but the lack of threading, first-class functions and dependency management tools make it waste at least as much time as it saves. I honestly don't believe in the concept of a programming language being able to make code more or less readable. A Java project with a POM file can be up and runn…

Ummm, not sure where you're getting your information but Python has threads ( http://docs.python.org/2/library/threading.html ), first-class functions ( http://en.wikipedia.org/wiki/First-class_function ), and dependency management tools (google for pip, easy_install, etc.)

Sorry, I should have said "lack of true threading in CPython" by which I mean the GIL limitation. And lambda expressions are lame compared to what you can do in JavaScript/Perl/Scala. And the build tools suck compared to Maven which is saying a lot. Maven is cross-platform and install-free. It takes a long time to makes head and tails of pip, easy_install and virtualenv and it still does less than Maven.

Re: Python at Netflix

#54
post #2

As one of the main drivers of Python at Netflix, this one is near and dear to my heart. Let me know if you have any questions, or come visit our booth!

would you be interested to talk about these tools on the http://foodfightshow.org devops podcast? we would love to discuss them with you!

Re: Python at Netflix

#55
post #53

Earlier quoted context omitted.

Ummm, not sure where you're getting your information but Python has threads ( http://docs.python.org/2/library/threading.html ), first-class functions ( http://en.wikipedia.org/wiki/First-class_function ), and dependency management tools (google for pip, easy_install, etc.)

Sorry, I should have said "lack of true threading in CPython" by which I mean the GIL limitation. And lambda expressions are lame compared to what you can do in JavaScript/Perl/Scala. And the build tools suck compared to Maven which is saying a lot. Maven is cross-platform and install-free. It takes a long time to makes head and tails of pip, easy_install and virtualenv and it still does less than Maven.

>And lambda expressions are lame compared to what you can do in JavaScript/Perl/Scala.

Python lets you pass functions other ways than using lambdas, you know. If you've defined a function with the standard "def foo" style, you can pass "foo" as an argument.

Even the standard "1-line lambdas LOL" limitation can be circumvented by defining the function elsewhere and making the lambda a call to it.

Re: Python at Netflix

#57
post #24
post #5

Earlier quoted context omitted.

Keep it up! Posts like these are great on a tech level, but they're also good recruitment - Seeing Python, along with hackers that I respect (like you) at Netflix make me at least the littlest bit tempted to go there ;)

You've caught on to our ploy! But in all seriousness, we like to share what we do because sharing is cool and it attracts other people who like to share cool technology, which are the kind of people we like to work with.

I strongly suspect preventing me from visiting PyCon is one of the reasons my CTO scheduled so many product launches for the next two weeks...

Re: Python at Netflix

#58
post #51
post #27

Earlier quoted context omitted.

I like Python for the speed of development. Also, at least for these internal apps, speed is generally of secondary concern, since these aren't typically high load applications. At reddit we solved the problem by writing the most often called parts in C and using c extensions. There is definitely more work to do there, but overall I think Python gets a bad rap as far as speed is concerned. It's certainly not the quic…

> At reddit we solved the problem by writing the most often called parts in C and using c extensions. I'm wondering if anyone has solved these sorts of problems with Cython in the real world. It looks like a great solution for a large set of problems in terms of Python performance, since it becomes a CPython extension, but I never hear anyone talk about it. Can anyone chime in?

I get the impression that Cython is used quite a bit in the scientific computing side of Python, but not so much for web development. pandas & pyzmq are two projects I know that make serious use of Cython.

Re: Python at Netflix

#59
post #9

Earlier quoted context omitted.

You should also come by our booth (Continuum Analytics) and we can talk shop!

We were actually talking about your company in our sprint planning meeting a few minutes ago, since we're doing in-house Python analytics. You might be getting a call.

Awesomeness. We are making great headway on Blaze and the new persistence format to supplant HDF5/PyTables (Although, Blaze will always work great with carray, NumPy, and Pytables), and the Numba compiler project is working out great.

Wakari (http://wakari.io) is nearing 1.0 and we're about to turn on paid monthly plans for beefier nodes and GPU nodes, and we have also been in talks with several companies about in-house private deployments.

Give us a shout or shoot me an email at pwang (at) continuum.io! :-)

Re: Python at Netflix

#60
post #2

As one of the main drivers of Python at Netflix, this one is near and dear to my heart. Let me know if you have any questions, or come visit our booth!

Where can I find more info about Sting? I didn't see a link in the article and some quick Google searches haven't proved fruitful.

Kurt Brown had some slides about data & Python at the NYC Python meetup last October. He did a Sting demo there and some of the slides talk about it (I think... it might be a different thing, though. October was a long time ago..)

http://www.meetup.com/nycpython/files/

Post reply on HN