Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Yes, Python is Slow, and I Don’t Care

hackernoon.com

Post reply on HN
Print view View original

171–180 of 206 posts

Re: Yes, Python is Slow, and I Don’t Care

#171

Post by flavio81 »

scarface74 wrote: post #20

I was all ready to savage his opinion after reading the headline but I agree looking at my architecture that I designed for the company I work for, CPU isn't the bottleneck. Every time I try to increase performance by multi threading as much as possible, the databases start screaming. On the other hand, the idea that dynamic languages are more productive than static languages are laughable. Statically type languages…

>Statically type languages prevent a lot of bugs

Correction:

Statically typed languages prevent a lot of the bugs that total beginners to programming would make; while they are useful in preventing the one, two or tree trivial, easy to solve bugs that a seasoned programmer would make once in a week.

They are also not useful at all to prevent the real-life, critical bugs that have nothing to do with types at all, and that are the ones that will negatively impact the product the most and will take serious time to get fixed.

You know what static typing is good for? It has nothing to do with "preventing bugs". It has to do with performance and compiler optimizations.

Re: Yes, Python is Slow, and I Don’t Care

#172

Post by contravariant »

traverseda wrote: post #127
contravariant wrote: post #70

Earlier quoted context omitted.

A static type system doesn't protect you from choosing the wrong abstraction, and all the bugs that result from it. But if used properly I think it does help.

How? I don't think I've ever gotten a type I wasn't expecting in any of my python code. It's just not a problem for me. That is, past the first time I run it. I've been known to pass something an x when it wanted a y, but that's a "fail early, fail loud" bug and not an actual problem.

This just means that you already have an idea of what kind of object should go where, you just never wrote it down. And because you never wrote it down it becomes hard to generalise things when you eventually want to reuse some method, since you never made explicit which operations an object should support to be able to reuse that method.

Also you have no guarantee that what you think the types should be is actually consistent.

Re: Yes, Python is Slow, and I Don’t Care

#173

Post by pg314 »

classybull wrote: post #88
pg314 wrote: post #54

Earlier quoted context omitted.

> I'm curious who or where the author heard that from (not specifically the people themselves but the domain they are in). In the telecom domain, I've dealt with data big enough that Python wasn't really feasible. Think 100 of millions of records in CSV format that need to be parsed and processed. Doing that in Python is going to be painful.

Python is insanely fast at data processing and analysis because it has very fast libraries. As a matter of fact, don't know if you've heard, but data processing it kind of like.. Python's thing...

> Python is insanely fast at data processing and analysis because it has very fast libraries.

It doesn't have fast libraries for everything. E.g. for the use case I brought up, it doesn't (or at least none that I could find at the time). Not all data are homogenous arrays that fit in numpy.

And are those libraries written in Python? Or in C because Python is too slow?

If you take your reasoning, any language that can bind to C (which is pretty much any language) is as fast as C. That is not very helpful when comparing the speed of languages. A slower language will force you to drop into C sooner than a faster one.

> As a matter of fact, don't know if you've heard, but data processing it kind of like.. Python's thing...

Things like audio and video codecs, or crypto code aren't written in Python.

Re: Yes, Python is Slow, and I Don’t Care

#174

Post by dragonwriter »

flavio81 wrote: post #171
scarface74 wrote: post #20

I was all ready to savage his opinion after reading the headline but I agree looking at my architecture that I designed for the company I work for, CPU isn't the bottleneck. Every time I try to increase performance by multi threading as much as possible, the databases start screaming. On the other hand, the idea that dynamic languages are more productive than static languages are laughable. Statically type languages…

> Statically type languages prevent a lot of bugs Correction: Statically typed languages prevent a lot of the bugs that total beginners to programming would make; while they are useful in preventing the one, two or tree trivial, easy to solve bugs that a seasoned programmer would make once in a week. They are also not useful at all to prevent the real-life , critical bugs that have nothing to do with types at all, an…

> They are also not useful at all to prevent the real-life, critical bugs that have nothing to do with types at all,

Most bugs are about types, just most languages’ type systems are too weak to express the logical types relevant to many important bugs (and, if they weren't, then you'd probably have to worry more the about bugs in the type-level code.)

Re: Yes, Python is Slow, and I Don’t Care

#175

Post by flavio81 »

scarface74 wrote: post #169
abhirag wrote: post #59

Earlier quoted context omitted.

"the idea that dynamic languages are more productive than static languages are laughable." -- being statically typed or dynamically typed comes with its own set of tradeoffs and what a person is more productive in is a highly subjective matter. Lispers are more productive in Lisp than Haskell and vice versa. "Statically type languages prevent a lot of bugs and allow for a lot of automated provably correct refactoring…

Clojure is trying to do that with Clojure.spec and specifications being checked at runtime can get you closer to things you could have automatically proved correct only with languages with dependent types, nothing against statically typed languages but I feel that your sweeping generalizations hurt the point you are trying to make. "Checking at runtime* is exactly the problem. Why would checking at runtime be more re…

>"Checking at runtime is exactly the problem. Why would checking at runtime be more reliable than compile time?*

If you use metaprogramming (i.e. Lisp macros, Scheme/Racket macros, Clojure macros), then the only way to have truly reliable checks is at run time.

In fact, for debugging and testing, it's much better to have a runtime able to do many things (such as live patching of the code).

Contrary to a language like C or C++, where the compiler produces machine language from the source and then goes out of the picture, in most of the Lisp family languages your program runs along with the runtime; this runtime also contains the compiler itself and is usually big on features; it will not just catch the type errors, it will show you exactly all kind of important information about any, error and will allow you to correct the error if you like, without having to recompile and start again.

Re: Yes, Python is Slow, and I Don’t Care

#176

Post by pg314 »

joshuamorton wrote: post #154
AstralStorm wrote: post #149

Earlier quoted context omitted.

By blazingly fast you mean 100x slower than C++ equivalent and only 20x slower is you're very careful to avoid accidental copies. For reference, MATLAB is about 30x slower with no special care. Pure Java on Hotspot was 5x slower except it dies on big data input due to very slow GC and goes to 50x slow. Source: handled big audio data from hdf5 database, gigabytes sized. C++ equivalent had no vectorization or magic BLA…

As I'll often say to these comments, then you're doing things wrong. Numpy code can be written to never leave the numpy sandbox, and at that point it should be as fast or faster than naive c++ (because you'll be getting SSE and stuff for free). There's a reason almost all deep learning is done in python.

Not all data is a good fit for Numpy: some data is non-numeric or not a homogenous array.

> There's a reason almost all deep learning is done in python.

The heavy-lifting in e.g. TensorFlow is done in C++. Bindings to Python make sense because it is one of the few sanctioned languages inside Google, and it is widely used outside of Google and easy to pick up.

Re: Yes, Python is Slow, and I Don’t Care

#177

Post by flavio81 »

dragonwriter wrote: post #174
flavio81 wrote: post #171

Earlier quoted context omitted.

> Statically type languages prevent a lot of bugs Correction: Statically typed languages prevent a lot of the bugs that total beginners to programming would make; while they are useful in preventing the one, two or tree trivial, easy to solve bugs that a seasoned programmer would make once in a week. They are also not useful at all to prevent the real-life , critical bugs that have nothing to do with types at all, an…

> They are also not useful at all to prevent the real-life, critical bugs that have nothing to do with types at all, Most bugs are about types, just most languages’ type systems are too weak to express the logical types relevant to many important bugs (and, if they weren't, then you'd probably have to worry more the about bugs in the type-level code.)

> Most bugs are about types, just most languages’ type systems are too weak to express the logical types relevant to many important bugs

Ok, so you are writing a function for a fast approximation of ATANH using IEEE floating point standard.

In type: IEEE floating point number

Out type: the same.

The function will report an incorrect value (bug), yet still be perfectly type safe.

Tell me how the magical type checking of the super-good-type-system programming language you propose will catch the error and thus prevent the bug.

And also, we're discussing about real-life problems, tackled with existing programming languages, not languages that don't exist yet.

Not even Haskell, with it's great type system, will help you in such cases. Not in the bugs that are caused by a flawed understanding of the business model, the business rules, or of the system as a whole.

Re: Yes, Python is Slow, and I Don’t Care

#178

Post by [deleted] »

flavio81 wrote: post #175
scarface74 wrote: post #169

Earlier quoted context omitted.

Clojure is trying to do that with Clojure.spec and specifications being checked at runtime can get you closer to things you could have automatically proved correct only with languages with dependent types, nothing against statically typed languages but I feel that your sweeping generalizations hurt the point you are trying to make. "Checking at runtime* is exactly the problem. Why would checking at runtime be more re…

>"Checking at runtime is exactly the problem. Why would checking at runtime be more reliable than compile time?* If you use metaprogramming (i.e. Lisp macros, Scheme/Racket macros, Clojure macros), then the only way to have truly reliable checks is at run time. In fact, for debugging and testing, it's much better to have a runtime able to do many things (such as live patching of the code). Contrary to a language like…

[deleted]

Re: Yes, Python is Slow, and I Don’t Care

#179

Post by kazinator »

scarface74 wrote: post #169
abhirag wrote: post #59

Earlier quoted context omitted.

"the idea that dynamic languages are more productive than static languages are laughable." -- being statically typed or dynamically typed comes with its own set of tradeoffs and what a person is more productive in is a highly subjective matter. Lispers are more productive in Lisp than Haskell and vice versa. "Statically type languages prevent a lot of bugs and allow for a lot of automated provably correct refactoring…

Clojure is trying to do that with Clojure.spec and specifications being checked at runtime can get you closer to things you could have automatically proved correct only with languages with dependent types, nothing against statically typed languages but I feel that your sweeping generalizations hurt the point you are trying to make. "Checking at runtime* is exactly the problem. Why would checking at runtime be more re…

A run-time check will work even if the compiler has a bug which mistranslated the code or neglected to do a check.

Re: Yes, Python is Slow, and I Don’t Care

#180

Post by kazinator »

flavio81 wrote: post #175
scarface74 wrote: post #169

Earlier quoted context omitted.

Clojure is trying to do that with Clojure.spec and specifications being checked at runtime can get you closer to things you could have automatically proved correct only with languages with dependent types, nothing against statically typed languages but I feel that your sweeping generalizations hurt the point you are trying to make. "Checking at runtime* is exactly the problem. Why would checking at runtime be more re…

>"Checking at runtime is exactly the problem. Why would checking at runtime be more reliable than compile time?* If you use metaprogramming (i.e. Lisp macros, Scheme/Racket macros, Clojure macros), then the only way to have truly reliable checks is at run time. In fact, for debugging and testing, it's much better to have a runtime able to do many things (such as live patching of the code). Contrary to a language like…

> If you use metaprogramming (i.e. Lisp macros, Scheme/Racket macros, Clojure macros), then the only way to have truly reliable checks is at run time.

That is a misconception. The macros output a target language and that can be checked. Plus macros can do their own checking before that. A macro expander can perform some static checks. In TXR Lisp, warnings about unbound functions and variables come from the macro-expanding code walk.

Post reply on HN