Python Is Eating the World • Hacker News Board

Re: Python Is Eating the World

#791

Post by jerf » Mon, Aug 12, 2019, 5:25 PM UTC

6gvONxR4sf7o wrote: post #777

jerf wrote: post #672

Earlier quoted context omitted.

Python needs FFI, because its native performance is fairly poor [1]. In general, Go gets away with less FFI because it's fast enough that it doesn't need things to be implemented in C to run quickly. This is especially true if you consider "Go" as "Go + its ASM", which is probably what you'd want if you think of it in terms of science programming. For another example of a similar effect, Rust has great FFI. Yet I wou…

It's not just about speed. Great mature libraries have been written in C over the decades. Rewriting them in a new language is a herculean effort. Writing bindings to them lets you use the fruit of all that labor.

What you say is true, yet, I observe that people undertake that herculean effort with some frequency.

Whether that's a good idea, and why that is, those would be entirely separate conversations. But it's just an observable fact that languages pick up native implementations of core functionality over time, subject to certain performance restrictions (e.g., I'm sure that if it wouldn't be unusably slow, Python would have a native-Python image library... it's just Python doesn't really have that option).

Re: Python Is Eating the World

#792

Post by TylerE » Mon, Aug 12, 2019, 5:25 PM UTC

seisvelas wrote: post #761

ghc wrote: post #642

Earlier quoted context omitted.

Python's actual competitors are Ruby, Perl, R, Shell, Visual Basic, Javascript, PHP, and Matlab. Nobody's going to bother out OCaml or Lisp for web development, data science, or OS scripting where Python is most often used.

Python and Lisps do directly compete as the preferred introductory language for university computer science classes.

In this decade?

Not even MIT teaches lisp anymore.

Re: Python Is Eating the World

#793

Post by tsm » Mon, Aug 12, 2019, 5:25 PM UTC

PaulMest wrote: post #782

takeda wrote: post #728

Earlier quoted context omitted.

Python is a dynamic language, that's what dynamic languages do, you don't have a type checker but have greater flexibility, but you don't have to settle on that, you can actually use mypy and annotate types and get best out of both worlds. > And another one, I typoed a variable name as groups_keys instead of group_keys (or vice-versa, I don't remember). Instead of just throwing an error, Python happily went along wit…

I'm pretty sure this is what they meant... def my_func(): group_keys = 'My thing' while group_keys == 'My Thing': if some_logic() == 42: groups_key = 'My other thing' # typo here

I don't see how that's avoidable in any language that doesn't require explicit variable declaration

Re: Python Is Eating the World

#794

Post by CydeWeys » Mon, Aug 12, 2019, 5:27 PM UTC

joshuamorton wrote: post #753

CydeWeys wrote: post #701

Earlier quoted context omitted.

I didn't explain the second one well. Here's some exact code. group_keys = ... if not isinstance(group_keys, list): groups_keys = [ group_keys ] So rather than listifying the non-list variable, it was creating a new variable. The cause of this bug is that Python doesn't distinguish between declaring new variables and overwriting existing ones.

FYI, the google style guide (or maybe the internal only version) suggests to avoid initialize-then-assign in favor of single assignment form: unclear_type_thing = ... if isinstance(unclear_type_thing, list): group_keys = unclear_type_thing else: group_keys = [unclear_type_thing] statically avoids this problem. In general, prefer immutable variables where possible. Single-assignment form is nice for a lot of reasons,…

All of these things are true, but they require a non-trivial level of experience and discipline to avoid most potential gotchas. Your average Python project on the Web isn't written to this level of quality, and when people are learning programming using Python in school they certainly aren't there yet, and are gonna hit all kinds of problems related to this stuff.

But is there a way to force immutable variables in Python? You can easily still end up in the same situation when you typo something (easy to do when plurals are involved), and then end up reassigning something when you meant to create a new variable.

Re: Python Is Eating the World

#795

Post by SEJeff » Mon, Aug 12, 2019, 5:28 PM UTC

NotSammyHagar wrote: post #130

advanceduser wrote: post #73

Earlier quoted context omitted.

You're gonna flip when you hear about the environmental impact of a developer.

just imagine using python of cryptocurrency mining.

Or javascript in the browser via malicious js files from hacked ad sites!

Re: Python Is Eating the World

#796

Post by CydeWeys » Mon, Aug 12, 2019, 5:28 PM UTC

cc81 wrote: post #727

mnky9800n wrote: post #706

Earlier quoted context omitted.

Maybe they could write unit tests to make sure what's being passed is lists and strings. But that's probably crazy.

That would not catch the bug if the input is not under is control. You could as well say "Just check if the object is a string" in the method, which would work but the point was rather that it is difficult to notice if you did not think about it. Compared to other languages that would crash or not compile instead.

Yeah, the input isn't really under control because it's coming from deserializing a YAML file. It worked for the exact type of input I was expecting, namely, when you configure a specific value as a list, but it wasn't working for anything else. And YAML has plenty of types it can split out, so my naive fix still only handled lists and strings properly!

Re: Python Is Eating the World

#797

Post by CydeWeys » Mon, Aug 12, 2019, 5:30 PM UTC

pytester wrote: post #710

CydeWeys wrote: post #685

Earlier quoted context omitted.

Here's some rational "hate" for Python then. I just returned to Python for the first time in a little while to collaborate on a side project and ran into a few tricky-to-debug errors that caused a fair bit of lost time. Know what the errors were? In one case, I was iterating over the contents of what was supposed to be a list, but in some rare circumstances could instead be a string. Instead of throwing a type error,…

>In one case, I was iterating over the contents of what was supposed to be a list, but in some rare circumstances could instead be a string. Instead of throwing a type error, Python happily went along with it, and iterated over each character in the string individually. I've been using python for about 13 years professionally and I wrote up a list of "things I wish python would fix but I think probably never will" an…

Iterating over characters in a string is something that's done very often in introductory CS classes, but very little in the real world. Python has support for string finding and regexes; why in the world would I be individually iterating over characters? Generally, when you see that, it's a code smell.

So yeah, I totally agree with you, it'd be better if trying to iterate over a string were a flat-out error, and if you really want it, you should mean it. Though Python being dynamic still means that you'll only spot this error at runtime.

As for linters, how do they know if your intent was to reassign the value of an existing variable, or to define a new one? The language has no way to indicate which of these is intended.

Re: Python Is Eating the World

#798

Post by blub » Mon, Aug 12, 2019, 5:31 PM UTC

noelwelsh wrote: post #391

blub wrote: post #294

Earlier quoted context omitted.

Over-sensitive individuals are hard to please and often unhappy. That's more of a personality flaw than a flaw with the current state of software engineering. If there's one thing wrong with our profession is a lack of ethics and accreditation - we're essentially letting random people build critical infrastructure and utilities. We don't have a tooling problem, in fact we have too many tools. I see so many people (es…

There are objective differences between languages. Examples include performance and compile-time checking. These differences are not irrelevant. Alternatively, if you believe the differences between languages are irrelevant why do you not program everything in assembler?

Those differences don't matter much when it comes to building software that fulfills specific requirements.

I am a big fan of compile-time checking, but there's a lot of good software built without it and sadly there's also a lot of successful slow software. These are disadvantages, not an impassable barrier.

Re: Python Is Eating the World

#799

Post by p1esk » Mon, Aug 12, 2019, 5:32 PM UTC

ernst_klim wrote: post #352

p1esk wrote: post #290

Earlier quoted context omitted.

Probably yes What does it mean? Have you done it in any of those language? Have you seen it done in any of those languages?

> What does it mean? Have you done it in any of those language? I did. I'm doing a image processing recently and use OCaml for prototyping. I've tried python (I've used it a lot for that long time ago), I've failed, it felt to awkward. I've described my experience here [1] If you have no experience whatsoever with ML family [2], and doing all the stuff in python, you'll most likely be much more productive with python…

Right now I’m experimenting with a pretty complicated model (60+ layers of multiple types), and I plan to train it on several hundred GB of data, using 8-16 node cluster (4 GPUs per node). Does Owl have a well tested and well documented autograd library with distributed GPU support (e.g. Horovod)? With a good choice of optimizers, regularizers, normalizers, etc, so I can focus on my model and not on debugging the plumbing or implementing basic ops from scratch. And last, but not least, it must be as fast as TF/Pytorch.

If the answer is “no”, then it does not matter whether I’m an OCaml expert, because I’m still going be more productive with Python.

p.s. Julia is nice though, hopefully it will keep growing.

Re: Python Is Eating the World

#800

Post by coldtea » Mon, Aug 12, 2019, 5:33 PM UTC

swat535 wrote: post #736

coldtea wrote: post #684

Earlier quoted context omitted.

> You need to test anyway. Actually, if your type system is powerful enough, you don't need to test. That's the source of the "if it compiles, 99% of the time it works right" people mention about Haskell (and even more so languages like Idris etc). Type systems are tests -- just formal and compiler-enforced, not ad-hoc "whatever I felt like testing" tests, like unit tests are. From there on it's up to the power of th…

> Actually, if your type system is powerful enough, you don't need to test. That's the source of the "if it compiles, 99% of the time it works right" people mention about Haskell (and even more so languages like Idris etc). Types only eliminate certain tests. You will always have system tests, acceptance tests and unit tests. One should use types to augment their system reliability. Types will not catch logical error…

>Types only eliminate certain tests. You will always have system tests, acceptance tests and unit tests.

Yes, so let's eliminate them with types, instead of doing them. "Acceptance tests" are not concerned with programming.

>Types will not catch logical errors in your code.

Actually, depending on the type system, it will.

That's how program logic is tested as "proof" and programs, implementations of algorithms are determined to be logically correct in more exotic languages (but even in C + some restrictions + the right statically checking tooling, NASA/JPL style project do that).

https://en.wikipedia.org/wiki/Formal_verification