Live data from Hacker News

Problems I Have with Python

darkf.github.io

51–60 of 239 posts

Re: Problems I Have with Python

#51
post #4

Some of the author's points are valid. However, many are subjective preferences, and some are gripes without solutions, and others make it difficult to understand the author's underlying philosophy. My main critique is that the author added this statement that puts a negative, entitled, and naive tone on the whole article: >>> to which no real improvements are being made for some reason. (Incompetence? Politics? Both…

the solutions to a problem (I'm thinking GIL) come with a lot of consequences which are not readily acceptable

Are not readily acceptable to Guido. Whether that's a good thing (for the language) or not is subjective. At least Guido has a vision and is sticking to it!

Re: Problems I Have with Python

#52
> The standard interpreter bring rather slow;

I'm tired about this one.

In the last 13 years, 97% if the projects I worked on didn't need Python to be any faster, it was not the bottleneck. The remaining ones could leverage some solution to bypass the problem. Python speed is indeed an issue to a few people, but it's not the red flag I can read about here and there.

I've been hearing this argument for ever. PHP is slow. Java is slow. The first one powered the Web for 10 years, the second one is the most used language is the world. A lot of time this argument is like hearing "I want a pony".

Actually the rare persons I met really needing speed never complained. They are usually hardcore professionals, and are already working on solutions.

Let's now talk about solutions.

Python is an interpretted and very dynamic language. It's though to speed up. If you look at the C code, you'll see the Python VM is quite well optimized already.

Now the author says:

> no real improvements are being made for some reason

But there have been:

- psyco

- unladden shallow

- stackless

- numpy and a lot of compiled extensions

- pypy

- pyston

- pyjion

- nuikta

- cython

- numba

People ARE actively working at the problem. It's a HARD problem which is why we don't have yet a definitive solution. And a lot of people working on it are non paid for this.

Yeah, JS became faster. You know how ? Google spent millions and hire a bunch of geniuses just to do it.

In 2011, the Python Software Foundation had ‎$750,000 to spend for the whole operation, including maintaint pypi, the documentation, the official website, the conferences they do and the various grants they provide. Even the few dev that are paid to work on Python (e.g: Guido) have to do it only part time.

So the authors worked with Python for 10 years. He made a living out of a free exceptional software and complain about a problem he may even doesn't have while people are working their ass off to solve it. And we writes an aggressive rant about it.

> Parallelism is very bad on CPython and PyPy

Yes, again, this is a HARD problem. Python is very old. Older than Java. We only had multi-core recently. We can't destroy mono-core perfs to get multi-core, and have to mainteaint a legacy code base.

We also have:

- a good multiprocessig story;

- 2 good async stories;

- tooling to pre-spaws, manage and scales processes;

- tooling to create task queues.

So while the community is trying, for free, to solve the problem. We have solutions. It's not perfect. But again what's the point of complaining like an hungry child à 4 o'clock unhappy it's not yet dinner time ?

> asyncio does not seem very well integrated, and does not seem as useful as libraries like eventlet. They seem to have wanted to reinvent Twisted, but did so half-assed and did not include useful protocols (Twisted has line-based protocols, HTTP, etc. built in and easily subclassable.)

What is he talking about we just got it ? How do you expect it to be well integrated yet ?

And Twisted is a framework (a very hard to use one) while asyncio is a low level lib.

eventlet doesn't let you choose where to switch context, it's basically like threads. We already have threads.

> Quite a few legacy projects are written in Python 2, and it can take some work to port them. This is particularly a pain for libraries where I expect to pip install them and have them "Just Work".

When the last time didn't that happen for anybody ?

Seriously:

http://py3readiness.org/

I've been coded in Python 3 for the last 2 years. It happened twice. Both time I was able to convert the code base in a few minutes. I said minutes. Not hours.

> There is an official tool 2to3 which does not work in all cases.

And the break in your car doesn't work in all cases either. Still it's a nice break.

Plus you got six and Python-future. Converting any pure-python code base is not hard. Compiled extension is harder, but my guess the author never needed to code one.

And i'll say it again...

People have 15 bloody years to migrates. It's not like JS tools breaking every 3 months. It's not like PHP skipping the version 6 or Perl taking 10 years to get V6.

No. Python 3 arrived quickly after many warnings. Then tools, tutorials and a looooooooooooot of time have been provided.

This is nowhere Python's fault. It's the best damn migration story I've ever witnessed in my life.

My only grudge on Python 3 is that it didn't break ENOUGH. I wished for stuff to have changed more.

> The standard library is sometimes inconsistent

One of my pet peeves as well.

> The BDFL himself, Guido van Rossum, has infamously declared that he does not like functional programming (odd, considering the language is built around FP concepts), and that map/reduce/filter should not be in the language. Well -- in my opinion that is a grave mistake, but more importantly the language suffers.

The author doesn't like the style of the language. So it's a matter of taste.

Well I like it that way.

Now what ?

> reduce is now tucked away inside the functools module (as of Python 3), even though it is the only one of map/filter that is not replaceable by list/set/dict comprehensions! Yet map and filter are still in the base global environment. What sense does that make?

Yes it does because reduce is seldom used. Grep github and you'll see. map and filter are still in the built-ins because people like the author complained a lot on the mailing list.

Yet, the majority of code base I read, including most of the libs I use (I spend a lot of time reading the content of my site-packages) don't use map/filter since we got comprehensions.

> I often find myself reimplementing flatten as flatten = lambda xs: itertools.chain.from_iterable(*xs)

Use comprehensions to flatten. Learn you language for van Rossum' sake !

(y for x in xs for y in x)

> The lack of tail call optimization in most implementations makes writing tail recursive algorithms rather pointless, unfortunately, even when they may be more legible than their iterative counterparts.

> There is no standard way (even in functools) to compose functions. There is partial application via functools.partial, at least...

Again it's because recursivity is not encouraged in Python. It's the philosophy of the language. One can dislike it but it's not a Python problem, it's a Python decision.

I stay in Python precisely for this. Everytime I go read functional heavy code, it's hard to read. I'm an expert coder and trainer. I'm paid up to 900€/day. Most code should be easy to understand given my experience. When it's not, I consider that a bug.

Functional lovers write smart code. I hate reading smart code. I want code that is easy to debug.

If you really need TCO, like when implementing a state machine, there are solutions:

http://neopythonic.blogspot.fr/2009/04/final-words-on-tail-c...

Not as elegant, but good enough since it's a rare occurence you do need it. Again. Rare.

The language is optimized for regular use cases and readability, not smart formulas.

> Lambda is awful

Lambda is wonderful. It keeps people from writting budge inline callback like they do everywhere else. It's the best decision Guido every took.

Xith lambda + decorators + list comprehension, the need for multi-lines callbacks is not huge.

You want more ? Write a regular function. How hard is it ?

It's not hard. So eventually it's matter of...

... wait for it ...

taste.

I would have liked a shorter keyword though. But I can live with it.

> Inadequate data modelling facilities

"Inadequate data modelling facilities" because classes are verboses ? Overkill title much ?

Beside, if you just need a container, you use a dict in Python. Not a class. At most you use SimpleNamespace:

>>> from types import SimpleNamespace >>> SimpleNamespace(a=1, b=True) namespace(a=1, b=True)

But again this is "pony"-worth complaining.

I do think classes are too verbose in Python (I use the attrs lib because of this). But this is childish.

algebraic data type and the whole dunder methods vs interfaces are more interesting debates.

> Lack of switch (or match)

> No, dicts with lambdas (see above) are not a replacement. No, long if-else chains are not a replacement. I want a nice way to match on data (preferably richly -- as with ADTs, ranges, ...) and associate matches with logic

Yes they are for switch. Since is the most overrated statement after go to. It's uneeded, as you can express it's logic perfectly without it. And again, "rare use case". There is nothing wrong with a bunch of if or a dict.

Now for match, it's a different story. Pattern matching would be a nice addition for Python IMO. But again things like:

> Please do not suggest awful hacks to do this, and fix your language instead.

Is arrogant and ignorant.

The mailling list have been discussing it for years. It hasn't happen because there no such thing as a magic way to make everybody agree then implement it and maintain it for free.

Things have cost. People have taste. Code base have legacy requirements.

Re: Problems I Have with Python

#53

The problem I have with Python is that for loops don't have their own scope, only methods. Add that to the lack of variable declarations (even optional ones, a la my in perl, var in javascript), and it gets hard to work out what scope of any given variable actually is. Surprising example: fns = [] for n in [1,2,3,4]: def fn(): print(n) fns.append(fn) for fn in fns: fn()

The obvious solution to that is to break your code into smaller functions.

Re: Problems I Have with Python

#54
post #12

I love how so much person focus on the GIL and multithreading, when GIL is much more a solution to make un-threadsafe libs safe to use, and that most people don't see POSIX threads are an inherently broken abstraction. [1] http://www.daemonology.net/blog/2011-12-17-POSIX-close-is-br... In fact it pretty much boils down to signals being broken on unices [2] https://lwn.net/Articles/683118/ Which even though I have a h…

The GIL doesn't magically make un-thread safe code thread safe. It makes Pythons reference counting implementation thread safe.

Re: Problems I Have with Python

#56
post #19

Python's semantics are unlikely to ever be fast and most Python users have already worked around its speed issues. asyncio is new; python3 porting is happening. It seems unfair to complain that no real improvements are being made and also complain that these new things are immature. reduce being pushed behind an import is stupid, but it's only one import. lambda is fine, if you're writing in functional style you're u…

Thank you for showing me attrs.

I love namedtuple, but this seems like a better implementation for light-weight classes.

Re: Problems I Have with Python

#57
post #6
post #4

Some of the author's points are valid. However, many are subjective preferences, and some are gripes without solutions, and others make it difficult to understand the author's underlying philosophy. My main critique is that the author added this statement that puts a negative, entitled, and naive tone on the whole article: >>> to which no real improvements are being made for some reason. (Incompetence? Politics? Both…

>However, many are subjective preferences Certainly, it is titled "Problems I Have" for a reason. :-) I do not expect everyone to agree with me, but it is what I feel I personally lack when using it quite a lot. > I imagine this statement could offend some of the smart and hard-working people who are working on improving the python language. That was certainly not my intention -- as stated, I do love the language and…

> I did not propose a solution because there are many, as you note; there are, however, implementations with decent solutions like AFAIK Jython.

There are no solutions that satisfy everyone that I am aware of yet. Guido has said in the past that he'd be happy to get rid of the GIL, and would merge a patch that solves it, as long as:

* It does not reduce the performance of single-threaded Python code.

* It stays compatible with all existing pure Python code and C extensions.

But in practice, GIL is not that much of an issue for many types of applications where Python is popular.

* It's not an issue for web apps, because these are typically served from multiple physical servers each running multiple python processes. These do not share a GIL anyway, and "thread safety" is pushed to database transactions.

* It is not an issue for apps which spend most of the time doing I/O. Most IO libraries release the GIL, and other threads can run while you're waiting for results from the database.

* It is not an issue for data science doing heavy number crunching with numpy and everything built on top of numpy. Numpy releases the GIL while doing large computations in C.

* It is not an issue for small scripts, as a "better than Bash".

The GIL is only an issue for apps that do heavy computation in pure Python code, and need parallelism within a single process (socket servers? text data processing?). As a result, many Python users just don't find it a big enough problem to be worth solving, if the solution comes with downsides for their use cases.

Re: Problems I Have with Python

#58

Earlier quoted context omitted.

That's really helpful. Can you make an example which doesn't require making the my_handler function 'async'? For cases when you don't want to make an entire async stack, you just want to slot some async code into your existing code. If the answer is "to get some async goodness, just use this easy code, plus rewrite your entire project to use a different framework and set of libraries", then we are only fooling oursel…

You need to use the asyncio (or equivalent) event loop if you want to use the asnycio module. loop.run_until_complete() is synchronous though, so you would simply call that and it will block the control flow despite that function being async. You can definitely mix it with legacy code. I would recommend against it, but if you had an existing framework, you could just make the endpoints lambdas that are something like…

^ this, but be aware that it's only worth it if you do > 1 external call in parallel. I.E this is pointless:

    res = await get('https://somesite.com')
    return Response(res['data'])
As you'd get the same thing if you just did it synchronously (without the await). But if you want to fetch 2 or more pages in parallel when it really pays off.

Re: Problems I Have with Python

#59
post #41
post #37

Earlier quoted context omitted.

That seems like an exceedingly error-prone thing to write. "x +=1" isn't a value and the unmanaged mutation will be surprising when it happens. On a Python implementation with parallelism (e.g. Jython) you could very easily end up losing updates - on CPython the GIL will probably mean your code accidentally doesn't exhibit that problem, but that doesn't seem a very desirable way to code.

What is the proposed alternative? (using a named function or method just seems to be semantically the same, just more characters)

Something like functional reactive programming style, where you explicitly define a pipeline from foo.click to x and explicitly gather together all your pipelines (explicitly defining the interleaving semantics rather than just "whenever an event happens it happens") and run them in one place.

Re: Problems I Have with Python

#60

The problem I have with Python is that for loops don't have their own scope, only methods. Add that to the lack of variable declarations (even optional ones, a la my in perl, var in javascript), and it gets hard to work out what scope of any given variable actually is. Surprising example: fns = [] for n in [1,2,3,4]: def fn(): print(n) fns.append(fn) for fn in fns: fn()

But doesn't this just happen because n is a pointer? What would you expect it to print? 1,2,3,4?
Post reply on HN