Live data from Hacker News

Why Is the Migration to Python 3 Taking So Long?

stackoverflow.blog

301–310 of 355 posts

Re: Why Is the Migration to Python 3 Taking So Long?

#301

Earlier quoted context omitted.

It doesn't have to treat "self" as a special keyword - it just has to desugar "foo.bar(x, y)" into "bar(foo, x, y)". There are some other languages that do that - e.g. in F#, you write: member this.foo(x, y) = ... Again, "this" is just an identifier here, and doesn't have any special meaning. It's more elegant firstly because it follow use, and secondly because it means that "def foo(x)" has the same meaning both ins…

> As it is, we need stuff like @staticmethod and @classmethod You would need those even with your suggestion (except you could drop @staticmethod if you add another layer of magic so that methods declared without an leading identifier were assumes static; you'd still need @classmethod some equivalent mechanism to distinguish which non-static methods were class vs instance methods.)

My suggestion implied removing the descriptor-based "magic" on regular functions that make them behave as methods when accessed using dot-member syntax, and instead making the "def self.foo" syntax produce a special kind of function that would have that behavior. @staticmethod today basically just suppresses that special behavior on regular functions, so it wouldn't be needed in this case. But yeah, we'd still need @classmethod.

Re: Why Is the Migration to Python 3 Taking So Long?

#302
post #272
post #133

Earlier quoted context omitted.

> On the other hand, they could progressively enhance upon a backwards compatible single 2 version. JS manages to do that just fine, as does Java... Common Lisp has a backwards compatibility which goes into decades, and implementations like SBCL had no difficulties at all to absorb Unicode. Racket even supports different language standards and completely different languages (such as Scheme and Algol) running on the s…

> It is, however, a huge problem for domains like scientific computing, where most code has no maintainers and even for very important code there is no budget or staff for maintenance: Data science is doing just fine, in fact is leading the migration: https://www.jetbrains.com/research/python-developers-survey-...

"data science" and "science" are quite different things. Science is the systematic and collaborative pursuit of knowledge in a long-term endeavour. It is based on sharing and open exchange of methods and tools. Numericas, mathematics and computational codes are just important tools to do that. As Hinsen in the blog post I cited above points out, the most part of important computational codes is written in one-off research projects which go for a few years, and the people who develop these codes normally have to move on and work for a different institution, if they manage to keep working in science. On the other hand, important codes and algorithms may be used for many many years.

"Data science" is a broad term but usually just means the application of numeric, and sometimes scientific, tools to commercial means. It is almost always done in companies. Typically, between such companies there is no open exchange of tools and methods, no exchange of knowledge, and no long-term use of generated codes. This is the reason why data science companies don't have the problems which Hinsen pointed out. But, they could become affected by a degrading suitability of Python for computational science, because their tools were initially developed by scientists.

Re: Why Is the Migration to Python 3 Taking So Long?

#303
post #276
post #215

Earlier quoted context omitted.

> Actually that's the behavior of python 2, it works fine, until you send invalid characters then it blows up. Not always. As far as I can tell writing garbage bytes to various APIs works fine unless they explicitly try to handle encoding issues. First time I noticed encoding issues in my code was when writing an xml structure failed on windows, all because of an umlaut in an error message I couldn't care less about.…

So you don't have to deal with it until user data includes _any non-ascii character_ (including emoji, weird spaces copied from other stuff, or loan words like café) "Dealing with unicode" is really just about dealing with it at the input/output boundaries (and even then libraries handle it most of the time). But without the clear delineation that Python 3 provides, when you _do_ hit some issue you probably insert a…

> So you don't have to deal with it until user data includes _any non-ascii character_ (including emoji, weird spaces copied from other stuff, or loan words like café)

Interesting text follows company set naming schemes, which means all english and ascii. The rest could be random bytes for all I have to care about. Many formats like plain text or zip don't have a fixed encoding and I am not going to start guessing which one it is for every file i have to read, there is no way to do that correctly. Dealing with that mess is explicitly something I want to avoid.

Re: Why Is the Migration to Python 3 Taking So Long?

#304
post #145
post #133

Earlier quoted context omitted.

> On the other hand, they could progressively enhance upon a backwards compatible single 2 version. JS manages to do that just fine, as does Java... Common Lisp has a backwards compatibility which goes into decades, and implementations like SBCL had no difficulties at all to absorb Unicode. Racket even supports different language standards and completely different languages (such as Scheme and Algol) running on the s…

> But the incompatibility between Python 2 and Python 3 is perhap s only a symptom of a larger problem. The Python developers have decided that backwards compatibility is not that important any more. Exactly: and that was a wrong decision for anybody but the developers of Python. Everybody else prefers having something that works: "The improvements are welcome, but please allow us to to run our old programs too, than…

And the reason why I think there is a deeper problem is that the issue is clearly not solved with the python2/python3 transition:

Why do some python developers have to maintain installations of a whole handful of python versions just to ensure that their code is working? Why all the mess with pyenv, virtualenv, and so on? If the python developers, as well as the library developers would support backwards compatibility, this would not be necessary at all.

Re: Why Is the Migration to Python 3 Taking So Long?

#305

Earlier quoted context omitted.

It doesn't have to treat "self" as a special keyword - it just has to desugar "foo.bar(x, y)" into "bar(foo, x, y)". There are some other languages that do that - e.g. in F#, you write: member this.foo(x, y) = ... Again, "this" is just an identifier here, and doesn't have any special meaning. It's more elegant firstly because it follow use, and secondly because it means that "def foo(x)" has the same meaning both ins…

> It's more elegant firstly because it follow use No it doesn't. Currently `def foo(self): pass` is called as `instance.foo()`. You're suggesting that `def self.foo(): pass` would be called as `instance.foo()`, except now it looks like self and instance are syntactically related in ways that they aren't. > Again, "this" is just an identifier here, and doesn't have any special meaning. But the grammar is no longer LL(…

Python grammar is not LL(1) in general; just look at set and dict literals. This is really no different than ":" after an expression being legal inside a dict literal (and how you know that it is a dict literal).

But also, there's no reason to make those legal only inside classes. All it needs to do is make "def foo.bar" produce a different type of function, that has the method descriptor-producing behavior that is currently implemented directly on regular functions.

As far as less vs more common case - I think it's more important to optimize for obviousness and consistency. If "def foo" is a function, it should always be a function, and functions should behave the same in all contexts. They currently don't - given class C and its instance I, C.f is not the same object as I.f, and only one of those two is what "def" actually produced.

What I meant by function references inside classes is this:

   class Foo:
      pass

   Foo.bar = lambda: 123
   foo = Foo()
   print(foo.bar())
This blows up with "TypeError: () takes 0 positional arguments but 1 was given", because lambda is of type "function", and it gets the magic treatment when it's read as a member of the instance. So you have to do this:

   Foo.bar = staticmethod(lambda: 123)
and even then this is only possible when you know that the value is going to end up as a class attribute. Sometimes, you do not - you pass a value to some public function somewhere, and it ends up stashed away as a class attribute internally. And it all works great, until you pass a value that just happened to be another function or lambda.

On the other hand, this only applies to objects of type "function", not all callables. So e.g. this is okay:

   Foo.bar = functools.partial(lambda x: x, 123)
because what partial() returns is not a function. Conversely, this means that you can't use partial() to define methods, which can be downright annoying at times. Suppose you have:

   class Foo:
      def frob(self, x, y): ...
and you want to define some helper methods for preset combinations of x and y. You'd think this would work:

   class Foo:
      def frob(self, x, y): ...
      frob_xyzzy = functools.partial(frob, x=1, y=2)
      frob_whammo = functools.partial(frob, x=3, y=4)
except it doesn't - while frob_xyzzy() and frob_whammo() both have the explicit "self" argument, they aren't "proper" functions, and thus that argument doesn't get treated as the implicit receiver:

   foo = Foo()
   foo.frob(x=0, y=0)    # okay
   foo.frob_xyzzy()      # TypeError: frob() missing 1 required positional argument: 'self' 
   foo.frob_whammo(foo)  # okay!
Which is to say, this all is a mess of special cases. You can argue that this all isn't really observable in the "common case" - the problem is that, as software grows more complex, the uncommon cases become common enough that you have to deal them regularly, and then those inconsistencies add even more complexity into the mix that you have to deal with - just when you already thought you had your plate full.

Re: Why Is the Migration to Python 3 Taking So Long?

#306
post #267
post #133

Earlier quoted context omitted.

> On the other hand, they could progressively enhance upon a backwards compatible single 2 version. JS manages to do that just fine, as does Java... Common Lisp has a backwards compatibility which goes into decades, and implementations like SBCL had no difficulties at all to absorb Unicode. Racket even supports different language standards and completely different languages (such as Scheme and Algol) running on the s…

> It is, however, a huge problem for domains like scientific computing, where most code has no maintainers and even for very important code there is no budget or staff for maintenance I think that's a tool selection problem, not just confined to the python world. If the language and libraries won't have a supported lifespan that matches with the maintenance budget of the projects using them then the wrong tool was ch…

Well, first thing is that most new code in scientific research is developed in PhD projects which last some three or maybe four years. The people who develop this do not have resources and time to maintain this code. Projects don't have a budget for that. There are projects which are very long-running (think CERN or ESO's VLT) but even there the true duration of the code usage is seldomly planned (AFAIK, ESO VLT has just started to transition from Tk/Tcl to Python).

You could say then, "well, then Python is perhaps just not a good match for those pesky scientists".

And this brings up two more points:

* A lot of important tools and libraries in the Python ecosystem was developed by scientists. Numarray/Numpy is a good example.

* If the core Python developers don't have the intention to maintain a backwards-compatible language version for more than, say, 15 years, they should perhaps clearly state on the python.org main page something like: "great, as you are a scientist, we welcome your contribution, but Python might not be suitable for tools that support long-term research".

Re: Why Is the Migration to Python 3 Taking So Long?

#307
post #282
post #145

Earlier quoted context omitted.

> But the incompatibility between Python 2 and Python 3 is perhap s only a symptom of a larger problem. The Python developers have decided that backwards compatibility is not that important any more. Exactly: and that was a wrong decision for anybody but the developers of Python. Everybody else prefers having something that works: "The improvements are welcome, but please allow us to to run our old programs too, than…

As an example for Linus' "kernel changes should not break user programs" policy, he was famous for using harsh words to pass the message to those who tired to steer otherwise (these words I don't have to repeat, so I'll just quote his main message): https://lkml.org/lkml/2012/12/23/75 "How long have you been a maintainer? And you still haven't learnt the first rule of kernel maintenance? If a change results in user p…

To explain that a bit, a Linux kernel maintainer is somebody who organizes and collects contributions of others, screens them, and when they are finished, passes them on to Linus to integrate them. That's a huge responsibility, and Linus uses such strong expressions only for people who have such responsibility and which do things which would harm his project. It is not the case that he talks like that to normal contributors (which should be guided by the maintainers).

Re: Why Is the Migration to Python 3 Taking So Long?

#308
post #304
post #145

Earlier quoted context omitted.

> But the incompatibility between Python 2 and Python 3 is perhap s only a symptom of a larger problem. The Python developers have decided that backwards compatibility is not that important any more. Exactly: and that was a wrong decision for anybody but the developers of Python. Everybody else prefers having something that works: "The improvements are welcome, but please allow us to to run our old programs too, than…

And the reason why I think there is a deeper problem is that the issue is clearly not solved with the python2/python3 transition: Why do some python developers have to maintain installations of a whole handful of python versions just to ensure that their code is working? Why all the mess with pyenv, virtualenv, and so on? If the python developers, as well as the library developers would support backwards compatibilit…

Obligatory Reference to Rich Hickeys talk on versioning and interface specs:

https://www.youtube.com/watch?v=oyLBGkS5ICk

Discussion:

https://news.ycombinator.com/item?id=13085952

Re: Why Is the Migration to Python 3 Taking So Long?

#309
post #133
post #69

Earlier quoted context omitted.

> This is backwards thinking. And the alternative is cargo cult "newer is better". > Yes, it's expensive to upgrade from Python 2 to Python 3, but it's also expensive for the Python project to maintain 2 versions of Python indefinitely. On the other hand, they could progressively enhance upon a backwards compatible single 2 version. JS manages to do that just fine, as does Java...

> On the other hand, they could progressively enhance upon a backwards compatible single 2 version. JS manages to do that just fine, as does Java... Common Lisp has a backwards compatibility which goes into decades, and implementations like SBCL had no difficulties at all to absorb Unicode. Racket even supports different language standards and completely different languages (such as Scheme and Algol) running on the s…

Discussion of Hinsen's post on HN:

https://news.ycombinator.com/item?id=17058269

Re: Why Is the Migration to Python 3 Taking So Long?

#310
post #179

Earlier quoted context omitted.

> Who wants to break old SQL? Nobody. Every couple of months there's a new startup / dev site that says "SQL is broken/old/bad, so we reinvented it!". They all sink without trace, but there's a cohort who agrees with them.

The problem I think is they typically try to go ahead and reinvent the entire RDBMS as well. It's not clear to me why postgres hasn't simply grown a whole array of frontends..

Agree totally. I had to climb the painful learning curve of psql because none of the front ends worked the way I wanted for some reason or another. Of course, having learned psql, I'm now scathing of anyone wanting a front end... hmm... maybe that's why ;)
Post reply on HN