Live data from Hacker News

Why Is the Migration to Python 3 Taking So Long?

stackoverflow.blog

311–320 of 355 posts

Re: Why Is the Migration to Python 3 Taking So Long?

#311
post #263
post #215

Earlier quoted context omitted.

> Actually that's the behavior of python 2, it works fine, until you send invalid characters then it blows up. Not always. As far as I can tell writing garbage bytes to various APIs works fine unless they explicitly try to handle encoding issues. First time I noticed encoding issues in my code was when writing an xml structure failed on windows, all because of an umlaut in an error message I couldn't care less about.…

What kind of text do you have to process at your job, that you never meet any unicode in it? Nowadays unicode is everywhere, especially with emojis. Even a simple IRC bot needs to handle that.

I deal with file formats that like plain text files and zip do not specify an encoding and have different encodings depending on where they come from. I think the generic approach is to guess, which means trying encodings until one successfully converts unknown input to garbage unicode resulting in output that is both wrong and different from the original input. Most of the time I can just treat the text contents as byte array, with a few exceptions that are specified to use ascii compatible names.

So you can throw in your emoji and they might not correctly show up on the xml logging metadata I write, because I don't care. But they will end up in the processed file the same way they came in instead of or some random Chinese or Japanese symbol that the guessing algorithm thought appropriate.

Re: Why Is the Migration to Python 3 Taking So Long?

#312

Earlier quoted context omitted.

> It's more elegant firstly because it follow use No it doesn't. Currently `def foo(self): pass` is called as `instance.foo()`. You're suggesting that `def self.foo(): pass` would be called as `instance.foo()`, except now it looks like self and instance are syntactically related in ways that they aren't. > Again, "this" is just an identifier here, and doesn't have any special meaning. But the grammar is no longer LL(…

Python grammar is not LL(1) in general; just look at set and dict literals. This is really no different than ":" after an expression being legal inside a dict literal (and how you know that it is a dict literal). But also, there's no reason to make those legal only inside classes. All it needs to do is make "def foo.bar" produce a different type of function, that has the method descriptor-producing behavior that is c…

> Python grammar is not LL(1) in general; just look at set and dict literals. This is really no different than ":" after an expression being legal inside a dict literal (and how you know that it is a dict literal).

Yes it is[0]. LL(1) Grammars can still be recursive, they just can't change the parsing rules based on distant context.

> As far as less vs more common case - I think it's more important to optimize for obviousness and consistency

Yes, but having the "easiest" thing you do:

    class Foo:
        def bar():
            pass
silently do a usually unwanted thing (create a staticmethod) instead of an obviously wrong thing (raise an error) isn't obvious. It's building a footgun into the language.

The rest of your comment complains about inconsistencies of how python converts various callables to methods. This is a fairly valid and interesting complaint, but has nothing to do with syntax, it is solely a semantic complaint that would be solved by having class creation treat all attributes that are callables as functions. In fact, you could customize class creation yourself this way using __new__, no syntactic changes required.

> as software grows more complex, the uncommon cases become common enough that you have to deal them regularly

While this is true, I think you vastly overestimate how common these constructs are. Like, you're in the realm of "this doesn't appear on github" levels of uncommon.

Personally, again, I think "you can't use partial() to define methods" is a very good thing: if you're doing this, you're into weird metaprogramming land. Its not any harder to, for example, write out

    class Foo:
      def frob(self, x, y): ...
      def frob_xyzzy(self): return self.frob(x=1, y=2)
      def frob_whammo(self): return self.frob(x=3, y=4)
unless you're doing weird metaprogrammy magic and then, as someone who does a lot of weird metaprogrammy magic

1. You deserve what you get

2. You can invoke deeper magic to solve these problems

[0]: https://discuss.python.org/t/switch-pythons-parsing-tech-to-...

Re: Why Is the Migration to Python 3 Taking So Long?

#313
post #304
post #145

Earlier quoted context omitted.

> But the incompatibility between Python 2 and Python 3 is perhap s only a symptom of a larger problem. The Python developers have decided that backwards compatibility is not that important any more. Exactly: and that was a wrong decision for anybody but the developers of Python. Everybody else prefers having something that works: "The improvements are welcome, but please allow us to to run our old programs too, than…

And the reason why I think there is a deeper problem is that the issue is clearly not solved with the python2/python3 transition: Why do some python developers have to maintain installations of a whole handful of python versions just to ensure that their code is working? Why all the mess with pyenv, virtualenv, and so on? If the python developers, as well as the library developers would support backwards compatibilit…

> the issue is clearly not solved with the python2/python3 transition

Exactly. The incompatibility mess continues when using different 3.x versions.

Re: Why Is the Migration to Python 3 Taking So Long?

#314
post #95

Earlier quoted context omitted.

I don't have a horse in the game and I'm not familiar with these people or the "fiasco" but your summary does not seem like accurate description of the issues raised in [5]. Here are the quotes that give me a very different take on the situation: The money was raised specifically to support development of requests 3 > [Reitz] announced that work had begun on "Requests 3", that its headline feature would be the native…

The people who can and should complain if they feel misled about how their money was used are the people who donated the money. Have any of them complained? I say this with some seriousness. NO WHERE that I can see did Reitz say he would hire NJS to do some work. Is Reitz even setup properly to report for taxes or withold taxes on amounts paid to NJS, check paperwork for work eligiblity etc? Would this even be allowa…

My take on it is that NJS was no so much upset/concerned about not getting paid as he was about the way he was treated and how this behavior affects the community.

> And on a more personal level, I felt his interactions with me were extremely manipulative. I felt like he tried to exploit me, and that he tried to make me complicit in covering up his lies to protect his reputation. I was extremely uncomfortable with the idea of going along with this, but he created a situation where my only other options were to either give up on working on async entirely, or else to go public with the whole story, at potentially serious cost to myself.

> Ultimately, I decided to speak out because I care deeply about the Python community and its members. If one of our community's most prominent members freely lies to donors and harms volunteers, and if we all let that go without saying anything, then that puts everything we've built together at risk. And I'm in a better position than many to speak up.

The intent seems not to be trying to get people to blacklist or dogpile on Reitz, but to simply make people aware of the issues so they won't get caught off guard.

> This is the classic "missing stair" problem. Those in the inner circle quietly work around the toxic person. Outsiders come in blind. I'm pretty well-connected in the Python world, and I came in blind.

> Since this is the internet, I have to say explicitly: Please do not harass or abuse Reitz. That's never appropriate

Re: Why Is the Migration to Python 3 Taking So Long?

#315
post #306
post #267

Earlier quoted context omitted.

> It is, however, a huge problem for domains like scientific computing, where most code has no maintainers and even for very important code there is no budget or staff for maintenance I think that's a tool selection problem, not just confined to the python world. If the language and libraries won't have a supported lifespan that matches with the maintenance budget of the projects using them then the wrong tool was ch…

Well, first thing is that most new code in scientific research is developed in PhD projects which last some three or maybe four years. The people who develop this do not have resources and time to maintain this code. Projects don't have a budget for that. There are projects which are very long-running (think CERN or ESO's VLT) but even there the true duration of the code usage is seldomly planned (AFAIK, ESO VLT has…

"ESO VLT has just started to transition from Tk/Tcl to Python" - they might regret this, new Tcl releases tend to take backward compatibility much more seriously than Python :-)

Re: Why Is the Migration to Python 3 Taking So Long?

#316
post #258

Earlier quoted context omitted.

> Actually that's the behavior of python 2, it works fine, until you send invalid characters then it blows up. > In python 3 it always blows up when you mix bytes with text so you can catch the issue early on. Sometimes you don't care about weird characters being print as weird things. In python 2 it works fine: you receive garbage, you pass garbage. In python 3 it shuts down your application with a backtrace. Dealin…

Those issues are common when you're having python 2 code that uses unicode datatype and you have a task to migrate it to python 3. You run your python 2 code on python 3 and it fails, most people at that point will place encode() or decode() in place where you have a failure. When the correct fix would be to place encode/decode at I/O boundary (writing to files (and in python 3 even that is not needed if you open fil…

I've been struggling with this recently when trying to print stdout from subprocess.communicate with code that runs on both 2 and 3. Such a headache - got any recommended reading around this area?

Re: Why Is the Migration to Python 3 Taking So Long?

#317

Earlier quoted context omitted.

Fair criticism. But shouldn't you be even more pissed that the Unicode release was botched and we're still talking about it more than 10 years after? When I upgrade to a new version of C# ... nothing happens. Backwards compatibility is what made Microsoft the company it is. I think Python deserves all the crap it gets for 2 vs 3.

They have to be backwards-compatible to persuade people to pay for their proprietary products rather than switch to another vendor.

So you're saying backwards-compatibility is awesome. I agree.

Re: Why Is the Migration to Python 3 Taking So Long?

#318

Earlier quoted context omitted.

JS and Java can do that because they were designed reasonably well from the start. All they tend to add are more features on top of the solid core language. Python's language was not solid (e.g., strings not unicode by default) so they needed a major overhaul.

Kind of an odd slam, given that Python (and most other widely used languages) predated Unicode. In any case, I think the jury is still out on whether Unicode in the primary string type is a good idea.

Didn’t C# come out after Unicode (1990s vs 2000)? And they still didn’t implement it properly. Instead, they went with the horrible UCS-2 because it made interfacing with the Windows API easier (it uses wchar_t (16-bit) and UCS-2).

Re: Why Is the Migration to Python 3 Taking So Long?

#319
post #179

Earlier quoted context omitted.

The problem I think is they typically try to go ahead and reinvent the entire RDBMS as well. It's not clear to me why postgres hasn't simply grown a whole array of frontends..

Agree totally. I had to climb the painful learning curve of psql because none of the front ends worked the way I wanted for some reason or another. Of course, having learned psql, I'm now scathing of anyone wanting a front end... hmm... maybe that's why ;)

I was actually thinking about the lack of SQL alternatives using the postgres engine; for example, why is datalog not simply available as an extension? Or MySQL syntax? PG implements a wide array of alternatives to PL/pgSQL (including standard programming languages eg python), but for whatever reason these SQL-alts never seem to consider being layered on top of the postgres engine.

However, the lack of GUI frontends is also really weird. I don't see why it'd be harder to support than any other DB, and afaik pg has gotten fairly popular..

Re: Why Is the Migration to Python 3 Taking So Long?

#320

Earlier quoted context omitted.

Again with the 'Tremendous amount of effort' meme. I've done many ports and they were all trivial: - run 2to3 - spend 2h max fixing any failing tests - cook of any remaining issues in a few days of beta testing like you'd do for any new release Now now doubt Python 2.7 is a excellent and solid release and will remain so for as long anyone keeps the bitrot in check, but to keep using it because porting is 'hard' is pa…

Behold the tremendous amount of effort for Mercurial: https://www.mercurial-scm.org/repo/hg/log?rev=py3&revcount=2... They've been porting hg into Python 3 for the last 10 years and are only now nearing completion. I've written a bit more about this in Lobsters: https://lobste.rs/s/3vkmm8/why_i_can_t_remove_python_2_from_...

Honest question, how can it possibly take 10 years to port hg to Python 3? If I am to believe the Wikipedia source for the first release of Mercurial[0], it would've been only 4 years old at the start of the 10 year porting process. How on earth does it take 10 years to port 4 year old software?

Even taking into account the fact that new features were still being added and not all focus was on porting, this doesn't really seem like a reasonable representation of what's going on; I have a suspicion that "10 years" of porting here does not entail nearly as much work as it seems.

[0] https://lkml.org/lkml/2005/4/20/45

Post reply on HN