Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why Is the Migration to Python 3 Taking So Long?

stackoverflow.blog

Post reply on HN
Print view View original

201–210 of 355 posts

Re: Why Is the Migration to Python 3 Taking So Long?

#201

Post by dyingkneepad »

takeda wrote: post #160
recursivecaveat wrote: post #81

The simple reason is that there was no compelling feature to reward you for upgrading. You'd spend a tremendous amount of effort for dubious return and (until recently) a smaller ecosystem. 1. Unicode support was actually an anti-feature for most existing code. If you're writing a simple script you prefer 'garbage-in, garbage-out' unicode rather than scattering casts everywhere to watch it randomly explode when an in…

> Unicode support was actually an anti-feature for most existing code. If you're writing a simple script you prefer 'garbage-in, garbage-out' unicode rather than scattering casts everywhere to watch it randomly explode when an invalid byte sneaks in. If you did have a big user-facing application that cared about unicode, then the conversion was incredibly painful for you because you were a real user of the old style.…

> Actually that's the behavior of python 2, it works fine, until you send invalid characters then it blows up.

> In python 3 it always blows up when you mix bytes with text so you can catch the issue early on.

Sometimes you don't care about weird characters being print as weird things. In python 2 it works fine: you receive garbage, you pass garbage. In python 3 it shuts down your application with a backtrace.

Dealing with this was one of my first Python experiences and it was very frustrating, because I realized that simply using #!/usr/bin/python2 would solve my problem but people wanted python3 just because it was fancier. So we played a lot of whack-a-mole to make it not explode regardless of the input. And the documentation was particularly horrible regarding that, not even the experienced pythoners knew how to deal with it properly.

Re: Why Is the Migration to Python 3 Taking So Long?

#202

Post by Johnny555 »

ageofwant wrote: post #171
recursivecaveat wrote: post #81

The simple reason is that there was no compelling feature to reward you for upgrading. You'd spend a tremendous amount of effort for dubious return and (until recently) a smaller ecosystem. 1. Unicode support was actually an anti-feature for most existing code. If you're writing a simple script you prefer 'garbage-in, garbage-out' unicode rather than scattering casts everywhere to watch it randomly explode when an in…

Again with the 'Tremendous amount of effort' meme. I've done many ports and they were all trivial: - run 2to3 - spend 2h max fixing any failing tests - cook of any remaining issues in a few days of beta testing like you'd do for any new release Now now doubt Python 2.7 is a excellent and solid release and will remain so for as long anyone keeps the bitrot in check, but to keep using it because porting is 'hard' is pa…

It's not so much that it's "hard", but that it's time consuming when you have hundreds or even thousands of python scripts to port -- and since those scripts already work and you probably weren't going to have to touch them at all, you're not really gaining anything for all of that porting effort.

Re: Why Is the Migration to Python 3 Taking So Long?

#203

Post by user5994461 »

LIV2 wrote: post #86
_skel wrote: post #71

I recently went through a fairly large upgrade from JDK8 to JDK 11 and it was a bit of a pain -- lots of dependencies to update, etc. But very few code changes were required, and the static type system made it pretty clear when the codebase was broken -- it just wouldn't build. It still took my team several weeks. Migrating from Python 2 to Python 3 is way worse than that -- code changes are required, and because Pyt…

Will this will make people less likely to use Python in future for some projects? I'm no developer or manager but I figure there'd be a thought in the back of my head thinking "what if we need to rewrite this all again for the next major release"

You're not gonna have to rewrite it in a few years. Languages only go through the mess of retrofitting unicode once.

Besides Java and Python already discussed, another big mess of a transition was from Qt 4 to Qt 5, where all the strings became unicode.

Re: Why Is the Migration to Python 3 Taking So Long?

#204

Post by takeda »

pmontra wrote: post #182
takeda wrote: post #160

Earlier quoted context omitted.

> Unicode support was actually an anti-feature for most existing code. If you're writing a simple script you prefer 'garbage-in, garbage-out' unicode rather than scattering casts everywhere to watch it randomly explode when an invalid byte sneaks in. If you did have a big user-facing application that cared about unicode, then the conversion was incredibly painful for you because you were a real user of the old style.…

> In python 3 it always blows up when you mix bytes with text so you can catch the issue early on. This is definitely the case. I've been wrestling with bytes and strings all the time during the port of a Django application to Python 3 for a costumer. I can see myself encoding and decoding response bodies and JSON for the time being. For reasons I didn't investigate I don't have to do that with projects in Ruby and E…

you don't have to do these things in python 3 either, your problem was that you had python 2 code that was already broken and you are started adding encode/decode to fix it, typically making the problem worse.

If you write code in python 3 from the start you rarely need to use encode() and decode(). Typically what you always want is a text not bytes.

Exception to it might be places where you want to serialize like IO (network or files, although even files are converted on the fly unless you open file in a binary mode).

Re: Why Is the Migration to Python 3 Taking So Long?

#206

Post by takeda »

CaptainMarvel wrote: post #197
pmontra wrote: post #182

Earlier quoted context omitted.

> In python 3 it always blows up when you mix bytes with text so you can catch the issue early on. This is definitely the case. I've been wrestling with bytes and strings all the time during the port of a Django application to Python 3 for a costumer. I can see myself encoding and decoding response bodies and JSON for the time being. For reasons I didn't investigate I don't have to do that with projects in Ruby and E…

I’ve worked in a variety of Django codebases, and the last time I had trouble with string encoding/decoding was with Python 2. Since moving to Python 3, I have rarely needed to manually encode or decode, and I genuinely can't remember the last time I did. Perhaps there’s something about a port that requires encoding/decoding bytes/strings?

The encoding/decoding is heavy in codebases that have to run on Python 2 and Python 3 at the same time, and authors are worried about handling unicode correctly on python 2.

Ironically when your python 2 app doesn't care about unicode, the porting to python 3 is actually much easier.

Re: Why Is the Migration to Python 3 Taking So Long?

#207

Post by user5994461 »

viraptor wrote: post #67
Animats wrote: post #48

First off. Python 2.6 and 2.7 supported Unicode just fine. I had a large all-Unicode system in Python 2.6. You had to write u'word" to get a Unicode constant, and use a "unicode(s)" function here and there. Also, the part that remained "compatible" was that "str" remained an array of bytes, even though there was also a type "bytes" and a "bytearray". Early Python 3 was hell for conversion. The syntax was changed for…

> Python 2.6 and 2.7 supported Unicode just fine It did, but in a way that chainsaws support sculpting just fine. Technically possible. Very advanced people will know how to handle it. Everybody else is just going to injure themselves randomly. Most people writing py2 got the text/binary processing working on accident. Things appear to work until you throw actual Unicode into parameters and then nobody knows what hap…

Pictures of chainsaw sculptures for reference: https://www.google.com/search?q=chainsaw+sculptures&tbm=isch

Some are quite good and finely detailed in my opinion. It's really nothing like what you'd expect after hearing chainsaw. There actually are small chainsaw, maybe even one-handed ones, to do that.

Re: Why Is the Migration to Python 3 Taking So Long?

#208

Post by ageofwant »

ben_jones wrote: post #199
ageofwant wrote: post #185

Earlier quoted context omitted.

Yes of course there will be exceptions. But the vast majority off Python code bases are not mercurial or dropbox or imgur. Just like the vast majority of software using companies are not google or facebook. The average few hundred to few thousand loc app, which should be 98% of all production code-bases will almost certainly port with no issue.

You're assuming a lot. What about codebases with python2 third party dependencies that don't work in python3? Now you have to port that entire library as well, or write it yourself while crossing your fingers that it is well documented and easy to work through. What about codebases without decent test suites? I'd argue most production codebases don't have good test suites, or at least the most complex of code is usua…

It happens... but maybe you are assuming to little.

There are almost no relevant 3'rd party libs that has not been ported at this stage. If they have not they probably have been abandoned and the client codebase have bigger issues. Same for uncovered code bases, and 'unprofessional' Python production code. That's hardly Python's fault.

Re: Why Is the Migration to Python 3 Taking So Long?

#209

Post by user5994461 »

at_a_remove wrote: post #125

In my current job, I can point out something that might be a contributing factor: I work in an industry where there is basically one 800lb gorilla of a vendor. They update rarely, because their product is a mission-critical, life-or-death sort of thing. Their current product is heavily, heavily integrated with x.y.z version of software from a different vendor in a different segment, but also weighing in at 800lb. Yes…

Surprised it's including pip, if it's really this old or tightly coupled, it's common to not have pip at all, or the old pip version simply can't run.

If it can help you. The trick I use is to install a normal python 2.7 interpreter with pip. Then you can use it to install software to any directory, including the one from the other application. There are flags to specify what to install, from where to where, internet or not, something like

    pip install packagename --target=/to/app/lib

Re: Why Is the Migration to Python 3 Taking So Long?

#210

Post by joshuamorton »

biggestdecision wrote: post #180
keypusher wrote: post #152

Earlier quoted context omitted.

I don't think you understand what is actually going on under the hood here, at all. Methods are member functions of a class, and must be invoked on an object. someInstance.foo() When invoking a method from within the class, you still need a reference to the object the method will be invoked on. In python, the reference is passed implicitly as the first argument, and usually called self. That has nothing to do with th…

The syntax of a language is an abstraction over what is actually going on under the hood. There's no reason python couldn't have designed their OOP abstraction more elegantly.

Why is self.foo more elegant? It's very rare that you'll call a given method as self.foo, and there's a slew of complications that come with such a syntax (the declaration grammar is more complex, `self` is now special, etc.)

It's true that languages are abstractions, but not all abstractions are useful.

Post reply on HN