Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why Is the Migration to Python 3 Taking So Long?

stackoverflow.blog

Post reply on HN
Print view View original

211–220 of 355 posts

Re: Why Is the Migration to Python 3 Taking So Long?

#211

Post by joshuamorton »

kzrdude wrote: post #132
chungy wrote: post #20

Earlier quoted context omitted.

Can you describe how OOP feels tacked on? One of the major changes in Python 3 is that new-style classes are the only style of classes.

Modern Python lacks coherent design: How do you define enums? with a superclass. How do you define data classes? With a class decorator. And then there's metaclasses too.

Enums are technically defined with a metaclass, as are abstract base classes and most other weird special classes (but in all cases you access the metaclass by subclassing something). Dataclasses are the one exception, and they do this for pragmatic reasons: multiple metaclasses are tricky. So a decorator allows the existence of an abstract dataclass, for example.

Re: Why Is the Migration to Python 3 Taking So Long?

#212

Post by user5994461 »

klyrs wrote: post #109

In my experience, the biggest issue that I face is "what do you mean by Python 3?" I count 4 minor versions which aren't fundamentally broken, and I encounter them all on a regular basis. A lot of my code is performance critical, and, for example, I'm still salty about dictionary operations taking O(log(n)). But the proliferation of active minor versions makes it very difficult to write portable, performant code. It'…

If it helps you, you can consider that no version below 3.5 is worth thinking of. It's the edge when python added back enough features to ease the migration and many libraries started being ported.

Current version is 3.7. If you expect your migration work to take a year, you should consider going for 3.7 and above only, because the previous minor versions will be dropped by the time you're done.

Re: Why Is the Migration to Python 3 Taking So Long?

#213

Post by Camillo »

ageofwant wrote: post #185
jordigh wrote: post #177

Earlier quoted context omitted.

Behold the tremendous amount of effort for Mercurial: https://www.mercurial-scm.org/repo/hg/log?rev=py3&revcount=2... They've been porting hg into Python 3 for the last 10 years and are only now nearing completion. I've written a bit more about this in Lobsters: https://lobste.rs/s/3vkmm8/why_i_can_t_remove_python_2_from_...

Yes of course there will be exceptions. But the vast majority off Python code bases are not mercurial or dropbox or imgur. Just like the vast majority of software using companies are not google or facebook. The average few hundred to few thousand loc app, which should be 98% of all production code-bases will almost certainly port with no issue.

Maybe now. When python3 came out, anything that touched the filesystem was a hideous mess to port. Let's say you have a simple script that takes a file name as an argument, reads the file, prints some message to stdout (which includes the name of the file), and creates a new output file whose name is based on the name of the first file.

In python2, that's trivial. Whatever system you're on would normally be configured so that filename bytes dumped to the terminal would be displayed correctly, so you could just treat the strings as bytes and it would be fine.

In python3, it was a nightmare. No, you could not just decode from/encode to UTF-8, even if that was what your system used! Python had its own idea of what the encoding of the terminal was, and if you used the wrong one, it wouldn't let you print. And if you tried to convert from UTF-8 to whatever it thought the terminal was using, it would also break, because not all characters were representable. And your script could just not tell Python to treat the terminal as UTF-8, either; you had to start digging into locale settings, and if you tried to fix those, then _everything else_ would break, and nobody had any idea what the right thing to do was, because you were using an obscure OS (the latest macOS at the time).

I assume that it works better now.

Re: Why Is the Migration to Python 3 Taking So Long?

#214

Post by ogre_codes »

ageofwant wrote: post #200
coldtea wrote: post #184

Earlier quoted context omitted.

> How do you define just fine? Besides the total domination of the web programming space, which is of course aided by it being the only option: 1) Used by choice even on the server and application development (where it was never the only option, and wasn't even preferable/viable before) 2) Fast pace of language development 3) A thriving package ecosystem with millions of packages 4) Adopted by all major companies 5)…

And yet Python is eating the world: http://pypl.github.io/PYPL.html Not that I'm implying that 'popularity' is a good measure of anything.

Javascript is the king of compromises. It's supported by every browser on the planet so of course support is massive. It's not as if a front end web developer can choose to work on the web and not use Javascript in some form or another.

Python is popular largely based on the fact that it's so approachable. It is the BASIC/ VB of modern times for whatever that is worth. It does scale up to larger projects and is frequently used for big scale stuff, but I suspect the fact that it's so ubiquitous has more to do with the fact that it's also easy to pick up and for companies to find people with Python dev skills (or train them up).

Re: Why Is the Migration to Python 3 Taking So Long?

#215

Post by josefx »

takeda wrote: post #160
recursivecaveat wrote: post #81

The simple reason is that there was no compelling feature to reward you for upgrading. You'd spend a tremendous amount of effort for dubious return and (until recently) a smaller ecosystem. 1. Unicode support was actually an anti-feature for most existing code. If you're writing a simple script you prefer 'garbage-in, garbage-out' unicode rather than scattering casts everywhere to watch it randomly explode when an in…

> Unicode support was actually an anti-feature for most existing code. If you're writing a simple script you prefer 'garbage-in, garbage-out' unicode rather than scattering casts everywhere to watch it randomly explode when an invalid byte sneaks in. If you did have a big user-facing application that cared about unicode, then the conversion was incredibly painful for you because you were a real user of the old style.…

> Actually that's the behavior of python 2, it works fine, until you send invalid characters then it blows up.

Not always. As far as I can tell writing garbage bytes to various APIs works fine unless they explicitly try to handle encoding issues. First time I noticed encoding issues in my code was when writing an xml structure failed on windows, all because of an umlaut in an error message I couldn't care less about. The solution was to simply kill any non ascii character in the string, not a nice or clean solution but the issue wasn't worth more effort.

> In python 3 it always blows up when you mix bytes with text so you can catch the issue early on.

That is nice if your job involves dealing with unicode issues. My job doesn't, any time I have to deal with it despite that is time wasted.

Re: Why Is the Migration to Python 3 Taking So Long?

#216

Post by your-nanny »

ben_jones wrote: post #199
ageofwant wrote: post #185

Earlier quoted context omitted.

Yes of course there will be exceptions. But the vast majority off Python code bases are not mercurial or dropbox or imgur. Just like the vast majority of software using companies are not google or facebook. The average few hundred to few thousand loc app, which should be 98% of all production code-bases will almost certainly port with no issue.

You're assuming a lot. What about codebases with python2 third party dependencies that don't work in python3? Now you have to port that entire library as well, or write it yourself while crossing your fingers that it is well documented and easy to work through. What about codebases without decent test suites? I'd argue most production codebases don't have good test suites, or at least the most complex of code is usua…

dependency rot is a more general problem. in my work we deal with lots of seldomly used applications, and I've found that reducing external dependencies tends to keep life happier

Re: Why Is the Migration to Python 3 Taking So Long?

#217

Post by coleifer »

recursivecaveat wrote: post #81

The simple reason is that there was no compelling feature to reward you for upgrading. You'd spend a tremendous amount of effort for dubious return and (until recently) a smaller ecosystem. 1. Unicode support was actually an anti-feature for most existing code. If you're writing a simple script you prefer 'garbage-in, garbage-out' unicode rather than scattering casts everywhere to watch it randomly explode when an in…

Solid take. I'd add that performance was worse for a number of releases, and there were significant warts and incompatibilities in versions before 3.4.

Personally, asyncio and type annotations are a big turnoff. I know this is a bit contrarian, but I've always favored the greenlet/gevent approach to doing cooperative multi-tasking. Asyncio (neé twisted) had a large number of detractors, but now that the red/blue approach has been blessed, it seems like many are just swallowing their bile and using it.

Type annotations really chafe because they seem so unpythonic. I like using python for it's dynamicity, and for the clean, simple code. Type annotations feel like an alien invader, and make code much more tedious to try and read. If I want static typing, I'll use a statically typed language.

Re: Why Is the Migration to Python 3 Taking So Long?

#218

Post by klyrs »

user5994461 wrote: post #212
klyrs wrote: post #109

In my experience, the biggest issue that I face is "what do you mean by Python 3?" I count 4 minor versions which aren't fundamentally broken, and I encounter them all on a regular basis. A lot of my code is performance critical, and, for example, I'm still salty about dictionary operations taking O(log(n)). But the proliferation of active minor versions makes it very difficult to write portable, performant code. It'…

If it helps you, you can consider that no version below 3.5 is worth thinking of. It's the edge when python added back enough features to ease the migration and many libraries started being ported. Current version is 3.7. If you expect your migration work to take a year, you should consider going for 3.7 and above only, because the previous minor versions will be dropped by the time you're done.

Thank you (and it's good to say so if another reader doesn't know), but yeah, the 4 versions I was referring to were 3.5-3.8... but my point is that it's now a perpetually moving target.

And fwiw "3.7 is the current version" doesn't help my users.

Re: Why Is the Migration to Python 3 Taking So Long?

#219

Post by j88439h84 »

oarabbus_ wrote: post #168
downerending wrote: post #8

Because there's been not enough carrot and too much stick. The only real killer feature of Python3 is the async programming model. Unfortunately, the standard library version is numbingly complex. (Curio is far easier to follow, but doesn't appear to have a future.) On the down side, switching to Unicode strings is a major hurdle. It mostly "just works", but when it doesn't, it can be difficult to see what's going on…

I ran into an issue recently specific to Pandas and python3 with unicode. pd.read_excel(filepath) will read an entire dataset even if it contains unicode characters. pd.ExcelFile() silently drops(!!) unicode rows. The resulting object will simply skip unicode-containing rows (in ANY column) them without even a warning. For example, if you had an excel file: word --- "hello" "hello" 你早 你早 "hello" then pd.read_excel()…

Is there an issue for this in the bug tracker?

Re: Why Is the Migration to Python 3 Taking So Long?

#220

Post by dragonwriter »

AnimalMuppet wrote: post #144
dragonwriter wrote: post #41

Earlier quoted context omitted.

I think for some people having the receiver as an explicit parameter of methods rather than references through a special syntactic variable bothers some people and makes it look to them like a procedural language playing OO, though I personally think it's the most clear representation of what every class-based OO language actually does operationally, since methods are attached to a class and take a reference to the i…

It may in fact be what every class-based OO language actually does operationally. But having the OO language do that for you is, to me, one of the lines that separates them from "a procedural language playing OO".

To me, it makes sense for a class-based OO language with first-class functions and (unlike, e.g., Java) available direct external access to data members to do what Python does because it reinforces the distinction between function members and methods.

Perhaps more importantly, it makes even more sense in a language like Python where classes are (unlike many, especially statically-typed, class-based OO languages) first class objects to do what Python does, because of the relation of methods to classes. It also, to me. makes unbound/bound methods slightly more intuitive.

Now, I too was initially thrown by it because I'd used a bunch of OO languages that did it the other way first, and for quite a long time.

Post reply on HN