Live data from Hacker News

Why Is the Migration to Python 3 Taking So Long?

stackoverflow.blog

291–300 of 355 posts

Re: Why Is the Migration to Python 3 Taking So Long?

#291

Earlier quoted context omitted.

I ran into an issue recently specific to Pandas and python3 with unicode. pd.read_excel(filepath) will read an entire dataset even if it contains unicode characters. pd.ExcelFile() silently drops(!!) unicode rows. The resulting object will simply skip unicode-containing rows (in ANY column) them without even a warning. For example, if you had an excel file: word --- "hello" "hello" 你早 你早 "hello" then pd.read_excel()…

Is there an issue for this in the bug tracker?

I searched, and this is the closest thing (https://github.com/pandas-dev/pandas/issues/11503) but it is not the issue I experienced.

I'm not sure how to submit a bug report, to be honest.

Re: Why Is the Migration to Python 3 Taking So Long?

#292

Earlier quoted context omitted.

I myself, speaking for my self and only my self find that, in regards to myself, the redundant self keyword, in my self's opinion, is somewhat selfish and easy for my self to accidentally omit. Self.

My experience with Python is a little lacking, but doesn't the self keyword mean the difference between an instance method, and a static method? Rust does the same thing - with self, it's an instance method, without, it's a static method.

"self" is not a keyword in Python, just a convention.

Any function inside a class declaration becomes an instance method, with its first argument becoming the explicit receiver. You have to use @staticmethod to prevent that (or @classmethod to get the class as the first argument, instead of the instance).

Furthermore, this behavior is not parse-time, but runtime. The "def" statement that defines a function produces a plain function object, regardless of whether it's inside a class or not. Once the class finishes defining, the function is still a function - which is why C.f gives you a plain instance that can be called by passing "self" explicitly as an argument.

However, Python allows objects to provide special behavior for themselves whenever they're retrieved via a member of some class - that is, when you write something like x.y, after y is retrieved from x, it gets the opportunity to peek at x, and substitute itself with something else. Function objects in Python (here I mean specifically the type of objects created with "def" and "lambda", not just any callable) use this feature to convert themselves to bound methods. So, when you say x.f, after retrieving f, f itself is asked if it wants to substitute something else in its place - and it returns a new callable object m, such that m(y) calls f(x, y). That's what makes x.f(y) work.

Re: Why Is the Migration to Python 3 Taking So Long?

#293

Earlier quoted context omitted.

Why not just migrate to 3? genuinely curious...

Because of the amount of work involved in porting my existing Python code. Python 3 doesn't offer any advantages that matter to me, so that's a lot of effort for little gain.

OK, thanks. Good luck with the bug fixing ;)

Re: Why Is the Migration to Python 3 Taking So Long?

#294

Earlier quoted context omitted.

I don't think you understand what is actually going on under the hood here, at all. Methods are member functions of a class, and must be invoked on an object. someInstance.foo() When invoking a method from within the class, you still need a reference to the object the method will be invoked on. In python, the reference is passed implicitly as the first argument, and usually called self. That has nothing to do with th…

The syntax of a language is an abstraction over what is actually going on under the hood. There's no reason python couldn't have designed their OOP abstraction more elegantly.

> There's no reason python couldn't have designed their OOP abstraction more elegantly.

Sure there is: consistency. If member functions on an object that happens to be a class (i.e., methods) did magic transformations that member functions of other objects did not do, the mental overhead to understand Python code would be higher, and the ability to build custom general abstractions would be weaker.

It would perhaps make the simplest cases microscopically easier, but at the expense of making the already hard things more confusing and difficult to wrap with generalities.

Most statically-typed OOPLs don't have first-class classes that are just normal objects, so this isn't an issue because the things that enables aren't available; other dynamic languages may use models where methods aren't data members of classes (e.g., Ruby where methods are associated with classes, but not as instance members which, in Ruby, wouldn't be externally accessible—while Ruby classes are objects, the ability to hold methods is a special power of Ruby classes/modules compared to other objects, not just having instance members. This is one way Ruby's model is more complex than Python’s, and it definitely bites you in terms of seeking general solutions some times..)

Re: Why Is the Migration to Python 3 Taking So Long?

#295

Earlier quoted context omitted.

> the broken programs would still be broken in either language. You need to slap a decode anyway on reads from subprocesses in python3, and files open in Unicode mode by default. Wouldn't that fix the majority of silly UTF-8 compat bugs? Or am I missing a class of bugs that's not avoided automatically by python3 strings?

Well, the summary of the argument is that the python3 UTF-8 does not actually solve the fundamental problem of multiple encoding formats existing. Think: Do you know that the process actually returns UTF-8, or that the file is actually encoded in UTF-8? No, you're just guessing. This puts people in the habit of attempting to turn everything into UTF-8 which could happen automatically and not require so much boilerpla…

> Think: Do you know that the process actually returns UTF-8, or that the file is actually encoded in UTF-8? No, you're just guessing.

Well, no, not really. You go read the docs and try to find out. Most of the time, there is a definitive encoding - if there weren't, a lot more things would be broken. Sometimes, it is not guaranteed, even though de facto that is the case - and this highlights broken interface specifications. When it is truly unknown, you explicitly treat it as raw bytes.

And the good thing about Python 3 is that it forces you to think about this. In Python 2, most of the time, data processing code can be hacked together, and it "just works", right until the point the input happens to include something unanticipated. Like, say, the word "naïve".

> On the other end, most programs don't actually care what the data encoding is. They just move it.

It doesn't necessarily mean that they get to dodge the bullet. In Python 2, if you read data from a file, you get raw bytes, but if you read data from parsed JSON, you get Unicode strings - because JSON itself is guaranteed to be Unicode. Guess what happens when the byte string you've read from the file, and the Unicode string you've read from a JSON HTTP response, are concatenated?

Re: Why Is the Migration to Python 3 Taking So Long?

#296

Py3 is for all practical purposes a language fork of Py2. So it doesn't really make sense to talk of a "migration". If Py2 becomes unworkable somehow then people will rewrite stuff. Some of it might even be in Py3. Considering all the stuff that is written in Py2 I really don't see it being out and out abandoned. That wouldn't really make any sense. With computer languages stuff never goes away.

Py2 is the new Fortran, I like to say.

Py2 is the new Fortran 77.

It was good in its time, and great things were done in it that are still around... but let's move onto F90 already.

Re: Why Is the Migration to Python 3 Taking So Long?

#297

Earlier quoted context omitted.

Why is self.foo more elegant? It's very rare that you'll call a given method as self.foo, and there's a slew of complications that come with such a syntax (the declaration grammar is more complex, `self` is now special, etc.) It's true that languages are abstractions, but not all abstractions are useful.

It doesn't have to treat "self" as a special keyword - it just has to desugar "foo.bar(x, y)" into "bar(foo, x, y)". There are some other languages that do that - e.g. in F#, you write: member this.foo(x, y) = ... Again, "this" is just an identifier here, and doesn't have any special meaning. It's more elegant firstly because it follow use, and secondly because it means that "def foo(x)" has the same meaning both ins…

> As it is, we need stuff like @staticmethod and @classmethod

You would need those even with your suggestion (except you could drop @staticmethod if you add another layer of magic so that methods declared without an leading identifier were assumes static; you'd still need @classmethod some equivalent mechanism to distinguish which non-static methods were class vs instance methods.)

Re: Why Is the Migration to Python 3 Taking So Long?

#298

Earlier quoted context omitted.

Well, PEP 394 suggested it be this way, so Python is also a bit complicit.

As the history section of that PEP notes, it was actually written in part as a response to distributions like Arch changing the default Python to Python3. Although it's true that the PEP originally said that python should point to Python2 (it doesn't any more), it also said that code requiring a specific version of Python shouldn't assume anything about the system default Python, instead using either python2 or pytho…

The original recommendation had to be made, because at the time, there was a lot of software from pre-Py3 days that would have #!/usr/bin/env python, and that was broken badly if it was pointing at Python 3. They have changed the PEP now that Python 2 is reaching EOL, since it wouldn't make sense to demand "python" to be an alias for an unsupported version.

Re: Why Is the Migration to Python 3 Taking So Long?

#299
post #9
post #6

Earlier quoted context omitted.

Red Hat has long support contracts for their server OSes that shipped with Python 2 when it was still kosher to do so. That means they'll patch Python 2 should vulnerabilities be found on their OS.

It’s a pain from the outside looking into their business. Their contracts, though, are their business.

Those support contracts aren't cheap, though. I don't think they're getting the short end of the stick here. If anything, the companies that run lots of Python 2 code in the house that haven't migrated yet - which is not uncommon in the enterprise - have the incentive to run that code on RedHat, if they aren't doing so already.

I wouldn't even be surprised if they provide support contracts specifically for Python 2 on their newer OSes. ActiveState announced that they'll continue providing paid support for Python 2 on all platforms, so some people at least feel like there's a business model there...

https://www.activestate.com/company/press/press-releases/act...

Re: Why Is the Migration to Python 3 Taking So Long?

#300

Earlier quoted context omitted.

Why is self.foo more elegant? It's very rare that you'll call a given method as self.foo, and there's a slew of complications that come with such a syntax (the declaration grammar is more complex, `self` is now special, etc.) It's true that languages are abstractions, but not all abstractions are useful.

It doesn't have to treat "self" as a special keyword - it just has to desugar "foo.bar(x, y)" into "bar(foo, x, y)". There are some other languages that do that - e.g. in F#, you write: member this.foo(x, y) = ... Again, "this" is just an identifier here, and doesn't have any special meaning. It's more elegant firstly because it follow use, and secondly because it means that "def foo(x)" has the same meaning both ins…

> It's more elegant firstly because it follow use

No it doesn't. Currently `def foo(self): pass` is called as `instance.foo()`. You're suggesting that `def self.foo(): pass` would be called as `instance.foo()`, except now it looks like self and instance are syntactically related in ways that they aren't.

> Again, "this" is just an identifier here, and doesn't have any special meaning.

But the grammar is no longer LL(1), and you have weird conditionally valid syntax, like `.` in a function name is valid only in a class block.

> "def foo(x)" has the same meaning both inside and outside of a class declaration

This is a stretch, especially since you're now optimizing for the uncommon case. Staticmethods are rare compared to instance methods (I'll go further and claim that static methods are an antipattern in python, modules are valid namespaces, you can stick a function in a module and couple it to a class also defined in that module and nothing bad will happen. Banning staticmethods entirely doesn't reduce expressiveness). Aligning staticmethods with functions, instead of aligning instance methods with functions (as python does currently) encourages you to do the wrong thing.

> classmethod

Your changes don't affect classmethod at all, if anything they'd make classmethod more of a special case. How do you signal that `self.foo()` takes `self` as the class instead of the instance?

> It's especially annoying when you have class attributes that happen to reference a function, because conversion from functions to methods is a runtime thing - so you have to remember to wrap those in staticmethod() as well.

What do you mean? Like

    class Foo:
        a = Foo.func()
        @staticmethod
        def func():
            return 1
I'll say again: staticmethods are an antipattern in python:

    def func()
        return 1

    class Foo:
        a = func()
works just as well, better in fact. Modules are great namespaces. Classes are more than namespaces, and if all you need is a namespace, you shouldn't use a class.

> because conversion from functions to methods is a runtime thing

I'd also quibble with this: it's a binding thing.

    class Foo:
        a = Foo.foo(None)
        def foo(self):
            return 1
will work, and if you check, type(Foo.foo) is still just `function`, its only when you create an instance of Foo that the function `foo` is bound to the instance, and when that is done, the bound `foo` is converted to a method object. This was different in python2, where Foo.foo and instance.foo were both "instancemethod" objects, but in python3, Foo.foo is a plain old function, and instance.foo is a method.

Specifically this means that if you can get your hands on the `method` constructor (like with `type(instance.method)`), you can then do silly things like

    class A():
      def foo(self): pass
    instance = A()
    def f(self): return 5
    assert instance.func() == 5
and this will work. You'll have bound the function to the instance. Of course, if you stick an attribute on `instance` (or `A`), and reference `self.attribute` in the function, this will still work. (this also lets you do things like bind a given instance of a function to a different instance of the class, but that's because the method constructor is essentially just partial with some bookkeeping for class information)
Post reply on HN