Live data from Hacker News

How Python 3 Should Have Worked (2012)

aaronsw.com

61–70 of 80 posts

Re: How Python 3 Should Have Worked (2012)

#61
post #36

Earlier quoted context omitted.

Don't get me wrong. I use python professionally. I like it. I know about CFFI. I'm just saying, it has its warts.

> I'm just saying, it has its warts. That statement is too broad to be useful though. All languages have warts.

To be fair, he did list a number of specific warts above.

Re: How Python 3 Should Have Worked (2012)

#62
post #32

Earlier quoted context omitted.

>I hate implicit hidden defaults and assumptions. Your implicit hidden default is my handy abbreviation. Imagine if we didn't have contractions in English. For that previous sentence, is it really at all frightening that the "didn't" meant "did not" but we haven't (ha) written it out? Could it possibly mean anything else? Wouldn't English be all the more stilted and ugly if there were only one explicit and verbose wa…

I'll agree with this entirely. As a newcomer to Python, I found the explicit "self" argument to be confusing. And now I just find it irritating. It breaks convention with every other OO language; it's additional meaningless noise in method definitions; and most importantly it violates the convention used by nearly every other programming language I'm aware of, which is that method definition argument lists and method…

I like the explicit self. It makes the language much simpler, clearer and less "magic".

Re: How Python 3 Should Have Worked (2012)

#63
post #56

This misses the point of why Python 3 was invented: Unicode. Python 2's string handling is broken in the presence of unicode characters, often leading to subtle errors that wouldn't cause exceptions until far away from the place where the error was introduced, and oftentimes didn't produce exceptions at all, just wrong data. Strings were defined as sequences of bytes, and then provided a .decode method to convert the…

It's exceedingly rare that you need a sequence of codepoints. As evidence, consider that the native string types in many languages with pervasive Unicode support, such as Java and C#, do not provide a codepoint sequence. Programs generally need to deal with textual data at one of three levels: 1. Manipulate strings and substrings. e.g., "does string x contain substring y"? Byte sequences of utf8 data are fine for thi…

The problem isn't access to individual codepoints, it's mixing strings with different encodings without any way of tracking the encoding or knowing that you're doing something wrong.

UTF-8 everywhere works great when you can enforce it. On the level of an individual project, you can enforce it. On the level of a language ecosystem, you can't, and you need to. Otherwises you end up with some libraries who assume their internals are UTF-8 encoded strings, some libraries that assume they are ASCII strings, some libraries that assume the caller is handling encoding issues and will take care of ensuring that it's all UTF-8, some that make no assumptions at all and carry around the encoding everywhere, and some that just haven't thought about the problem and silently break when you use them with data that came from other libraries.

It's interesting that Go and Java - both languages with mature Unicode handling - still have a distinction between uninterpreted bytes and UTF-8 or UCS-2 text. In Go, you have separate []bytes and string types, even though Go strings are just UTF-8 byte sequences. In Java, you have byte[] and String. The problem is not a technical one of how to represent strings, it's a social one of how to get all the library authors on a language to agree on a convention of how to handle encoding.

Re: How Python 3 Should Have Worked (2012)

#64
post #8

It's also really off putting for beginners that try to learn the language. Python 3 is being served as the main download when you search for it. And while you search for tutorials, most of them are in python 2 - and 80% of them DO NOT state whether they are for python 2 or python 3 ( because most were made during the python 2 times? ). So people try to learn to code with python 3.x and get frustrated because simplest…

This is the major result when you Google "python download": http://www.python.org/getit/ It is the same as this page: http://www.python.org/download/ That page gives fairly equal weight to the two versions (I guess that could have changed over time). Edit: It might make sense to have a warning about matching the interpreter version up with the tutorial, but clear wording for it is not obvious to me.

The reason for the "getit" page was, at least at one point, because the "download" page was blocked throughout China.

Re: How Python 3 Should Have Worked (2012)

#65

This misses the point of why Python 3 was invented: Unicode. Python 2's string handling is broken in the presence of unicode characters, often leading to subtle errors that wouldn't cause exceptions until far away from the place where the error was introduced, and oftentimes didn't produce exceptions at all, just wrong data. Strings were defined as sequences of bytes, and then provided a .decode method to convert the…

It's possible to just force everyone into unicode strings in a way that doesn't cause too much trouble. It's impossible to try to do this gradually via a future import, and it's impossible to work out all the potential scenarios where things would behave differently, and particularly the performance characteristics can change and pretty wildly at that, but it sounds like this still, as painful as it would be, sounds like a better approach than P2->P3.

Basically, all strings would internally be represented as unicode (OR as a byte array + an encoding, that might be a little too ambitious though), and it has 2 APIs that it can be accessed by. non-instance related operations (such as making new strings) similarly has 2 APIs. pre-unicode-switch code gets an API that emulates as best as is possible the behaviour as before, and post-unicode-switch code gets the full unicode API as exists now for unicode strings.

Even if encoding is flat out broken, if you put in a stream of chars and store that as , and then later you query this thing, the broken usually ends up undoing itself. Yes, this will fail spectacularly once you try to for example concatenate a string with broken encoding to a string with a different encoding, but presumably places that are international proof have switched to unicode strings long ago, and places that aren't really aware of the importance of anything that isn't in ASCII 32-127 will find all their code to magically 'just work'.

Similar things should be possible for the iterator business. Internally things are iterators, but if ever any P2 code touches them, they turn into a (memory hogging, etc) list. yes, of course, this will flat out break if you attempt to pass an infinite iterator to code that isn't used to it, but these are all transitional pains, and the key point is: As long as you don't fork the pre-big-switch version (beyond security updates), sooner rather than later libraries will fix the bugs, or their community of users will just die out as someone else writes a new one to fill the void.

Painful? yes. very. We've seen this in programming land before. Java5 introduced generics and as a result any interaction with pretty much any library written before it resulted in a cavalcade of unsafe/raw warnings. It sucked. Huge communities stuck to 1.4 (IBM WebSphere notably stayed there for almost half a decade before moving on to 1.5).

But, today? Libraries have upgraded or have been replaced. The chance you still run into pre-generics code is tiny. It took long, it sucked, yadayada, but for all intents and purposes that was java's Python2->Python3, and everyone is on Python3 at this point. In the same time span as P2->P3, roughly speaking, and that seems like it's nowhere near complete.

Re: How Python 3 Should Have Worked (2012)

#66
post #37
post #32

Earlier quoted context omitted.

>I hate implicit hidden defaults and assumptions. Your implicit hidden default is my handy abbreviation. Imagine if we didn't have contractions in English. For that previous sentence, is it really at all frightening that the "didn't" meant "did not" but we haven't (ha) written it out? Could it possibly mean anything else? Wouldn't English be all the more stilted and ugly if there were only one explicit and verbose wa…

This just seems like flamewar material, much in the same vein as programmers (brogrammers?) that get red in the face over Python cramping their style by forcing structure via whitespace.

Please do not confuse dislike of semantic whitespace for structure-less preferences. I want my blocks to be explicit so the computer can know the difference between between incorrect indentation and a block ending. I also do not want to make tabs versus spaces any more flamewar inducing than it already is.

Re: How Python 3 Should Have Worked (2012)

#67
post #56

This misses the point of why Python 3 was invented: Unicode. Python 2's string handling is broken in the presence of unicode characters, often leading to subtle errors that wouldn't cause exceptions until far away from the place where the error was introduced, and oftentimes didn't produce exceptions at all, just wrong data. Strings were defined as sequences of bytes, and then provided a .decode method to convert the…

It's exceedingly rare that you need a sequence of codepoints. As evidence, consider that the native string types in many languages with pervasive Unicode support, such as Java and C#, do not provide a codepoint sequence. Programs generally need to deal with textual data at one of three levels: 1. Manipulate strings and substrings. e.g., "does string x contain substring y"? Byte sequences of utf8 data are fine for thi…

This is the first time I've ever seen someone write with an understanding of combining characters, glyphs, codepoints vs encoding of said codepoints - and yet arrive at this conclusion.

What's the largest codebase you tried a unicode-ification project on? It's a nightmare unless you keep de/encoding as close to the i/o operations as possible.

I can't understand how you've ever found it just as easy to do "string x contain substring y" on bytes vs uc strings. Any case-insensitive test will fail miserably unless you only ever see ASCII input. Then there's sorting and tokenization. Oh god, the sorting bugs...

Even measuring the length of string is a miserable fail. And blind substitution of utf8 bytes horribly mangles the output causing mysterious segfaults or silent corruption.

On a large codebase, programmers can't keep track of what encoding is being used in which parts of the code. Eg. Let's allow the users to specify input file encoding! But our OS does filenames in UTF16-LE. And the Web API is UTF-8... nasty stuff. It's far saner to use character strings everywhere except immediately after/before I/O operations.

Re: How Python 3 Should Have Worked (2012)

#68
post #51

Earlier quoted context omitted.

>@classmethod and @staticmethod are somewhat important language features And strings aren't? Whatever, I understand that there are tradeoffs and to people that only use Python, the aesthetic blemishes tend to matter less. To people that use it in the context of other languages, it sticks out like a bad paint job or a prominent stain. This, and many more opinions than I could express succinctly, are on the reddit thre…

> And strings aren't? Not sure how explicit self breaks strings? You've lost me on the way I'm afraid. If we're quoting Reddit comments at each other: > You may be used to a different kind of magic. Perhaps the magic of a variable called this appearing inside your method, or the magic in which an un-prefixed variable name somevar is sometimes a local but at other times a member variable this.somevar. So it may take a…

Re: strings, I was referring to what I said already and you quoted only part of it:

> It's odd to me that they were willing to break so much code with 3.0 (by "fixing" strings and Unicode) but a couple decorators held this one back.

Re: How Python 3 Should Have Worked (2012)

#69
post #68

Earlier quoted context omitted.

> And strings aren't? Not sure how explicit self breaks strings? You've lost me on the way I'm afraid. If we're quoting Reddit comments at each other: > You may be used to a different kind of magic. Perhaps the magic of a variable called this appearing inside your method, or the magic in which an un-prefixed variable name somevar is sometimes a local but at other times a member variable this.somevar. So it may take a…

Re: strings, I was referring to what I said already and you quoted only part of it: > It's odd to me that they were willing to break so much code with 3.0 (by "fixing" strings and Unicode) but a couple decorators held this one back.

How much code did the new strings break, exactly? What code it may have broken most likely benefits from the new unicode treatment, it was the worst thing about Python strings.

Removing self would be a far more massive change - it wouldn't just impact certain strings, it would impact every Python class.

Post reply on HN