Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What’s New in Python 3.8

docs.python.org

Post reply on HN
Print view View original

371–380 of 381 posts

Re: What’s New in Python 3.8

#371

Post by tripzilch »

m463 wrote: post #164
emmelaich wrote: post #104

Earlier quoted context omitted.

As someone who has used Python for over 10 years and C, Perl, etc for over 20, I've been craving the walrus operator forever. I love the walrus operator with a love that is unholy.

I wonder what you think of "for else" loops. (although I think "else" should have been a different keyword)

Every time I come across a use case when the "for else" construct would be applicable, I decide against using it just because the "else" keyword is just very unintuitive to people not familiar with this rarely used construct ... so yeah, I too wish it was a different keyword :)

Re: What’s New in Python 3.8

#372

Post by hannibalhorn »

hannibalhorn wrote: post #291
CivBase wrote: post #83

Earlier quoted context omitted.

The f-strings from 3.6 are a (relatively) recent feature that I have absolutely loved. I'd go so far as to say they are my favorite feature introduced by Python 3. I'm also looking forward to PEP-554 [0], which allows for "subinterpreters" for running concurrent code without removing the GIL or incurring the overhead of subprocesses. [0] https://www.python.org/dev/peps/pep-0554/

Sounds like PEP-544 is actually included in 3.8 on a "provisional" basis, which I guess just means they reserve the right to change the API. That's a killer feature for me, thanks for pointing it out!

Correction: So the PEP said it'll be in 3.8, but apparently it's been postponed to 3.9.

Re: What’s New in Python 3.8

#373

Post by dharmab »

spott wrote: post #336
mintplant wrote: post #76

Buried in the notes for the `typing` module: > “Final” variables, functions, methods and classes. See PEP 591, typing.Final and typing.final(). The final qualifier instructs a static type checker to restrict subclassing, overriding, or reassignment: > pi: Final[float] = 3.1415926536 As I understand it, this means Python now has a way of marking variables as constant (though it doesn't propagate into the underlying va…

The thing is, it is only enforced in mypy... code that modifies a Final object (pi = 3 for example) later on will run fine.

Correct- as usual the annotation indicates intent, but is not enforced at runtime.

Re: What’s New in Python 3.8

#374

Post by eru »

joshuamorton wrote: post #347
eru wrote: post #204

Earlier quoted context omitted.

In Haskell we use sorted maps as persistent data structures. Persistent in this context means that the insert-method returns a new map and the old map is still around, if you need it. It's basically copy-on-write. The old and new map share all but O(log n) data. You could probably do something like that with an unsorted map, but it's a good fit for a sorted one.

Sure, but fix your comparison metric as "timestamp of insertion" (and forget for a moment that this isn't technically pure, you can fiddle to make it pure), and you still get all the CoW-niceness, but this is internal to the mapping type, my keys and values don't need to be comparable or orderable, only hashable. I'm given an ordering, the ordering is arbitrary, but I don't care, because CoW/persistent hash maps are…

Could you please explain in some more detail how that would work, and especially how (logical) timestamp of insertion would help at all?

Re: What’s New in Python 3.8

#375

Post by lolsal »

diminoten wrote: post #349
lolsal wrote: post #341

Earlier quoted context omitted.

Some people use software as a tool and aren't zealots about versions or constantly chasing updates.

Yeah, and even those people would snort derisively at a project designed to keep 2.7 alive like this. "Getting shit done" doesn't happen if you can't move forward eventually. I'd have substantial concerns hiring someone who can't get over Python 2.7.

I think it's less that someone "can't get over Python 2.7" and more "why should I?". Nothing in 3 except bytes/string handling is compelling. Lots of 2 libraries haven't been ported. There is no legal reason to move and no other time pressure.

I would argue that people have been "getting shit done" with C for decades, despite newer shinier flavors of the month popping up, so your argument doesn't hold water to me.

Re: What’s New in Python 3.8

#376

Post by joshuamorton »

eru wrote: post #374
joshuamorton wrote: post #347

Earlier quoted context omitted.

Sure, but fix your comparison metric as "timestamp of insertion" (and forget for a moment that this isn't technically pure, you can fiddle to make it pure), and you still get all the CoW-niceness, but this is internal to the mapping type, my keys and values don't need to be comparable or orderable, only hashable. I'm given an ordering, the ordering is arbitrary, but I don't care, because CoW/persistent hash maps are…

Could you please explain in some more detail how that would work, and especially how (logical) timestamp of insertion would help at all?

So on eway to think of a hash table/map is a set of tuples of . Imagine we extend that to . All of your conventional mapping methods (get/has/put) work on the key and value and ignore the timestamp. But anything that relies on iteration takes advantage of the timestamp[1], and iterates in timestamp order (or, in fact how this is normally done, is that you have a sparse array/map of and a dense array of . Whenever you insert a new key/value, you always append value to the end of the dense array (so that array is dense), and the key is hashed as normal, but the value in the hash table is a pointer into the dense array where the timestamp/value pair is stored. So internally, get is

    return (map[hash(key) % map_size])->value
or approximately that (I haven't written C in a while).

Then iteration over objects in the array is consistent: you just iterate over the dense array. Removal from the dense array is done by tombstoning, and possibly eventual compaction. This is essentially how python's current hash table implementation works.

IIRC, if you're table can assume only insertions, this actually becomes really, really nice as a persistent data structure, since you can replace the backing vector with a backing linked list, and you only really lose out on iteration speed. Then you further extend that by swapping the linked list to a tree, and you share the entire backing structure.

[1]: And you can use an increment-only mutation counter instead of a timestamp to make this pure.

Re: What’s New in Python 3.8

#377

Post by xapata »

j88439h84 wrote: post #251
xapata wrote: post #148

Earlier quoted context omitted.

Unfortunately, that'd mean a package manager for C, Fortran, etc. as well, since so many Python packages have non-Python dependencies. I've started using Docker for ensuring prod behaves the same as dev. Going from Mac dev to Linux prod would otherwise cause trouble.

Sounds like Conda.

Conda can't fix some problems, like Python 3.7 wanting ncurses >6, but a dependency wanting ncurses 5.9

Re: What’s New in Python 3.8

#378

Post by xapata »

eru wrote: post #197
xapata wrote: post #186

Earlier quoted context omitted.

I don't see the point of `final` without an optimizing compiler. Name mangling is sufficient for stashing references to avoid accidental side-effects of overriding.

You can guard yourself against overriding in the same module this way, too. For name mangling, I think you need to start your variable with underscores, and then it won't be accessible for reading outside the module either?

Nope, you don't need to start the name with underscores.

    class Foo:
        bar = 'public'
        __bar = bar # stashed

Re: What’s New in Python 3.8

#379

Post by dgrunwald »

hannibalhorn wrote: post #372
hannibalhorn wrote: post #291

Earlier quoted context omitted.

Sounds like PEP-544 is actually included in 3.8 on a "provisional" basis, which I guess just means they reserve the right to change the API. That's a killer feature for me, thanks for pointing it out!

Correction: So the PEP said it'll be in 3.8, but apparently it's been postponed to 3.9.

As someone familiar with the Python C extension API, I somewhat doubt that it'll make 3.9 either. Disclaimer: I'm only familiar with CPython as a user of the extension API; I have no idea on how the Python team plans to address the challenges I'm mentioning here, and what their current progress is.

The current extension module API encourages global state, e.g. types (and objects too) are allocated statically in a global C variable. For example, there is a global C variable `_Py_NoneStruct` that is the Python `None` value, and extension modules are accessing this variable directly.

Every use of this object needs to adjust its reference count, and that reference count is directly stored within the C global variable.

`_Py_NoneStruct` is currently even exposed in the PEP 384 stable ABI. Existing extension module binaries are commonly directly touching `_Py_NoneStruct.ob_refcnt` without any synchronization. Breaking the PEP 384 compatibility promise is fundamentally unavoidable here.

One of two things must happen: Alternative one: All refcount operations must be made atomic for thread-safety. These are really really common in the Python interpreter, but atomic operations are expensive on modern CPUs (especially if there's contention). But multiple threads using the value `None` would be quite common in Python code, so I doubt you'd gain any speed even at today's core counts -- in fact I'd expect the constant inter-CPU-communication for the refcounts to make everything slower than just using a single core with today's Python!

So alternative two, ensure no Python objects are shared between the subinterpreters. That's the plan. But that also means it's a breaking change for extension modules. And it's not just an ABI change (which would be handled by merely recompiling against the new headers). Any extension modules that do not yet support PEP 489 are already incompatible with subinterpreters, so that will take quite a bit of work until the ecosystem is upgraded. But there will probably also be some other breaking API changes. I think type objects are currently shared across subinterpreters, and those are frequently defined as a `Py_TypeObject` global variable in extension module code. Also, if every subinterpreter has its own GIL, extension modules calling `PyGILState_Ensure()` will have to specify which subinterpreter they will be using, so that the appropriate lock can be acquired.

My prediction: 3.9 may have the basic functionality, but it still won't be able to run on multiple cores concurrently. That will take a bunch of more work and breaking changes, and will likely be released as Python 4.0.

There will be another slow upgrade process ("my dependencies must upgrade before I can") until the Python ecosystem is multi-subinterpreter-compatible. But at least this one only affects extension modules.

Re: What’s New in Python 3.8

#380

Post by m463 »

tripzilch wrote: post #371
m463 wrote: post #164

Earlier quoted context omitted.

I wonder what you think of "for else" loops. (although I think "else" should have been a different keyword)

Every time I come across a use case when the "for else" construct would be applicable, I decide against using it just because the "else" keyword is just very unintuitive to people not familiar with this rarely used construct ... so yeah, I too wish it was a different keyword :)

maybe we will get "The 3to4 tool will automatically adapt regretfully-named-keywords when converting your sources to Python 4."
Post reply on HN