Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Free-threaded CPython is ready to experiment with

labs.quansight.org

Post reply on HN
Print view View original

291–300 of 398 posts

Re: Free-threaded CPython is ready to experiment with

#291

Post by kstrauser »

hot_gril wrote: post #226
kstrauser wrote: post #103

Earlier quoted context omitted.

It’s hard to say because we’ve come up with a lot of ways to work around the fact that threaded Python has always sucked. Why? Because there’d been no demand to improve it. Why? Because no one used it. Why? Because it sucked. I’m looking forward to seeing how people use a Python that can be meaningfully threaded. While It may take a bit to built momentum, I suspect that in a few years there’ll be obvious use cases th…

Maybe a place to look for obvious use cases is in other languages. JS doesn't have threads, but Swift does. The reason I can't think of one is, free threads are most useful for full parallelism that isn't "embarrassingly parallel," otherwise IPC does fine. So far, I've rarely seen that. Best example I deal with was a networking project with lots of communication across threads, and that one was too performance-sensit…

That's the kind of thing I stumble across all the time. Indexing all the symbols in a codebase:

  results = Counter()
  for file in here.glob('*.py'):
      symbols = parse(file)
      results.update(symbols)
Scanning image metadata:

  for image in here.glob('*.png'):
      headers = png.Reader(image)
      ...
Now that I think about it, most of my use cases involve doing expensive things to all the files in a directory, but in ways where it'd be really sweet if I could do it all in the same process space instead of using a multiprocessing pool (which is otherwise an excellent way to skin that cat).

I've never let that stop me from getting the job done. There's always a way, and if we can't use tool A, then we'll make tool B work. It'll still be nice if it pans out that decent threading is at least an option.

Re: Free-threaded CPython is ready to experiment with

#292

Post by __MatrixMan__ »

pletnes wrote: post #274
__MatrixMan__ wrote: post #83

Earlier quoted context omitted.

Python's dependency management sucks because they're audacious enough to attempt packaging non-python dependencies. People always bring Maven up as a system that got it right, but Maven only does JVM things. I think the real solution here is to just only use python dependency management for python things and to use something like nix for everything else.

This is what we used to have and it was much worse. Source: lived that life 10-15 y ago.

15y ago I was using apt-get to manage my c++ dependencies with no way of keeping track of which dependency went with which project. It was indeed pretty awful.

Now when I cd into a project, direnv + nix notices the dependencies that that project needs and makes them available, whatever their language. When I cd into a different project, I get an entirely different set of dependencies. There's pretty much nothing installed with system scope. Just a shell and an editor.

Both of these are language agnostic, but the level of encapsulation is quite different and one is much better than that other. (There are still plenty of problems, but they can be fixed with a commit instead of a change of habit.)

The idea that every language needs a different package manager and that each of those needs to package everything that might my useful when called from that language whether or not it is written in that language... It just doesn't scale.

Re: Free-threaded CPython is ready to experiment with

#293

Post by neonsunset »

zarzavat wrote: post #290
mondrian wrote: post #287

Earlier quoted context omitted.

The critique is that "static typing" is not really the right term to use, even if preceded by "optional". "Type hinting" or "gradual typing" maybe. In static typing the types of variables don't change during execution.

If there’s any checking of types before program runs then it’s static typing. Gradual typing is a form of static typing that allows you to apply static types to only part of the code. I’m not sure what you mean by variables not changing types during execution in statically typed languages. In many statically typed languages variables don’t exist at runtime, they get mapped to registers or stack operations. Variables…

'dynamic' in C# is considered a design mistake and pretty much no codebase uses it.

On the other hand F# is much closer to the kind of gradual typing you are discussing.

Re: Free-threaded CPython is ready to experiment with

#294

Post by GTP »

kortex wrote: post #140
GTP wrote: post #34

Earlier quoted context omitted.

> BTW, Typescript also does not enforce types at runtime. Heck, C++ does not enforce types at runtime either. It does not mean that their static typing systems don't help during at development time. Speaking of C here as I don't have web development experience. The static type system does help, but in this case, it's the compiler doing the check at compile time to spare you many surprises at runtime. And it's part of…

https://github.com/python/mypy > Python itself doesn't do that The type syntax is python. MyPy is part of Python. It's maintained by the python foundation. Mypy is not part of CPython because modularity is good, the same way that ANSI C doesn't compile anything, that's what gcc, clang, etc are for. Mojo is literally exactly the same way, the types are optional, and the tooling handles type checking and compilation.

> Mojo is literally exactly the same way.

No, because in Mojo, type checking is part of the language specification: you need no external tool for that. Python defines a syntax that can be used for type checking, but you need an external tool to do that. GCC does type checking because it's defined in the language specification. You would have a situation analogous to Python only if you needed GCC + some other tool for type checking. This isn't the case.

Re: Free-threaded CPython is ready to experiment with

#295

Post by ipsod »

nly wrote: post #250
ipsod wrote: post #185

Earlier quoted context omitted.

Sure! I tried pybind11, and some other things. cppyy was the first I tried that didn't give me any trouble. I've been using it pretty heavily for about a year, and still no trouble.

Last I checked cppyy didn't build any code with optimisations enabled (same as cling)

It seems like you might be able to enable some optimizations with EXTRA_CLING_ARGS. Since it's based on cling, it's probably subject to whatever limitations cling has.

To be honest, I don't know much about the speed, as my use-case isn't speeding up slow code.

Re: Free-threaded CPython is ready to experiment with

#296

Post by GTP »

infamia wrote: post #129
VeejayRampay wrote: post #37

Earlier quoted context omitted.

python will never be "properly typed" what it has is "type hints" which is way to have richer integration with type checkers and your IDE, but will never offer more than that as is

> what it has is "type hints" which is way to have richer integration with type checkers and your IDE, but will never offer more than that as is Python is strongly typed and it's interpreter is type aware of it's variables, so you're probably overreaching with that statement. Because Python's internals are type aware, it's how folks are able to create type checkers like mypy and pydantic both written in Python. Maybe…

I don't think you can say that a language is strongly typed if only the language's internals are. The Python interpreter prevents you from summing an integer to a string, but only at runtime when in many cases it's already too late. A strongly typed language would warn you much sooner.

Re: Free-threaded CPython is ready to experiment with

#297

Post by kortex »

cwalv wrote: post #176
kortex wrote: post #144

Earlier quoted context omitted.

That's true of most compiled languages. Unless we are talking about asserts, reflection, I think type erasure, and maybe a few other concepts, language runtimes don't check types. C does not check types at runtime. You compile it and then rely on control of invariants and data flow to keep everything on rails. In python, this is tricky because everything is behind at least one layer of indirection, and thus virtually…

>> and the interpreter is not throwing an error which was my point. > That's true of most compiled languages True of most statically typed languages (usually no need to check at runtime), but not true in Python or other dynamically typed languages. Python would have been unusable for decades (prior to typehints) if that was true.

That's just reflection. That's a feature of code, not language runtime. I think there are some languages which in fact have type checking in the runtime as a bona-fide feature. Most won't, unless you do something like isinstance()

Re: Free-threaded CPython is ready to experiment with

#298

Post by GTP »

andrewaylett wrote: post #58
GTP wrote: post #46

Earlier quoted context omitted.

Then we have very different ideas of what proper typing is :D Look at this function, can you tell me what it does? def plus(x, y): return x+y If your answer is among the lines of "It returns the sum x and y" then I would ask you who said that x and y are numbers. If these are strings, it concatenates them. If instead you pass a string and a number, you will get a runtime exception. So not only you can't tell what a f…

It calls x.__add__(y). Python types are strictly specified, but also dynamic. You don't need static types in order to have strict types, and indeed just because you've got static types (in TS, for example) doesn't mean you have strict types. A Python string is always a string, nothing is going to magically turn it into a number just because it's a string representation of a number. The same (sadly) can't be said of J…

> It calls x.__add__(y)

Your answer doesn't solve the problem, it just moves it: can you tell me what x. __add__(y) does?

Re: Free-threaded CPython is ready to experiment with

#299

Post by mondrian »

zarzavat wrote: post #290
mondrian wrote: post #287

Earlier quoted context omitted.

The critique is that "static typing" is not really the right term to use, even if preceded by "optional". "Type hinting" or "gradual typing" maybe. In static typing the types of variables don't change during execution.

If there’s any checking of types before program runs then it’s static typing. Gradual typing is a form of static typing that allows you to apply static types to only part of the code. I’m not sure what you mean by variables not changing types during execution in statically typed languages. In many statically typed languages variables don’t exist at runtime, they get mapped to registers or stack operations. Variables…

Python is dynamically typed because it type-checks at runtime, regardless of annotations or what mypy said.

Re: Free-threaded CPython is ready to experiment with

#300

Post by GTP »

vhcr wrote: post #90
GTP wrote: post #46

Earlier quoted context omitted.

Then we have very different ideas of what proper typing is :D Look at this function, can you tell me what it does? def plus(x, y): return x+y If your answer is among the lines of "It returns the sum x and y" then I would ask you who said that x and y are numbers. If these are strings, it concatenates them. If instead you pass a string and a number, you will get a runtime exception. So not only you can't tell what a f…

In theory it's nice that the compiler would catch those kinds of problems, but in practice it doesn't matters.

It can matter also in practice. Once I was trying some Python ML model to generate images. My script ran for 20 minutes to then terminate with an exception when it arrived at the point of saving the result to a file. The reason is that I wanted to concatenate a counter to the file name, but forgot to wrap the integer into a call to str(). 20 minutes wasted for an error that other languages would have spotted before running the script.
Post reply on HN