Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Grumpy: Go running Python

opensource.googleblog.com

Post reply on HN
Print view View original

351–360 of 463 posts

Re: Grumpy: Go running Python

#351

Post by elementry »

vram22 wrote: post #51
sfifs wrote: post #27

The main reason I moved from coding in Python to Go as my main language many years back is because concurrency was such a pain in standard Python (the other was compile time error checking). It's interesting to see the same pain has now made caused the runtime itself to be implemented in Go. It's a pity C extensions (often used in scientific computing) are not supported but Go does have support via CGO, so maybe some…

Does anyone know the technical reasons why C extensions are not (at least easily, apparently) supported by Go? Is it to do with Go's being a GC'd language? I would have thought that should not be a reason per se, since Python also has GC, but has plenty of extensions written in C. But I'm not a language internals expert. Also, further signs that GC may not be the reason, is that D also has GC, but can link to C libra…

> Does anyone know the technical reasons why C extensions are not (at least easily, apparently) supported by Go?

It's because Python's C API is inherently non-thread safe. The API lacks passing an interpreter pointer as a parameter (as Lua's API does for example). So Python is forced to use a terrible thread local storage hack involving the Global Interpreter Lock to swap interpreter instances which is insanely inefficient and limits compute-bound programs to a single thread.

Python 3.x had a chance to fix the API and do away with the GIL once and for all, but inexplicably they did not. There was a misguided notion that C extensions between 2.x and 3.x could be interoperable.

Re: Grumpy: Go running Python

#352

Post by hellofunk »

Pxtl wrote: post #251

I can't help but see this balkanization of Python as a sign that the core language is falling apart. How many interpreters are there now? And how many of them have even close to 100% compatibility with Python 2.7 or 3.N? Guido has lost control of the language, but has he's still officially the BDFL there's no real standardization body. His stubborn view on functional mechanisms have held the language back syntactical…

Actually, historical evidence suggests the opposite of your argument: multiple implementations of a language typically mean it is succeeding, not failing.

Re: Grumpy: Go running Python

#353

Post by beagle3 »

earthboundkid wrote: post #345
beagle3 wrote: post #242

Earlier quoted context omitted.

The GIL is not just for GC; it does coordinate access across threads, albeit at a very low level -- if two threads execute "mylist.append(v)" at the same time, it is the GIL that makes sure it actually works as expected, and from comments above it seems grumpy uses per-object locks for that.

That's not a good characterization of the GIL. It doesn't prevent races or make multithreaded mylist.append safe. It makes sure that mylist.append doesn't cause a segfault as the bytes in RAM are in an inconsistent state during an update. Beyond that, it doesn't really protect you from your bad threaded code.

(assuming mylist is a standard python list) it does prevent races inside mylist.append. It does make mylist.append safe.

When I wrote "append from two threads .. as expected" I meant "two items will be added, which one first is unspecified", and the GIL certainly takes care of that.

I agree it does not protect you from your bad threaded code - but then, nothing short of STM does (and even STM doesn't guarantee starvation in the general sense - nothing can).

Re: Grumpy: Go running Python

#354

Post by Animats »

weberc2 wrote: post #263
Animats wrote: post #112

Earlier quoted context omitted.

That's fascinating. It's creating run-time data structures similar to CPython's for data, and manipulating them with very general code. There seems to be a type comparable to Python's internal CObject, and it's used for most (all?) data. It's not generating Go that looks anything like human-written Go. There's no sign of type inference, although it's hard to tell from such a simple example. It's a lot like a Python r…

> Still, once you can do that, you can start optimizing, such as inferring that something is an integer and using ordinary Go arithmetic types. I was hoping for something more aggressive even, like compiling Python classes to Go structs so long as the program doesn't need the dynamic behavior. Alternatively, Grumpy could support declaring native Go types via some sort of pragma or a new `struct` keyword or some such,…

I'd expect to see that in time. If you analyze the whole program to find all the fields of an object and verify the absence of code which dynamically adds a field, you can then make it a struct of "CObject" like entries. Then, try type inference on the fields. Some will clearly be integers, booleans, floats, or strings. Those can be represented with type-specific representations.

If you can identify the built-in types, that's most of the potential win; you get to do hardware arithmetic. If you represent integers as 64 bits and check for overflow, you probably don't need bignum promotion outside of crypto code.

Re: Grumpy: Go running Python

#355

Post by simula67 »

LeifCarrotson wrote: post #180
coldtea wrote: post #156

Earlier quoted context omitted.

The "Wall" is just numbers for Python3, without context for how it compares to Python 2. For 2.7 Pypy reported 419,227,040 downloads for 2016. At the same time, for ALL 3.x versions combined (up to 3.6) there are just: ~52 million downloads. That's 1/8th of the Python 2 downloads.

Given that there are only 7.5 billion humans on the planet, and that rather significantly fewer than 1 in 20 people are PyPy-using developers, perhaps those numbers should be taken with a grain of salt? The message I would take from those statistics is that needing a fresh download of Pypy is less common among 3.x users than among 2.7 users, who apparently needed to reinstall from the web at least a few times a day d…

> The message I would take from those statistics is that needing a fresh download of Pypy is less common among 3.x users than among 2.7 users, who apparently needed to reinstall from the web at least a few times a day during 2016.

Occam's Razor would suggest that there are fewer Python 3 users

Re: Grumpy: Go running Python

#356

Post by int_19h »

BuckRogers wrote: post #249
int_19h wrote: post #147

Earlier quoted context omitted.

Everyone who likes the numerous new language features, or new syntax for old features, in Python 3?

Those can be built into Grumpy if anyone cares to do so. You don't need Python3 for that stuff. Someone, one developer, recently released a Python 2.8 that backported almost every new Python3 feature. I'm hoping it becomes a permanent fork of Python2 that uses the Go runtime. That would really be great and exactly what they've got now.

If you backport every new Python3 feature to Python2, it becomes Python3, by definition.

Re: Grumpy: Go running Python

#358

Post by pjmlp »

orf wrote: post #316
pjmlp wrote: post #223

Earlier quoted context omitted.

I do follow Julia development and I am aware that it isn't quite there, but at least their community does embrace JIT compilation, not like Python that PyPy is just yet another project, ignored by the reference implementation. > SciPy, which, you may not be aware, is basically C and Fortran code wrapped in Python API. Which for me personally means, that I would rather C and Fortran directly or better yet, a C++, .NET…

> Which for me personally means, that I would rather C and Fortran directly or better yet, a C++, .NET or Java binding to them. Being able to describe things with a syntax that looks almost like pseudocode and runs highly-optimized C/Fortran code to do heavy lifting has huge, huge advantages.

Scala, Clojure, F# also allow it, with the added benefit of industrial strength JIT/AOT compilers.

Re: Grumpy: Go running Python

#359

Post by loeg »

ensignavenger wrote: post #295
loeg wrote: post #248

Earlier quoted context omitted.

The community will fork the 2.7 codebase and continue to support it, even if Python.org EOLs it.

I can see "the community" doing security fixes, maybe some bug fixing and a few back ports, but so far I havn't seen much effort from any community to bring active development of new features.

It seems like the 2.7 community is happy enough without the new features -- just need the bugfixes and continued backwards compatibility to keep that segment happy.

From a new features perspective, the other reply's Placeholder is fascinating. (I haven't looked into it thoroughly yet.)

Re: Grumpy: Go running Python

#360

Post by tavert »

JPKab wrote: post #219
pjmlp wrote: post #181

Earlier quoted context omitted.

It might be, but that it is usually a consequence of not knowing any better or making use of existing libraries. Just like people learned 8-bit BASIC and went on to do business applications and games on it. I went Z80 ASM instead. Personally I would only use Python for shell scripting and advise for using Julia instead. Of course, others see it differently.

Julia? Look, I enjoy playing with Julia, but have you actually used it? Because most of the people I meet who turn up their nose at Python and say nice things about Julia haven't ever used Julia. It's got great potential but has some major gaps and warts, and is just plain dog shit slow for certain things compared to SciPy, which, you may not be aware, is basically C and Fortran code wrapped in Python API.

> is just plain dog shit slow for certain things compared to SciPy

Do you have real examples here or is this just FUD? SciPy is not known for using absolute state of the art algorithms or perfectly optimized implementations. It can be pretty easy to improve on the naively implemented or legacy pieces of SciPy, e.g. http://tullo.ch/articles/python-vs-julia/

Post reply on HN