Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Grumpy: Go running Python

opensource.googleblog.com

Post reply on HN
Print view View original

151–160 of 463 posts

Re: Grumpy: Go running Python

#151

Post by hueving »

Twirrim wrote: post #5

This seems like an odd engineering choice. Presumably the effort to create a python->go translator would be non-trivial. Why not just start rewriting components into Go, and migrating them out of Python, leaving python as essentially the presentation layer at most?

I take it you've never worked on a large code base with a large team? Even ignoring the actual task of rewriting the logic in a different language with different idioms, the change is going to be a massive hit to productivity for many months as people learn the new architecture of the rewrite, the tooling, and idiomatic Go.

For a project the size of YouTube, that will be millions of dollars of engineering hours and weeks/months of lost productivity for an unknown gain and almost guaranteed bugs. It's a terrible value proposition so it's better to squeeze every last drop of performance out of the code base you have, which at the scale if this project includes paying engineer(s) to work on a completely new runtime.

Re: Grumpy: Go running Python

#152

Post by devy »

ericfrederich wrote: post #25

Python needs a new runtime. This talk shows how bad of shape it's really in. https://www.youtube.com/watch?v=qCGofLIzX6g&list=PLRdS-n5seL... Basically, the language doesn't have a "spec" per-se. The language is whatever the defacto CPython implementation happens to do within it's giant eval loop. Another great talk about CPython internals: http://pgbovine.net/cpython-internals.htm

> Basically, the language doesn't have a "spec" per-se.

It does[1]. And process of improving it is called PEP[2].

[1]: https://docs.python.org/

[2]: https://en.wikipedia.org/wiki/Python_(programming_language)#...

Re: Grumpy: Go running Python

#153

Post by Antwan »

"So we asked ourselves a crazy question: What if we were to implement an alternative ?"

As always with Google...

What if you humbly contribute to open source projects rather than creating new stuffs, labelled with your own brand, controlled by your own engineers, and with your own design choices, however good they may be ?

Re: Grumpy: Go running Python

#154

Post by bsaul »

kodablah wrote: post #96
stiff wrote: post #78

It's actually a transpiler written in Python that generates Go code.

Nice. I've been toying with doing similar for the JVM [0]. I think people are coming to realize that while Go is not a great/powerful language (IMO), the runtime is great. I would not be surprised if more and more people start targetting Go if they want a GC and cross platform static compilation w/out LLVM complications and get a nice stdlib for free. 0 - https://github.com/cretz/goahead

First to write a language that is basically go + generics wins

Re: Grumpy: Go running Python

#155

Post by weberc2 »

TillE wrote: post #66
Animats wrote: post #48

- Amusingly, it runs Python 2.7, even though this project started long after Python 3.x came out. - It's a hard-code compiler, not an interpreter written in Go. That implies some restrictions, but the documentation doesn't say much about what they are. PyPy jumps through hoops to make all of Python's self modification at run-time features work, complicating PyPy enormously. Nobody uses that stuff in production code,…

> Nobody uses that stuff in production code Nobody uses the features of Python which make it a dynamic language? Google must write some really weird Python if their compiler is that strict.

I'm not sure what is meant by "dynamic language" in that sentence, but examining the compiler output, it supports the features of Python which I think of when I think of "dynamic language" (e.g., a class `Foo` gets compiled into a runtime `*Object` with collections of properties and methods, not into a `type Foo struct` with fixed fields and methods).

Re: Grumpy: Go running Python

#156

Post by coldtea »

dbaupp wrote: post #148
coldtea wrote: post #125

Earlier quoted context omitted.

> and everyone is writing Python 3 (or should be) You'd be surprised. If anything, the numbers show the opposite. The vast majority of Python codebases, legacy or new, are 2.7 or older.

Which numbers? People keep saying things like that, but things like the Python 3 Wall of Superpowers https://python3wos.appspot.com/ don't seem to support it. Do you have something more concrete?

The "Wall" is just numbers for Python3, without context for how it compares to Python 2.

For 2.7 Pypy reported 419,227,040 downloads for 2016.

At the same time, for ALL 3.x versions combined (up to 3.6) there are just: ~52 million downloads.

That's 1/8th of the Python 2 downloads.

Re: Grumpy: Go running Python

#157

Post by Twirrim »

zigzigzag wrote: post #89

How does this compare to Jython? The blog post says they looked at alternative runtimes but didn't like that they had tradeoffs (implying that Grumpy has no tradeoffs??). But Jython has been around for quite a while and also has no GIL: http://www.jython.org/jythonbook/en/1.0/Concurrency.html It can also handle Python's dynamic aspects.

Every time I've tried to use Jython, I've found it won't work with pre-existing code all that well.

In part it has exposed CPython "implementation quirks" that people were wittingly or otherwise taking advantage of. In other cases there doesn't seem to be obvious reasons for the differences and has required special-casing the python code to handle it.

It has been great with code written from scratch, specifically for it.

Re: Grumpy: Go running Python

#158

Post by Certhas »

coldtea wrote: post #125
cdelsolar wrote: post #110

Earlier quoted context omitted.

That is pretty ridiculous. Pretty much all major libraries are Python 3 compatible and everyone is writing Python 3 (or should be). (Yes, I'm still on Python 2 but moving soon).

> and everyone is writing Python 3 (or should be) You'd be surprised. If anything, the numbers show the opposite. The vast majority of Python codebases, legacy or new, are 2.7 or older.

There's plenty of 2.7 out there, but we're moving over slowly, basically due to the nice function annotation/type checking work. That's to me the first really compelling reason to use the latter.

Re: Grumpy: Go running Python

#159

Post by jamestimmins »

Does anyone know if there is an easy way to get involved in open source projects like this? The readme mentions adding PRs, but as someone that doesn't have much experience working with open source projects like this, I don't even know where to begin. It sounds like an incredible learning opportunity though.

Re: Grumpy: Go running Python

#160

Post by Spiritus »

lelandbatey wrote: post #100
sfifs wrote: post #27

The main reason I moved from coding in Python to Go as my main language many years back is because concurrency was such a pain in standard Python (the other was compile time error checking). It's interesting to see the same pain has now made caused the runtime itself to be implemented in Go. It's a pity C extensions (often used in scientific computing) are not supported but Go does have support via CGO, so maybe some…

I "grew up" on Python, then wrote a whole bunch of Go for my job. Then this past Autumn re-visited Python to implement a networked terminal based game[1]. With what I learned about Go and concurrency, I would say that currently in Python, writing concurrent code is not very hard, and is as close to Go as you can get without actually just writing Go. Now, you may be saying "but Python has the GIL, how can concurrency…

Using Gevent[1] is quite similar to Go when it comes to concurrency in Python.

[1] http://sdiehl.github.io/gevent-tutorial/

Post reply on HN