I'm curious to why they started this project when there already is low hanging fruit that can speed up Python (e.g. pypy). What makes this better other than to satisfy the inner go-fanboy?
GIL still exists in pypy
Grumpy: Go running Python
71–80 of 463 posts
Re: Grumpy: Go running Python
#72This seems like an odd engineering choice. Presumably the effort to create a python->go translator would be non-trivial. Why not just start rewriting components into Go, and migrating them out of Python, leaving python as essentially the presentation layer at most?
Creating a new Python runtime in Go is obviously non-trivial, but so is rewriting the heart of YouTube in Go. Heck, YouTube may be a bigger codebase than Python for all I know.
And there's a lot more side benefit in a better multithreading Python runtime for Google for other Python code Google has (or hosts), whereas the benefits of a YouTube rewrite are more narrowly limited to YouTube.
Re: Grumpy: Go running Python
#73As someone who works on both python and go day to day, I find this to be quite interesting. Just tried this out on a reasonably complex project to see what it outputs. Looks like it only handles individual files and not any python imports in those files. So for now you have to manually convert each file in the project and put them into the correct location within the build/src/grumpy/lib directory to get your depende…
I'm sure as this gets hacked on they'll be able to support consuming imports and doing all the conversion to go recursively
Re: Grumpy: Go running Python
#74This actually pretty great. Even if you will never run your code in grumpy, it underlines the status of Python. Having large corps investing in Python will help pulling in new talent into the Python environment also it strengthens the ecosystem of Python.
Actually, I would interpret this as "CPython is broken for scalability. It's not worth trying to fork/fix it, we'd rather just roll our own runtime". Not exactly an endorsement. Sadly, the code was just dumped into a new Git repo, so no way to tell how many people contributed internally so far.
I agree and disagree ... on one side this may signal a shift away from Python. Something to help the migration effort and the ultimate goal is to write everything in Go.
... but then I don't see Python going anywhere anytime soon. Didn't Microsoft just start a project to get Python's runtime to use CoreCLR's JIT?
There was an article, can't find it now, about an upcoming Python renaissance saying there may be an influx of new interpreters. There's PyPy, Microsoft's CoreCLR thing, now this, etc. It seems people really want to program in Python so there is an effort to make it faster.
*edit: found the article: https://lwn.net/Articles/691070/
Re: Grumpy: Go running Python
#75There's no mention of whether Grumpy passes the CPython test suite. Until it doesn't, it's not a Python runtime, it's a compiler for a language with Python-like syntax. Compilers like this, from almost-Python to say C/C++, have existed for a while: Cython, Shedskin, Nuitka are some examples.
It does not, yet. The standard library is very incomplete at this point.
experimental ~= alpha, give the team a break
Re: Grumpy: Go running Python
#76Earlier quoted context omitted.
Ask yourself how much Python code you think Google has. Then multiply it by some large single-digit number, minimum. Then compare the effort of spending 30 seconds per line on that code vs. writing a new Python interpreter/compiler/runtime, especially when you can trivially get people who are capable of doing that. There's a reason why the large companies often end up working on new runtimes/interpreters/compilers li…
I wouldn't think that a tool for transpiling Python to Go would be harder than creating a whole new runtime. Most of the work is already done; you can get the compiled ASTs directly out of Python itself. Of course, if it started out as a "this seems like a cool project", that skews the "a is more efficient than b" ratio significantly.
Sorry, I think my post may have been excessively specific; I've edited it to say "writing a new Python interpreter/compiler/runtime" instead of just "writing a new Python interpreter". I was trying to make a generic point, not one specifically about a Python interpreter. (At least, if I'm interpreting your reply correctly.)
Re: Grumpy: Go running Python
#77[deleted]
Re: Grumpy: Go running Python
#78It's actually a transpiler written in Python that generates Go code.
Re: Grumpy: Go running Python
#79As someone who works on both python and go day to day, I find this to be quite interesting. Just tried this out on a reasonably complex project to see what it outputs. Looks like it only handles individual files and not any python imports in those files. So for now you have to manually convert each file in the project and put them into the correct location within the build/src/grumpy/lib directory to get your depende…
Thanks for trying it out! And sorry about the lacking documentation. I'll be fleshing it out over the next little while.
Your assessment is right: the grumpc compiler takes a single Python file and spits out a Go package. Incidentally, this means you can import a Python module into Go code pretty easily.
I don't have a ready solution for building a large existing project but I'll write up a quick doc to outline the process. The trickiest bit is that the Python statement "import foo.bar" translates to a Go import: import "prefix/foo/bar". Currently prefix always points at the grumpc/lib directory so that's one way to integrate your code, but I need to make it more configurable.
Re: Grumpy: Go running Python
#80Earlier quoted context omitted.
> Amusingly, it runs Python 2.7, even though this project started long after Python 3.x came out. Basically, we needed to support a large existing Python 2.7 codebase. See discussion here: https://github.com/google/grumpy/issues/1 > It's a hard-code compiler, not an interpreter written in Go. That implies some restrictions, but the documentation doesn't say much about what they are. PyPy jumps through hoops to make a…
> There are restrictions. I'll update the README to make note of them. Basically, exec and eval don't work. Since we don't use those in production code at Google, this seemed acceptable. I'm guessing pretty much the entire AST module is a no-go?
I think the CPython AST module is written as a C extension module so currently it's a no-go. I don't think there's a fundamental reason Grumpy couldn't run a pure Python AST module, though.