Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Grumpy: Go running Python

opensource.googleblog.com

Post reply on HN
Print view View original

321–330 of 463 posts

Re: Grumpy: Go running Python

#321

Post by lomnakkus »

orf wrote: post #314
lomnakkus wrote: post #282

Earlier quoted context omitted.

Uh, what? Claiming that your [1] is in any way a specification for the language is utterly absurd. It's far too vague. (Compare to even an IETF RFC, and you'll see what I mean. If you want to compare to a real language spec, compare to ISO C++.)

CPython is the spec (or really more the CPython test suite). Just like the Ruby MRI. It's a simple, plain interpreter without many frills, and to add or remove a feature you have to submit a PEP which goes through a specification process. Python started as a one-man-band project and of course didn't have a specification.

Yes, but that's what the PP was arguing wasn't the case (by linking to a "spec"). I'm not sure why you're restating the obvious (OPs position).

Re: Grumpy: Go running Python

#322

Post by BuckRogers »

mdaniel wrote: post #273
BuckRogers wrote: post #257

Earlier quoted context omitted.

GVR, the PSF and the core dev team overestimated their influence. They still truly believe the majority will come around to Python3. I agree with your sentiments and balkanization is the right word. Guido won't even read these comments. He thinks it's all some unjust slander and nonsense that will be forgotten in 3 years. :) It is a good time to jump off the Python train in general, and I say that as someone invested…

> Here's to hoping Grumpy takes on a life of its own and is the new de facto Python. I can appreciate you have that opinion, but I'll be livid if that's true - the last thing in the entire world I want is to do battle with dependencies and the very, very strict/opinionated Go build system. If your idea is that all Py27 is just transpiled into Go and then jettisoned, that's fine, but keeping one foot in each world sou…

Go compiles down to native code (x86, ARM assembly etc). That's what Grumpy code generates as well but it still needs to be maintained in the original Python2 or Go (depending on whatever your original source is). What I'm suggesting is that Python3 is jettisoned and Grumpy takes the forsaken throne that Python left behind.

One thing is clear with all these new compilers/runtimes, you want to be writing Python2 syntax because that's where all the action is. I hope Grumpy succeeds and new features are added and becomes it's own ecosystem that plays nicely with Go code. These folks at Google have really done what Guido & Co should've done.

This is Python3 as most of us wanted it to be, it's worth rewriting all your code for... but you don't even have to do that. Valid Python2 is Grumpy already. I don't know what else I'd want. It compiles existing Python2 AND offers a legitimate upgrade from CPython at the same time.

As far as all of the lost C extensions? You won't need them with the performance that the Go runtime has. That's been the answer all this time, not maintaining C-extension compatibility.

They nailed this thing, it's the answer to "what's the future of Python?" that everyone has been wondering for the past 9 years.

Re: Grumpy: Go running Python

#323

Post by statsmatscats »

trotterdylan wrote: post #201
statsmatscats wrote: post #150

Earlier quoted context omitted.

Have you thought about using type hints to help type inference? Some work on that is discussed here. I would love a dropbox google colab (though also targeting 3.x :) ) https://github.com/python/mypy/issues/1862

Yes, leveraging type hints for optimization purposes is a long term goal. Thanks for pointing me to that issue, I'll keep an eye on it. One of the goals of open sourcing was to get feedback and work with outside folks so I'm definitely open to collaboration!

Awesome!

Re: Grumpy: Go running Python

#324

Post by weberc2 »

emmelaich wrote: post #315
weberc2 wrote: post #259

Earlier quoted context omitted.

Interesting to note that `print()` is supported out of the box--no need to `from __future__ import print_function`.

Sure, but that works without from `__future__ import print_function` in Python2 as well. But it gives different output; the print() prints a tuple whereas the function print() prints a newline. Also compare print(1) and print(1,2) with and without the __future__ import.

Oh, weird. I could have sworn I've seen errors for using parens with `print` in Python 2...

Re: Grumpy: Go running Python

#325

Post by tiffanyh »

>> "The front-end server that drives youtube.com and YouTube’s APIs is primarily written in Python, and it serves millions of requests per second! YouTube’s front-end runs on CPython 2.7, so we’ve put a ton of work into improving the runtime and adapting our application to work optimally within it."

So has YouTube already migrated over to using Grumpy (and no longer running python in production)?

Re: Grumpy: Go running Python

#326

Post by eudoxus »

monokrome wrote: post #247
Animats wrote: post #48

- Amusingly, it runs Python 2.7, even though this project started long after Python 3.x came out. - It's a hard-code compiler, not an interpreter written in Go. That implies some restrictions, but the documentation doesn't say much about what they are. PyPy jumps through hoops to make all of Python's self modification at run-time features work, complicating PyPy enormously. Nobody uses that stuff in production code,…

It's written to run Python 2.7 because these problems are largely solved in Python 3, and needs solved for people on Python 2.x versions. "Upgrade to Python3" is the usual defense to that, but it's not really practical for large companies with software such as YouTube completely written in Python 2.x.

No, its written to support 2.7 because the majority of Google's code they want to over haul is in 2.7. I don't think much of this is fixed by python3, at the very least the speed benefits you get don't even compare. See the graph on the OP comparing # of threads using CPython and Grumpy

Re: Grumpy: Go running Python

#327

Post by bootload »

"The downside is less development and deployment flexibility, but it offers several advantages. For one, it creates optimisation opportunities at compile time via static program analysis."

Optimisation. This is a smart move, hard though. A compiler, written well allows the back end to improve the code. So the whole code base can improve with improved analysis.

"The biggest advantage is that interoperability with Go code becomes very powerful and straightforward: Grumpy programs can import Go packages just like Python modules!"

Extending Python (youtube codebase) with Go modules. That's interesting.

Re: Grumpy: Go running Python

#328

Post by trotterdylan »

gok wrote: post #296
trotterdylan wrote: post #80

Earlier quoted context omitted.

I think the CPython AST module is written as a C extension module so currently it's a no-go. I don't think there's a fundamental reason Grumpy couldn't run a pure Python AST module, though.

So I take that means Grumpy can't run itself?

Correct, Grumpy cannot yet run Grumpy :)

Re: Grumpy: Go running Python

#329

Post by trotterdylan »

naasking wrote: post #208
trotterdylan wrote: post #116

Earlier quoted context omitted.

Although Grumpy is compiled, it is just as dynamic as Python, in that method dispatch involves dictionary lookups, etc. The main reason why Grumpy's slower for most single threaded benchmarks is that most Python workloads involve creating and freeing a bunch of small Python objects. In Go, these objects are garbage that need to be GC'd in a very general way. In CPython, there are free lists, arenas and other optimiza…

> Although Grumpy is compiled, it is just as dynamic as Python, in that method dispatch involves dictionary lookups, etc. Right, I suppose I assumed that straightforward numerical-looking code would be translated to Go numerical code. Perhaps they just aren't that ambitious yet.

We hope to implement optimizations like you're describing eventually. But the core functionality needs a lot of work before we start down that path.

Re: Grumpy: Go running Python

#330

Post by guitarbill »

BuckRogers wrote: post #322
mdaniel wrote: post #273

Earlier quoted context omitted.

> Here's to hoping Grumpy takes on a life of its own and is the new de facto Python. I can appreciate you have that opinion, but I'll be livid if that's true - the last thing in the entire world I want is to do battle with dependencies and the very, very strict/opinionated Go build system. If your idea is that all Py27 is just transpiled into Go and then jettisoned, that's fine, but keeping one foot in each world sou…

Go compiles down to native code (x86, ARM assembly etc). That's what Grumpy code generates as well but it still needs to be maintained in the original Python2 or Go (depending on whatever your original source is). What I'm suggesting is that Python3 is jettisoned and Grumpy takes the forsaken throne that Python left behind. One thing is clear with all these new compilers/runtimes, you want to be writing Python2 synta…

> you want to be writing Python2 syntax because that's where all the action is.

> This is Python3 as most of us wanted it to be.

> Valid Python2 is Grumpy already.

> It compiles existing Python2 AND offers a legitimate upgrade from CPython at the same time.

> As far as all of the lost C extensions? You won't need them

None of the statements are true. You seem to be very confused what Grumpy can and can't do, and what the need of actual Python developers are.

The only benefit of Grumpy is speed (I don't think go "interop" counts). Now, that's a pretty big benefit for some, but comes with significant drawbacks and probably always will. Even though CPython is only the reference implementation, many clever people have worked to make it faster. Getting rid of the GIL is also very difficult. The easiest way to gain speed is to limit Python to a subset of features and then optimise for that. While this is a fair approach, hailing it as the future of Python is terribly misguided.

Post reply on HN