Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Grumpy: Go running Python

opensource.googleblog.com

Post reply on HN
Print view View original

331–340 of 463 posts

Re: Grumpy: Go running Python

#331

Post by cthalupa »

sergiotapia wrote: post #320
Animats wrote: post #48

- Amusingly, it runs Python 2.7, even though this project started long after Python 3.x came out. - It's a hard-code compiler, not an interpreter written in Go. That implies some restrictions, but the documentation doesn't say much about what they are. PyPy jumps through hoops to make all of Python's self modification at run-time features work, complicating PyPy enormously. Nobody uses that stuff in production code,…

I was in second year of college and the Python 2 vs Python 3 was a couple of years running. Is this fight -still- not resolved? I'm not a python developer so I'm out of the loop.

The arguing will continue for years after Python 2 is legitimately dead, but the shift has been happening and will continue to happen. Python3 is the future, and more and more new projects are being started with it, and more and more legacy 2.7 codebases are being moved to Python 3 or deprecated in favor of Python 3 replacements.

Re: Grumpy: Go running Python

#332

Post by guitarbill »

lomnakkus wrote: post #282
devy wrote: post #152

Earlier quoted context omitted.

> Basically, the language doesn't have a "spec" per-se. It does[1]. And process of improving it is called PEP[2]. [1]: https://docs.python.org/ [2]: https://en.wikipedia.org/wiki/Python_(programming_language)#...

Uh, what? Claiming that your [1] is in any way a specification for the language is utterly absurd. It's far too vague. (Compare to even an IETF RFC, and you'll see what I mean. If you want to compare to a real language spec, compare to ISO C++.)

> to a real language spec, compare to ISO C++.

No thanks. Written specs can always have interesting implications or undefined behaviour. Just because it's written in a more verbose language (English) doesn't mean it's less vague.

E.g. GGC is the de facto C spec for many. Code/platforms as spec makes more sense and is easier to maintain/update, with quicker iterations of language features (c.f. Ruby/Python to C++).

Re: Grumpy: Go running Python

#333

Post by zardeh »

X-Istence wrote: post #288
zardeh wrote: post #225

Earlier quoted context omitted.

You could perhaps write a feature-bare clone. Here's some basic features: - search (the various search pages) - trending - channels - subscription - video uploads - history - comments - likes - upload - video editing (most of this is in browser, but still) - livestream - video analytics - payment and ad management - video comment review and moderation - translation - captioning - video replication, multiscaling, cach…

> resolution management based on network speed Not so much a feature of YouTube as it is a feature of HLS/Dash... but yes, it means you've gotta transcode the source video into multiple different bitrates.

Even outside of that, I can override the bitrate in some instances. But yeah, I was mainly talking about the necessary replication, cache, and transcoding (and I actually forgot about the multiple codecs, which they also provide, in addition to multiple resolutions/bitrates)

Re: Grumpy: Go running Python

#334

Post by kristianp »

nilved wrote: post #215
dragonwriter wrote: post #72

Earlier quoted context omitted.

Creating a new Python runtime in Go is obviously non-trivial, but so is rewriting the heart of YouTube in Go. Heck, YouTube may be a bigger codebase than Python for all I know. And there's a lot more side benefit in a better multithreading Python runtime for Google for other Python code Google has (or hosts), whereas the benefits of a YouTube rewrite are more narrowly limited to YouTube.

Just because the site is popular doesn't mean the codebase is big. It's a simple site. I could probably write YouTube in Go in an afternoon and I don't even know Go.

See "Code: It's Trivial" https://blog.codinghorror.com/code-its-trivial/

Re: Grumpy: Go running Python

#335

Post by zardeh »

omaranto wrote: post #294
zardeh wrote: post #58

Earlier quoted context omitted.

It seems to be that it pesudo-transpiles python to go and compiles that down using a normal go toolchain.

What's the difference between transpiling and pesudo-transpiling? (Even if you meant pseudo-transpiling, I still don't know what the difference between that and transpiling is.)

That was a typo. I meant pseudo. I don't actually know that there is any difference in this case. I made that statement hastily. It transpiles, nothing pseudo about it.

Re: Grumpy: Go running Python

#336

Post by toyg »

zigzigzag wrote: post #166
Twirrim wrote: post #157

Earlier quoted context omitted.

Every time I've tried to use Jython, I've found it won't work with pre-existing code all that well. In part it has exposed CPython "implementation quirks" that people were wittingly or otherwise taking advantage of. In other cases there doesn't seem to be obvious reasons for the differences and has required special-casing the python code to handle it. It has been great with code written from scratch, specifically for…

That sounds like the kind of issue you'd have with any reimplementation of CPython though. It sounds like Grumpy doesn't support quite a few things, so I still wonder how they compare and why developing a new runtime was considered easier than reusing Jython.

There might be a strategic element: Jython development has been erratic throughout the years.

Re: Grumpy: Go running Python

#337

Post by guitarbill »

MBCook wrote: post #13
brian_herman wrote: post #2

No C extensions boo.

But isn't that really what lets them do this? In following PyPy (and other alternate Python discussions) it seems like eliminating the GIL and getting better performance out of Python, even in C, isn't that hard if you drop the C extensions . As soon as something can see into the guts of the interpreter you have to maintain compatibility which is a pain/waste. Worse than that is that view wasn't designed for multi-th…

The irony is that some C extensions exist because they have better performance than Python.

On the other hand, there are many C extensions which are just interop/wrappers for existing C code. I think no language is naive enough to think they can get away without C interop.

C extensions are actually pretty nice because you can wrap the C code into idiomatic Python. With ctypes, you have to e.g. maintain two structure definitions, which is very problematic for some codebases.

> In following PyPy (and other alternate Python discussions) it seems like eliminating the GIL and getting better performance out of Python, even in C, isn't that hard if you drop the C extensions.

AFAIK, many projects initially struggle to even achieve performance parity with CPython. The GIL isn't evil, it's a simple solution to a hard problem. But at this point, a complex solution to a hard problem is better if it's faster, and some people care more about speed than C interop (and vice versa).

What would be awesome is a Python runtime that could run with fine-grain locking until a C extension is loaded, and then continue with coarse locking. But the problem is still there's no upgrade path for C interop.

The only thing I can think of is using type annotations and something like Cython's cdef to write Python-implementation-independent C interop that doesn't suck as bad as ctypes. Then the Python rumtime could also lock the arguments at a very fine level while the function is being called.

Re: Grumpy: Go running Python

#338

Post by est »

BuckRogers wrote: post #257
Pxtl wrote: post #251

I can't help but see this balkanization of Python as a sign that the core language is falling apart. How many interpreters are there now? And how many of them have even close to 100% compatibility with Python 2.7 or 3.N? Guido has lost control of the language, but has he's still officially the BDFL there's no real standardization body. His stubborn view on functional mechanisms have held the language back syntactical…

GVR, the PSF and the core dev team overestimated their influence. They still truly believe the majority will come around to Python3. I agree with your sentiments and balkanization is the right word. Guido won't even read these comments. He thinks it's all some unjust slander and nonsense that will be forgotten in 3 years. :) It is a good time to jump off the Python train in general, and I say that as someone invested…

> They still truly believe the majority will come around to Python3.

People will upgrade if they make py3 more appealing, something like a 20% speed boost would be nice.

Re: Grumpy: Go running Python

#339

Post by merb »

trotterdylan wrote: post #61
Animats wrote: post #48

- Amusingly, it runs Python 2.7, even though this project started long after Python 3.x came out. - It's a hard-code compiler, not an interpreter written in Go. That implies some restrictions, but the documentation doesn't say much about what they are. PyPy jumps through hoops to make all of Python's self modification at run-time features work, complicating PyPy enormously. Nobody uses that stuff in production code,…

> Amusingly, it runs Python 2.7, even though this project started long after Python 3.x came out. Basically, we needed to support a large existing Python 2.7 codebase. See discussion here: https://github.com/google/grumpy/issues/1 > It's a hard-code compiler, not an interpreter written in Go. That implies some restrictions, but the documentation doesn't say much about what they are. PyPy jumps through hoops to make a…

> Basically, exec and eval don't work.

Couldn't they supported with a slower runtime implementation? I mean I still love the idea and actually like the idea.

Re: Grumpy: Go running Python

#340

Post by foxylad »

jonasvp wrote: post #176
brianwawok wrote: post #163

Earlier quoted context omitted.

Are namedtuples that popular? They always felt awkward to me. If some temp variable with multiple values inside a loop, I either use normal tuple or a dict. If passing data around a dict or a real class. I never got the huge win from namedtuple?

Attrs ( https://attrs.readthedocs.io/ ) replaced namedtuple for us (and many others). It's slightly more verbose but allows all class goodness such as methods, attribute validation, etc.

Aaaaaarrrrrgggggh! I've had that particular itch for every one of my ten years with python, and at last I get to scratch it!

Thanks so much for bringing it up.

Post reply on HN