Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Grumpy: Go running Python

opensource.googleblog.com

Post reply on HN
Print view View original

231–240 of 463 posts

Re: Grumpy: Go running Python

#231

Post by kstrauser »

kj01a wrote: post #203
cdelsolar wrote: post #110

Earlier quoted context omitted.

That is pretty ridiculous. Pretty much all major libraries are Python 3 compatible and everyone is writing Python 3 (or should be). (Yes, I'm still on Python 2 but moving soon).

>(Yes, I'm still on Python 2 but moving soon). I'm like really new to programming and I'm still just learning the basics, but I see this little addendum a lot from people who say everyone should be writing Python 3.

Here, I'll give you a counterexample:

We're on Python 3 for all new stuff, and are migrating the old whenever we can.

Re: Grumpy: Go running Python

#232

Post by kstrauser »

coldtea wrote: post #156
dbaupp wrote: post #148

Earlier quoted context omitted.

Which numbers? People keep saying things like that, but things like the Python 3 Wall of Superpowers https://python3wos.appspot.com/ don't seem to support it. Do you have something more concrete?

The "Wall" is just numbers for Python3, without context for how it compares to Python 2. For 2.7 Pypy reported 419,227,040 downloads for 2016. At the same time, for ALL 3.x versions combined (up to 3.6) there are just: ~52 million downloads. That's 1/8th of the Python 2 downloads.

A huge confounding factor: newer Py3 codebases are more likely to be built with newer pipeline tooling like devpi (to cache PyPI downloads), wheel (to cache locally-built packages), and Docker (which caches all the things).

Our legacy Python 2 build pipelines that we're actively moving off of hit PyPI far more often than our Py3 processes.

Re: Grumpy: Go running Python

#233

Post by [deleted] »

brianwawok wrote: post #220
justinsaccount wrote: post #186

Earlier quoted context omitted.

Doesn't work for everything, but you can subclass a namedtuple: from collections import namedtuple class Foo(namedtuple("Foo", "a b c")): @property def sum(self): return self.a + self.b + self.c f = Foo(1,2,3) print f.sum

That doesn't look super awesome to me. I.e. classes or attrs both seem better.

[deleted]

Re: Grumpy: Go running Python

#234

Post by petters »

nilved wrote: post #215
dragonwriter wrote: post #72

Earlier quoted context omitted.

Creating a new Python runtime in Go is obviously non-trivial, but so is rewriting the heart of YouTube in Go. Heck, YouTube may be a bigger codebase than Python for all I know. And there's a lot more side benefit in a better multithreading Python runtime for Google for other Python code Google has (or hosts), whereas the benefits of a YouTube rewrite are more narrowly limited to YouTube.

Just because the site is popular doesn't mean the codebase is big. It's a simple site. I could probably write YouTube in Go in an afternoon and I don't even know Go.

This comment made me wonder what you consider to be a complex site.

Re: Grumpy: Go running Python

#235

Post by derefr »

tedmielczarek wrote: post #193
poooogles wrote: post #59

Earlier quoted context omitted.

>- Amusingly, it runs Python 2.7, even though this project started long after Python 3.x came out. Python 2.7 is what's running at Google. Not really surprising they're looking at this considering the fast approaching end of (core dev) support for Python 2.7. Write an interpreter in another language and programatically port modules to Go. Seems pretty sensible to me.

Given the failure of their unladen-swallow work to make it into CPython, I think Google is probably tired of trying to make Python faster. Some of their stated goals with Go were to be a faster, compiled Python, so this makes a lot of sense for their use case. They face the choice of fixing all their existing Python code to run in Python 3 (which won't make anything faster), or just porting everything to a different…

> I don't know that this makes sense for anyone but Google

I'm not sure; as a Go developer, I kind of like the idea of having access to the Python library ecosystem from Go, without being forced to create an IPC bridge and building up the requisite release-management and deploy-time goop.

Plus, I'm just not a Python developer; in the case where the only library that exists to do something is written in Python, I'd much rather write Go that calls that Python library than Python that calls that Python library.

Re: Grumpy: Go running Python

#236

Post by bjt »

TillE wrote: post #66
Animats wrote: post #48

- Amusingly, it runs Python 2.7, even though this project started long after Python 3.x came out. - It's a hard-code compiler, not an interpreter written in Go. That implies some restrictions, but the documentation doesn't say much about what they are. PyPy jumps through hoops to make all of Python's self modification at run-time features work, complicating PyPy enormously. Nobody uses that stuff in production code,…

> Nobody uses that stuff in production code Nobody uses the features of Python which make it a dynamic language? Google must write some really weird Python if their compiler is that strict.

Anybody running Django uses this. It uses the pattern of specifying plugins as class paths in strings in the config, which are then looped over and instantiated at runtime.

Frameworks do lots of such dynamic tricks in order to provide nice DSLs for building apps.

Re: Grumpy: Go running Python

#237

Post by petters »

Twirrim wrote: post #5

This seems like an odd engineering choice. Presumably the effort to create a python->go translator would be non-trivial. Why not just start rewriting components into Go, and migrating them out of Python, leaving python as essentially the presentation layer at most?

The thing is, you can not use Python in production at Google. YouTube was acquired a long time ago and a rewrite of the inherited Pyton code base has not happened for very good reasons.

Re: Grumpy: Go running Python

#238

Post by kstrauser »

jerf wrote: post #118
trotterdylan wrote: post #88

Earlier quoted context omitted.

Thanks for trying it out! Yeah, Grumpy does not currently support old-style classes. Since all of our code internally requires new-style classes, this was not a high priority feature. It is something that we'll get to.

I did not mean that as a complaint against your very young codebase, I meant that as a defense against Python people complaining about my code. :)

I don't think they'd do that. All Python 2 code I've seen uses `class Foo(object)`, at least since 2.2 came out.

Re: Grumpy: Go running Python

#239

Post by bjt »

azinman2 wrote: post #109
TillE wrote: post #66

Earlier quoted context omitted.

> Nobody uses that stuff in production code Nobody uses the features of Python which make it a dynamic language? Google must write some really weird Python if their compiler is that strict.

Their python is weird if it doesn't use eval?

or getattr/setattr, or dynamically building classes with type(), or probably more features I can't remember now.

Re: Grumpy: Go running Python

#240

Post by beagle3 »

brianwawok wrote: post #163
Walkman wrote: post #142

Earlier quoted context omitted.

Is there not a single namedtuple in the entire Google codebase? That's strange :o

Are namedtuples that popular? They always felt awkward to me. If some temp variable with multiple values inside a loop, I either use normal tuple or a dict. If passing data around a dict or a real class. I never got the huge win from namedtuple?

namedtuples are tuples, meaning they are stored efficiently, and are constant (thus can also be used as dictionary keys). Unlike regular tuples, they can be accessed like a class/dictionary for readability, but requiring much less allocations (compared to dict/class), so much faster. Also, as they are tuples, you have well defined methods (printing, comparison, hash value, ) you'd have to implement yourself for dict/class.

If you like writing in functional style, namedtuples are much more natural than dict or classes, and more efficient to boot.

Post reply on HN