Grumpy: Go running Python • Hacker News Board

Re: Grumpy: Go running Python

#221

Post by yakubin » Wed, Jan 04, 2017, 8:36 PM UTC

jamestimmins wrote: post #159

Does anyone know if there is an easy way to get involved in open source projects like this? The readme mentions adding PRs, but as someone that doesn't have much experience working with open source projects like this, I don't even know where to begin. It sounds like an incredible learning opportunity though.

You can probably mail a maintainer or a developer for hints of a project you'd be interested in. Or better you can send a question to the project's mailing list. People there often are very helpful. I've done that. It worked. (I haven't tried contributing to any Google-managed projects though.)

Re: Grumpy: Go running Python

#222

Post by JPKab » Wed, Jan 04, 2017, 8:38 PM UTC

alrs wrote: post #168

thedjinn wrote: post #47

I'm curious to why they started this project when there already is low hanging fruit that can speed up Python (e.g. pypy). What makes this better other than to satisfy the inner go-fanboy?

Because they want to get off of Python completely, and forever. They're not looking for speed, they're looking to ditch the liability of supporting Python and its ecosystem.

Yeah, they didn't at all say that.

Re: Grumpy: Go running Python

#223

Post by pjmlp » Wed, Jan 04, 2017, 8:40 PM UTC

JPKab wrote: post #219

pjmlp wrote: post #181

Earlier quoted context omitted.

It might be, but that it is usually a consequence of not knowing any better or making use of existing libraries. Just like people learned 8-bit BASIC and went on to do business applications and games on it. I went Z80 ASM instead. Personally I would only use Python for shell scripting and advise for using Julia instead. Of course, others see it differently.

Julia? Look, I enjoy playing with Julia, but have you actually used it? Because most of the people I meet who turn up their nose at Python and say nice things about Julia haven't ever used Julia. It's got great potential but has some major gaps and warts, and is just plain dog shit slow for certain things compared to SciPy, which, you may not be aware, is basically C and Fortran code wrapped in Python API.

I do follow Julia development and I am aware that it isn't quite there, but at least their community does embrace JIT compilation, not like Python that PyPy is just yet another project, ignored by the reference implementation.

> SciPy, which, you may not be aware, is basically C and Fortran code wrapped in Python API.

Which for me personally means, that I would rather C and Fortran directly or better yet, a C++, .NET or Java binding to them.

Re: Grumpy: Go running Python

#224

Post by fiddlerwoaroof » Wed, Jan 04, 2017, 8:42 PM UTC

Animats wrote: post #48

- Amusingly, it runs Python 2.7, even though this project started long after Python 3.x came out. - It's a hard-code compiler, not an interpreter written in Go. That implies some restrictions, but the documentation doesn't say much about what they are. PyPy jumps through hoops to make all of Python's self modification at run-time features work, complicating PyPy enormously. Nobody uses that stuff in production code,…

> - It's a hard-code compiler, not an interpreter written in Go. That implies some restrictions, but the documentation doesn't say much about what they are. PyPy jumps through hoops to make all of Python's self modification at run-time features work, complicating PyPy enormously. Nobody uses that stuff in production code, and Google apparently dumped it.

Does it really imply many restrictions? Common Lisp, for example, is probably more dynamic than Python and it's been a compiled language for ~20 years.

Re: Grumpy: Go running Python

#225

Post by zardeh » Wed, Jan 04, 2017, 8:44 PM UTC

nilved wrote: post #215

dragonwriter wrote: post #72

Earlier quoted context omitted.

Creating a new Python runtime in Go is obviously non-trivial, but so is rewriting the heart of YouTube in Go. Heck, YouTube may be a bigger codebase than Python for all I know. And there's a lot more side benefit in a better multithreading Python runtime for Google for other Python code Google has (or hosts), whereas the benefits of a YouTube rewrite are more narrowly limited to YouTube.

Just because the site is popular doesn't mean the codebase is big. It's a simple site. I could probably write YouTube in Go in an afternoon and I don't even know Go.

You could perhaps write a feature-bare clone. Here's some basic features:

    - search (the various search pages)
    - trending
    - channels
    - subscription
    - video uploads
    - history
    - comments
    - likes
    - upload
    - video editing (most of this is in browser, but still)
    - livestream
    - video analytics
    - payment and ad management
    - video comment review and moderation
    - translation
    - captioning
    - video replication, multiscaling, caching, and resolution management based on network speed
    - video recommendation
    - cards, overlays, annotations, etc.
    - music and sound effect search
    - antispam
    - dmca and copyright tooling

And that was 5 minutes of poking around.

Re: Grumpy: Go running Python

#226

Post by solidsnack9000 » Wed, Jan 04, 2017, 8:44 PM UTC

weberc2 wrote: post #183

bb88 wrote: post #123

Earlier quoted context omitted.

Jython is a python interpreter written in Java. Grumpy is a python transpiler that converts python to navtive go object code. Edited to add: The difference is that Jython doesn't covert python to JVM bytecode.

What's the advantage of writing an interpreter? Go already has an excellent runtime (scheduler, GC, etc)--why should this project reimplement it?

The interpreter could still use that stuff.

One advantage of an interpreter in general is that one important use case for Python is interactive scripting, as data scientists do.

Re: Grumpy: Go running Python

#227

Post by teabee89 » Wed, Jan 04, 2017, 8:46 PM UTC

From looking in the code, this is my favorite part so far: https://github.com/google/grumpy/blob/75a35b4b30afb049b9cfff... making python threads as lightweight as goroutines must be a breeze!

Re: Grumpy: Go running Python

#228

Post by takeda » Wed, Jan 04, 2017, 8:48 PM UTC

vram22 wrote: post #111

moosingin3space wrote: post #92

Earlier quoted context omitted.

It's because Go uses a different stack structure, called "segmented stacks", in order to enable cheap goroutines. Basically, Go stacks start tiny (8 KiB, as opposed to much larger C stacks), then it grows them in small segments. Additionally, Go code runs inside an event loop, which enables excellent I/O performance without kernel context-switches, and ordinary C function calls conflict with this event loop.

>Additionally, Go code runs inside an event loop, which enables excellent I/O performance without kernel context-switches Interesting, didn't know this (that Go code runs in an event loop). Is the reason something to do with goroutines and channels? something like, a routine gets info that data is available for it to read (on a channel, sent by another goroutine), via an event it receives? Also, can you explain this…

Since we also talking about Python. If you ever used AsyncIO you will see that programming in it is a bit different than you usually write code without it.

Before you can call any coroutine you first need to start an event loop and schedule something in it. This essentially enables the language to schedule another async function each time you use await.

Since Go by default always is async, before your main function is called, it sets up the even loop and then calls your main, which technically is also a coroutine. Your code appears to be sequential, but it is not executed that way.

Re: Grumpy: Go running Python

#229

Post by takeda » Wed, Jan 04, 2017, 8:51 PM UTC

oblio wrote: post #6

It seems interesting but my concern is that it's just another Unladen Swallow.

To me it looks like Google essentially is moving everything to Go, and this project is to help with it.

They port their python libraries so they can reference them from their Go code, and then module by module will rewrite it in pure Go.

Re: Grumpy: Go running Python

#230

Post by petters » Wed, Jan 04, 2017, 8:52 PM UTC

pjmlp wrote: post #181

jszymborski wrote: post #139

Earlier quoted context omitted.

I think that that's rather dismissive of a language that runs huge web, scientific, and general purpose applications daily.

It might be, but that it is usually a consequence of not knowing any better or making use of existing libraries. Just like people learned 8-bit BASIC and went on to do business applications and games on it. I went Z80 ASM instead. Personally I would only use Python for shell scripting and advise for using Julia instead. Of course, others see it differently.

This made be wary of Julia: http://danluu.com/julialang/