Grumpy: Go running Python • Hacker News Board

Re: Grumpy: Go running Python

#111

Post by vram22 » Wed, Jan 04, 2017, 6:40 PM UTC

moosingin3space wrote: post #92

vram22 wrote: post #51

Earlier quoted context omitted.

Does anyone know the technical reasons why C extensions are not (at least easily, apparently) supported by Go? Is it to do with Go's being a GC'd language? I would have thought that should not be a reason per se, since Python also has GC, but has plenty of extensions written in C. But I'm not a language internals expert. Also, further signs that GC may not be the reason, is that D also has GC, but can link to C libra…

It's because Go uses a different stack structure, called "segmented stacks", in order to enable cheap goroutines. Basically, Go stacks start tiny (8 KiB, as opposed to much larger C stacks), then it grows them in small segments. Additionally, Go code runs inside an event loop, which enables excellent I/O performance without kernel context-switches, and ordinary C function calls conflict with this event loop.

>Additionally, Go code runs inside an event loop, which enables excellent I/O performance without kernel context-switches

Interesting, didn't know this (that Go code runs in an event loop). Is the reason something to do with goroutines and channels? something like, a routine gets info that data is available for it to read (on a channel, sent by another goroutine), via an event it receives?

Also, can you explain this point:

"which enables excellent I/O performance without kernel context-switches" ?

Re: Grumpy: Go running Python

#112

Post by Animats » Wed, Jan 04, 2017, 6:41 PM UTC

jerf wrote: post #46

For those who are interested, I've used grumpy to compile the following Python code and placed it at https://play.golang.org/p/YP1SP7WsdR . (Note the playground can't run this, it just had convenient formatting support for Go; the generated source wasn't 100% gofmt compliant.) class Test(object): def __init__(self, value): self.value = value def method(self): print(self.value) class Test2(Test): pass t = Test("hello"…

That's fascinating. It's creating run-time data structures similar to CPython's for data, and manipulating them with very general code. There seems to be a type comparable to Python's internal CObject, and it's used for most (all?) data. It's not generating Go that looks anything like human-written Go. There's no sign of type inference, although it's hard to tell from such a simple example. It's a lot like a Python run time environment, where everything is a CObject. Still, once you can do that, you can start optimizing, such as inferring that something is an integer and using ordinary Go arithmetic types.

All that stuff with "switch" seems to be to handle Python exceptions in a language that doesn't have exceptions. Maybe later, analysis can tell that some function can't raise an exception, and translated calls for such functions can be simpler.

Re: Grumpy: Go running Python

#113

Post by dom0 » Wed, Jan 04, 2017, 6:42 PM UTC

ericfrederich wrote: post #99

dijit wrote: post #37

Earlier quoted context omitted.

cpython is the reference implementation, so it makes sense that it's; A) Not well optimised. B) Touting features before the spec/standard. EDIT: people really dislike that I said this, and I'm having trouble finding my original citation- it was on one of the many python books I own. Most likely "Learn Python The Hard Way" but I'll dig out the exact chapter where they compare pypy to cpython and mention that because c…

Please watch that first video, it's a good one. It explains how CPython essentially _is_ the spec because its internals leak into the spec when they have no business being there.

Pretty much the same is true about most other languages that have a single main implementation. This is even true, to some degree, for Java, which had competing implementations relatively early on.

Re: Grumpy: Go running Python

#114

Post by pjmlp » Wed, Jan 04, 2017, 6:43 PM UTC

sfifs wrote: post #27

The main reason I moved from coding in Python to Go as my main language many years back is because concurrency was such a pain in standard Python (the other was compile time error checking). It's interesting to see the same pain has now made caused the runtime itself to be implemented in Go. It's a pity C extensions (often used in scientific computing) are not supported but Go does have support via CGO, so maybe some…

I never saw Python as anything more than a scripting language to portably automate tasks across UNIX and Windows environment, even back in the Zope days.

My experience with Tcl teached me to stay away from languages that don't have either a JIT or AOT compiler on their reference implementation.

Re: Grumpy: Go running Python

#115

Post by waitingkuo » Wed, Jan 04, 2017, 6:44 PM UTC

Will Grumpy be a good fit for go's scientific packages such like gonum?

Re: Grumpy: Go running Python

#116

Post by trotterdylan » Wed, Jan 04, 2017, 6:44 PM UTC

naasking wrote: post #4

I wonder why the Grumpy Fibonacci is so much slower than CPython for 1 thread. Seems weird given Grumpy is compiled.

Although Grumpy is compiled, it is just as dynamic as Python, in that method dispatch involves dictionary lookups, etc.

The main reason why Grumpy's slower for most single threaded benchmarks is that most Python workloads involve creating and freeing a bunch of small Python objects. In Go, these objects are garbage that need to be GC'd in a very general way. In CPython, there are free lists, arenas and other optimizations for allocating small (especially immutable) objects. And cleaning up garbage in CPython involves pushing unreferenced objects back onto the free lists for later reuse.

Re: Grumpy: Go running Python

#117

Post by curtis » Wed, Jan 04, 2017, 6:44 PM UTC

One of my complaints about Go is that it seems to be designed with the (mistaken, in my opinion) notion that what we really need is just a better C.

One way to leverage C's widespread availability and high-performance while side-stepping some of its deficiencies was simply to use it as a target for a different language. The Cfront C++ compiler which generated C code is probably the most famous example, but I recall that there were many others.

Maybe Go, like C, will make a fruitful target for other language implementations.

Re: Grumpy: Go running Python

#118

Post by jerf » Wed, Jan 04, 2017, 6:44 PM UTC

trotterdylan wrote: post #88

jerf wrote: post #46

For those who are interested, I've used grumpy to compile the following Python code and placed it at https://play.golang.org/p/YP1SP7WsdR . (Note the playground can't run this, it just had convenient formatting support for Go; the generated source wasn't 100% gofmt compliant.) class Test(object): def __init__(self, value): self.value = value def method(self): print(self.value) class Test2(Test): pass t = Test("hello"…

Thanks for trying it out! Yeah, Grumpy does not currently support old-style classes. Since all of our code internally requires new-style classes, this was not a high priority feature. It is something that we'll get to.

I did not mean that as a complaint against your very young codebase, I meant that as a defense against Python people complaining about my code. :)

Re: Grumpy: Go running Python

#119

Post by ehsankia » Wed, Jan 04, 2017, 6:45 PM UTC

ProAm wrote: post #93

the_duke wrote: post #31

Earlier quoted context omitted.

Well, if Google is pushing it and actually using it internally... They also have the benefit of being able to push features into the Go core that they might need for this.

> Well, if Google is pushing it and actually using it internally... It will be silently abandoned in 18 months....

Not if it turns out to be usable and have a significant impact on the performance. You need to realize that at the scale Youtube works, even a small performance boost translates to huge savings in server cost.

So the worst case here is if they never improve this past their own needs (which is a pretty limited subset). But if it's successful for them, and it's opensource, I could very easily see other people who run heavy stuff on Python contributing to it and helping it grow.

That's the thing with opensource, even if Google doesn't actively work on it, others can (if it has actual value and is useful to people).

Re: Grumpy: Go running Python

#120

Post by quotemstr » Wed, Jan 04, 2017, 6:46 PM UTC

cdelsolar wrote: post #110

BuckRogers wrote: post #103

Earlier quoted context omitted.

Python3 was never the future of Python. It never got over the hump all new languages need to if attempting to reach relevance. It's likely code using lots of C-extensions will continue with CPython2 and new code will be written in Grumpy (pure Python2).

That is pretty ridiculous. Pretty much all major libraries are Python 3 compatible and everyone is writing Python 3 (or should be). (Yes, I'm still on Python 2 but moving soon).

I've been writing Python 3 for years.