Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Experience report on a large Python-to-Go translation

gitlab.com

Post reply on HN
Print view View original

91–99 of 99 posts

Re: Experience report on a large Python-to-Go translation

#91

Post by nimmer »

downerending wrote: post #82
nimmer wrote: post #76

Earlier quoted context omitted.

Nim is the closest language to Python. Translating to Python is a breeze and you get speeds comparable with pure C.

You seem like a Nimmer. What would you describe as the biggest downsides of Nim wrt Python? (aside from the fact that it's currently a niche language.)

The only two that I can think of are due to static typing:

You cannot enforce types only where you want, like mypy. Yet, type inference and automated conversions help quite a bit.

There's an unofficial REPL in the compiler but you cannot do dir() or tab-complete methods/procs. You have to rely on documentation, nimsuggests or IDEs instead.

Re: Experience report on a large Python-to-Go translation

#92

Post by pk2200 »

DougBTX wrote: post #46

There would probably be a much longer list of issues if ESR had converted to Rust instead, but the syntax for error returns is quite interesting. Rust and Go both opt not to have exceptions, instead they use error return values. The original Python code using exceptions was: sink = transform3(transform2(transform1(source))) Making that use error return values looks quite verbose in Go, but Rust has syntax specificall…

He evaluated Rust and Go a few years ago and strongly preferred Go.

https://blog.ntpsec.org/2017/01/18/rust-vs-go.html

Re: Experience report on a large Python-to-Go translation

#93

Post by pk2200 »

AnimalMuppet wrote: post #23
mistrial9 wrote: post #13

significant mastery ahead! This is a success story and a teaching document. .. have to point to this : "Now that I’ve seen Go strings… holy hell, Python 3 unicode strings sure look like a nasty botch in retrospect. " (!)

I wish he had been a bit more explicit there, TBH. What's better about Go strings?

He wrote about this more detail in an earlier blog post (about halfway down the page).

http://esr.ibiblio.org/?p=8161

Re: Experience report on a large Python-to-Go translation

#94

Post by erik_seaberg »

marcus_holmes wrote: post #40
erik_seaberg wrote: post #35

Earlier quoted context omitted.

The one-time cost of learning a good abstraction is strictly less than the ongoing cost of understanding and then continually reimplementing it by hand. The purpose of a high-level language is to make programs more concise and clear; a language that doesn't do this may somehow become popular but that shouldn't be mistaken for successful .

> The one-time cost of learning a good abstraction is strictly less than the ongoing cost of understanding and then continually reimplementing it by hand. There is a danger of a "no true Scotsman" fallacy here - it's easy to define a "good" abstraction as one that is worth the one-time cost of learning it. So, given that there are both "good" and "bad" abstractions in every language, is the net gain from learning the…

I look on it as falling out of the https://en.wikipedia.org/wiki/Rule_of_least_power. When I see reuse of an abstraction I know not only what it does but what it doesn't do. If I skip that and start writing custom code, well, that code could do anything at all so everyone has to continually reread it carefully. I find poring over tedious boilerplate to be a waste of precious lifetime compared to learning better building blocks.

Re: Experience report on a large Python-to-Go translation

#95

Post by 0xjnml »

rauhl wrote: post #56
Izkata wrote: post #31

Earlier quoted context omitted.

Skimming over a post from a quick search [0], it looks like Go strictly uses byte strings and manipulates them with various keywords and functions. No encoding/decoding between byte strings (and picking an encoding) and unicode strings like in python. [0] https://blog.golang.org/strings

I think that’s not really the right way to put it. Rather, go offers UTF-8 strings and byte slices, with a simple typecast in either way, with various keywords and functions to DTRT to each. One still must worry about encoding & decoding if one doesn’t want UTF-8, and one must worry about invalid UTF-8 in a byte slice when casting, but in general Go does what one would expect with a minimum of fuss.

Go strings are just a sequence of bytes in no particular encoding, i.e they can contain arbitrary data. Converting strings to byte slices and vice versa works always without ever changing a single bit.

Re: Experience report on a large Python-to-Go translation

#96

Post by adiabatty »

jerf wrote: post #72
stubish wrote: post #30

Earlier quoted context omitted.

You wouldn't take the huge pile of features in an all or nothing. You get to pick and choose. I think the authors wish list is similar to most people who are experienced with more expressive languages. I don't see calls for Python f-strings or async syntax to end up in Go. But something like a list-comprehension syntax I think would be really popular (based on what I saw when it was introduced to Python, originally '…

"Interesting how Python is now considered a huge pile of features" I've been tracking Python since 1.5.2 was common and 2.0 was just coming out. For me, it's not been a problem because the new features were added incrementally. But I've seen new programmers try to come at it in 2019 and 2020, swallowing what I got spread out in ~15 major releases over ~20 years all in one big chunk, and learning even the core languag…

> And, yes, the "you can pick and choose the features" is a non-starter in a team environment. At best, the team can pick and choose, and that takes a rather strong hand and alignment to achieve. It is a very common case that you'll just end up with what your teammates use.

I wonder if this wouldn't be much of a problem by, say, 2025. Already I'm used to having multiple linters run on save that fix up spacing and indentation like Prettier does. I'd expect people to make tools that'll complain if you're about to check in code that's too complicated for your team (uses lots of parentheses in a line without temporary variables, uses overly-complicated reduce() calls, etc.).

Prettier: https://prettier.io

Re: Experience report on a large Python-to-Go translation

#97

Post by ptx »

outworlder wrote: post #7

> The problem directed the choice of Go, not the other way around. I seriously considered OCaml or a compiled Lisp as alternatives. I concluded that in either case the semantic gap between Python and the target language was so large that translation would be impractical. Only Go offered me any practical hope. I wish they expanded more on this point. Do they mean that rewriting in, say, Lisp would be longer because it…

Yeah, that point doesn't seem to make a lot of sense. When people talk about Python and Go being similar, or filling a similar niche, I gather that this refers mostly to the short compilation time, rich standard library and easy of use. They are quite different in other ways.

Go might also be more similar in syntax than to Python than Lisp is, but the article talks about the "semantic gap" (not syntactic gap) which seems much narrower in Lisp vs. Python than in Go vs. Python.

> I did examine two automated tools for Python to Go translation, but rejected them because I judged the generated Go code would have been a maintainability disaster.

Perhaps indicating that the languages are not all that similar semantically.

Re: Experience report on a large Python-to-Go translation

#98

Post by ptx »

downerending wrote: post #81
jerf wrote: post #72

Earlier quoted context omitted.

"Interesting how Python is now considered a huge pile of features" I've been tracking Python since 1.5.2 was common and 2.0 was just coming out. For me, it's not been a problem because the new features were added incrementally. But I've seen new programmers try to come at it in 2019 and 2020, swallowing what I got spread out in ~15 major releases over ~20 years all in one big chunk, and learning even the core languag…

That's a good point. One of the original appeals of Python is that it started as a pedagogical language, and it shows. Even Python2 today still has most of that feel. Python3 on the other hand, just isn't. You're pretty much required to constantly deal with Unicode and its related issues, and this is a burden for beginners. In Python2, you can kick that can down the road almost indefinitely.

That hasn't been my experience. In Python 2 your program would appear to work fine initially, but the first time your data happens to contain a non-ASCII character you would get a mysterious UnicodeDecodeError somewhere far away from where the actual problem is.

In Python 3 it just works again, like it did in Python 1.5, before the whole str/unicode implicit-conversion mess was introduced in Python 2.

Re: Experience report on a large Python-to-Go translation

#99

Post by ptx »

pdonis wrote: post #28
ageofwant wrote: post #10

I do not get why people do these total rewrites, especially for working Python systems. Why throw out the baby with the bathwater ? Python is fundamentally a composing toolkit. Rewrite the slow bits in C++/Rust/Go and wrap it. That's how all major Python components like Numpy, Scipy, Tensorflow, PyTorch etc. does it. And that's a major reason why Python dominates today. Align with the core strengths of Python's philo…

> Rewrite the slow bits in C++/Rust/Go and wrap it. For reposurgeon, you can't. Not the author, but I have done some hacking on it. "The slow bits" are not things you can rewrite in a different language and wrap. They are much too integral to the code. > That's how all major Python components like Numpy, Scipy, Tensorflow, PyTorch etc. does it. And what all of these have in common is that the "slow bits" are not like…

So which parts and things are the slow bits of reposurgeon? ESR seemed to be saying[1] that the last time he tried profiling it was seven years ago.

[1] http://esr.ibiblio.org/?p=8161&cpage=1#comment-2065946

Post reply on HN