If you want a real surprise, try a rewrite in Elixir.
Or many other languages.
Experience report on a large Python-to-Go translation
71–80 of 99 posts
Re: Experience report on a large Python-to-Go translation
#72Earlier quoted context omitted.
"Interesting. I'd have expected more than a 50% code expansion going to Go, maybe even 3x or 5x." This has been my extensive experience as well. I wouldn't be able to use Go if it was that much more verbose than Python. It certainly isn't as succinct as Python by any means, but it's not the night-and-day nightmare a lot of HN posters seem to think it is... provided you actually learn the language. In fact, one of the…
You wouldn't take the huge pile of features in an all or nothing. You get to pick and choose. I think the authors wish list is similar to most people who are experienced with more expressive languages. I don't see calls for Python f-strings or async syntax to end up in Go. But something like a list-comprehension syntax I think would be really popular (based on what I saw when it was introduced to Python, originally '…
"Interesting how Python is now considered a huge pile of features"
I've been tracking Python since 1.5.2 was common and 2.0 was just coming out. For me, it's not been a problem because the new features were added incrementally.
But I've seen new programmers try to come at it in 2019 and 2020, swallowing what I got spread out in ~15 major releases over ~20 years all in one big chunk, and learning even the core language now is definitely "a pile of features", to say nothing of the various library ecosystems. I don't even mean this necessarily as a criticism per se, just a description. It's not an easy-to-learn language anymore, and I don't recommend that people model it as such. It's a power tool for developers, not an easy learning language.
And, yes, the "you can pick and choose the features" is a non-starter in a team environment. At best, the team can pick and choose, and that takes a rather strong hand and alignment to achieve. It is a very common case that you'll just end up with what your teammates use.
Re: Experience report on a large Python-to-Go translation
#73There would probably be a much longer list of issues if ESR had converted to Rust instead, but the syntax for error returns is quite interesting. Rust and Go both opt not to have exceptions, instead they use error return values. The original Python code using exceptions was: sink = transform3(transform2(transform1(source))) Making that use error return values looks quite verbose in Go, but Rust has syntax specificall…
ESR tried Rust a while back, and decided to go with Go in general.
Re: Experience report on a large Python-to-Go translation
#74Earlier quoted context omitted.
I've been saying for a long time that every abstraction has a cost. Sometimes that cost is hard to quantify or externalized, but our inability to quantify it doesn't mean the cost doesn't exist. Abstractions have to at least pay for themselves many times over to be worth the extra cognitive burden and we need to get better at measuring these trade-offs. The success of languages like Go hint that there's more costs th…
The one-time cost of learning a good abstraction is strictly less than the ongoing cost of understanding and then continually reimplementing it by hand. The purpose of a high-level language is to make programs more concise and clear; a language that doesn't do this may somehow become popular but that shouldn't be mistaken for successful .
"The one-time cost of learning a good abstraction is strictly less than the ongoing cost of understanding and then continually reimplementing it by hand."
In theory, I agree with you. However, the main thrust of my post is that the practice is not working out as that theory predicts.
I haven't fully worked out all the bits and pieces, but I do know that just repeating that the theory must be correct because it must be correct even if it contradicts the evidence is not the correct way forward. Theory bows to evidence, not the other way around.
Re: Experience report on a large Python-to-Go translation
#75Adding lookbehinds to the regexp library is a terrible idea. > The regexp implementation provided by this package is guaranteed to run in time linear in the size of the input. Python's is exponential, because it inherits all the non-regular "regular" expression mess (such as lookbehind and backrefs) from perl. One would assume esr would have marinated in unix culture for long enough to be aware of this.
Some projects need performant regexes, and some honestly just don't. I agree that keeping the base regex library linear is admirable, but it's be nice if they offered a well-marked thing like regex.slow_and_perl_like in the stdlib
Re: Experience report on a large Python-to-Go translation
#76Interesting. I'd have expected more than a 50% code expansion going to Go, maybe even 3x or 5x. Similarly, he's using 40x speedup as a rule of thumb. I usually think of Python as 20x slower than C. Personally I'd be loathe to convert a working Python system to Go, but it sounds like he had good reasons. I do wonder a bit whether divide-and-conquer or a C extension might not have worked instead.
When you have a system that is at big scale, even a 2x speedup can relieve a huge amount of problems. I think that Go, which is very inexpressive for modern languages, only expanded the code size that much, should make people consider whether we should ever be using interpreted, dynamic typed languages. Perhaps various JITted/compiled and statically typed languages can offer the same productivity on medium to large s…
Nim is the closest language to Python. Translating to Python is a breeze and you get speeds comparable with pure C.
Re: Experience report on a large Python-to-Go translation
#77If it was too slow in Python and now moving to Go. Could there a time when there is a need to move to Rust/C/C++ for even faster performance? Go seems an odd choice based on performance consideration alone.
Someone once told me a tale of a student who had every variable defined as a single element array. When queried, she responded: "What if I need two of them?"
If you consider the could, there's always a way to make things arbitrarily more complex. Reality is that your target is simply "sufficient".
Also another provided reason was language consolidation, so ease of developer-project-flexibility was probably not the lowest of concerns
Re: Experience report on a large Python-to-Go translation
#78Interesting. I'd have expected more than a 50% code expansion going to Go, maybe even 3x or 5x. Similarly, he's using 40x speedup as a rule of thumb. I usually think of Python as 20x slower than C. Personally I'd be loathe to convert a working Python system to Go, but it sounds like he had good reasons. I do wonder a bit whether divide-and-conquer or a C extension might not have worked instead.
When you have a system that is at big scale, even a 2x speedup can relieve a huge amount of problems. I think that Go, which is very inexpressive for modern languages, only expanded the code size that much, should make people consider whether we should ever be using interpreted, dynamic typed languages. Perhaps various JITted/compiled and statically typed languages can offer the same productivity on medium to large s…
Having done a significant amount of programming in both Python-like and statically typed languages, I think the idea that they offer similar productivity (i.e., time to correct, maintainable implementation) is simply false.
And in a number of cases I've turned an existing program in a language like C or Java into a faster program in Python. One of the benefits of a fluent language is that you can explore more of the possibility space and look for better solutions. That's difficult and costly in static languages.
Re: Experience report on a large Python-to-Go translation
#79Interesting. I'd have expected more than a 50% code expansion going to Go, maybe even 3x or 5x. Similarly, he's using 40x speedup as a rule of thumb. I usually think of Python as 20x slower than C. Personally I'd be loathe to convert a working Python system to Go, but it sounds like he had good reasons. I do wonder a bit whether divide-and-conquer or a C extension might not have worked instead.
> it sounds like he had good reasons Meh. > Subversion-to-git conversion of the Gnu Compiler Collection history, at over 280K commits (over 1.6M individual change actions), was the straw that broke the camel’s back. Using PyPy with every optimization on semi-custom hardware tuned for this job still yielded test conversion times of over 9 hours, which is death on the recipe-debugging cycle. Just how often does one nee…
I was wondering that, too. My guess is that he's using this as a (large) set of test data to test his "recipes" and has to do that a lot. Having to run many nine-hour trials serially is painful.
Not sure I'd have bothered, but it's a defensible choice. And produced this interesting example of comparable implementations.
Re: Experience report on a large Python-to-Go translation
#80Interesting. I'd have expected more than a 50% code expansion going to Go, maybe even 3x or 5x. Similarly, he's using 40x speedup as a rule of thumb. I usually think of Python as 20x slower than C. Personally I'd be loathe to convert a working Python system to Go, but it sounds like he had good reasons. I do wonder a bit whether divide-and-conquer or a C extension might not have worked instead.
"Interesting. I'd have expected more than a 50% code expansion going to Go, maybe even 3x or 5x." This has been my extensive experience as well. I wouldn't be able to use Go if it was that much more verbose than Python. It certainly isn't as succinct as Python by any means, but it's not the night-and-day nightmare a lot of HN posters seem to think it is... provided you actually learn the language. In fact, one of the…
I have pretty good mastery of Python2, and in my hands, a few lines of Python would blow up into a page of Go. This really frustrated me and was one of the reasons I bounced off of it.
Other reasons: no fork(), poor make integration, weird source dir structure, and I once caught one of the go tools trying to write system files in /usr/lib or something (which fortunately failed on permissions). Can't recall the specifics of that last, and perhaps it was somehow misconfigured.
Python2 has a good feature set, and one of the downsides of Python3 is that it greatly enlarges what you need to know without much expanding what you can do. It's a real problem, and it would be a shame if it ended up where C++ is these days. ("Poor Joe--a C++ spec fell off a high shelf and killed him...")