I do however find that when you have a python project that is heavily dependent on third party libraries - these things get significantly larger and more problematic. That's not really a commentary on Go, inasmuch that it's a byproduct of the longevity of Python.
Experience report on a large Python-to-Go translation
61–70 of 99 posts
Re: Experience report on a large Python-to-Go translation
#62Earlier quoted context omitted.
I've been saying for a long time that every abstraction has a cost. Sometimes that cost is hard to quantify or externalized, but our inability to quantify it doesn't mean the cost doesn't exist. Abstractions have to at least pay for themselves many times over to be worth the extra cognitive burden and we need to get better at measuring these trade-offs. The success of languages like Go hint that there's more costs th…
The one-time cost of learning a good abstraction is strictly less than the ongoing cost of understanding and then continually reimplementing it by hand. The purpose of a high-level language is to make programs more concise and clear; a language that doesn't do this may somehow become popular but that shouldn't be mistaken for successful .
The religious adherence to "use all abstractions" is as dangerous as the religious adherence to "use no abstractions." My point is we have to get better at quantifying this stuff otherwise we'll forever be stuck in this cycle of arguing which abstractions are appropriate and when but never actually being able to quantify when we are right.
Re: Experience report on a large Python-to-Go translation
#63The missing 'keyword arguments' could have been replaced with a struct passed to a function, no? Unless I'm missing something from Python, in Go you could replace this type of function: func f(x int, y int, c string) with something like this: type funcOptions struct { x, y int c string } func f(o funcOptions) {} f(funcOptions{x:3, y:-1, c: "hello"}) So the readability hit would have been more 'minimal.
In reality, its a small benefit to readability. You gain parameter names, which is very nice, but you end up with huge function call lines.
describeTableOutput, err := dynamodbSvc.DescribeTableWithContext(ctx, &dynamodb.DescribeTableInput{
TableName: "users",
})
With the main sources of pollution here being:1) The forced inclusion of the package specifier on the type, rather than being able to directly reference the type via an import alias or something. You can alias the package name, but that's not a great general solution.
2) The forced inclusion of the type name at all. What, exactly, would be the type inference challenge if you were allowed to do something like this?
describeTableOutput, err := dynamodbSvc.DescribeTableWithContext(ctx, &{
TableName: "users",
})
I understand, if the parameter in the signature were an interface, this would not work. But its not; its a struct. It feels to me like this kind of inference should be allowed when a parameter is a struct, but maybe I'm missing some subtle corner case where it would not work.3) All the context stuff; both as a function parameter, and the words "WithContext". I hate this. To be clear; Context is awesome. Every function which has the possibility of making network calls should accept a context. And that's the problem; modern Go libraries liter it everywhere, adding "WithContext" mirrors of existing functions to maintain backcompat. Context really should be "contextual"; omnipresent, overrideable, and only getting in the way when its needed. 95% of functions which interface with a context accept the context and pass the context; they're middlemen. They shouldn't have to even care about the context. Something like:
func (d DynamoDB) DescribeTable(input *DescribeTableInput) (*DescribeTableOutput, error) {
ctx := getContext()
}
func main() {
setContext(context.Background())
describeTableOutput, err := dynamodbSvc.DescribeTable(&{
TableName: "users",
})
}
In other words, available via magic global functions. I'm sure there's some reason why this wouldn't work, or would have unintentional negative consequences, but I illustrate it only for the point that there are ways to accomplish this goal that are (probably) cleaner than making it a function parameter.Re: Experience report on a large Python-to-Go translation
#64Interesting. I'd have expected more than a 50% code expansion going to Go, maybe even 3x or 5x. Similarly, he's using 40x speedup as a rule of thumb. I usually think of Python as 20x slower than C. Personally I'd be loathe to convert a working Python system to Go, but it sounds like he had good reasons. I do wonder a bit whether divide-and-conquer or a C extension might not have worked instead.
When you have a system that is at big scale, even a 2x speedup can relieve a huge amount of problems. I think that Go, which is very inexpressive for modern languages, only expanded the code size that much, should make people consider whether we should ever be using interpreted, dynamic typed languages. Perhaps various JITted/compiled and statically typed languages can offer the same productivity on medium to large s…
Re: Experience report on a large Python-to-Go translation
#65Go is probably more verbose because it’s missing List comprehensions, for example. It needs map, filter, reduce to reduce line count. Swift, while probably not as performant as Go, makes writing in a more Pythonic style. [1,2,3,4,5,6,7,8,9].filter {$0 % 2 == 0}.map {$0 * 2}.reduce(0, +) ["550", "a", "6", "b", "42", "99", "100"].compactMap{Int($0)}.filter {$0 https://github.com/melling/SwiftCookBook/blob/master/functi…
Mind you I do like and prefer functional style, I've done a lot of the iteration style processing back in Java 1.5 and never gone back to that yet. Functional style expresses the 'what', whereas iterative style spends a lot of code expressing the 'how'.
I'd spread the operations out over multiple lines, one operation per line at least. But that's a personal style preference.
Re: Experience report on a large Python-to-Go translation
#66[edit: fixed links] This was discussed recently on the go-nuts mailing list: https://groups.google.com/d/msg/golang-nuts/u-L7PRa2Z-w/kfUS... There was also discussion around an earlier post he made about the work: https://groups.google.com/d/msg/golang-nuts/WstriKt2jTA/lsZy...
Re: Experience report on a large Python-to-Go translation
#67Earlier quoted context omitted.
"Interesting. I'd have expected more than a 50% code expansion going to Go, maybe even 3x or 5x." This has been my extensive experience as well. I wouldn't be able to use Go if it was that much more verbose than Python. It certainly isn't as succinct as Python by any means, but it's not the night-and-day nightmare a lot of HN posters seem to think it is... provided you actually learn the language. In fact, one of the…
You wouldn't take the huge pile of features in an all or nothing. You get to pick and choose. I think the authors wish list is similar to most people who are experienced with more expressive languages. I don't see calls for Python f-strings or async syntax to end up in Go. But something like a list-comprehension syntax I think would be really popular (based on what I saw when it was introduced to Python, originally '…
In theory; in practice though, you need to have all developers (and yourself) strictly aligned on what features to use. In my limited experience with Scala and working with Scala developers, there's as many styles and preferences as there are developers on a codebase.
Re: Experience report on a large Python-to-Go translation
#68The missing 'keyword arguments' could have been replaced with a struct passed to a function, no? Unless I'm missing something from Python, in Go you could replace this type of function: func f(x int, y int, c string) with something like this: type funcOptions struct { x, y int c string } func f(o funcOptions) {} f(funcOptions{x:3, y:-1, c: "hello"}) So the readability hit would have been more 'minimal.
I use intellij which can offer inline parameter name hints, I think that's a good middle ground but it doesn't make things more readable outside of that editor.
Re: Experience report on a large Python-to-Go translation
#69Pretty interesting. It is scary to make your "learn a new language" task to port 14,000 lines of code, but with that in mind, this all seems to have gone well. Some random thoughts: > I had to write my own set-of-int and set-of-string classes map[int]struct{}, map[string]struct{} ints[42] = struct{}{} // insert delete(ints, 42) // delete for i := range ints { ... } // iterate if _, ok := ints[42]; ok { ... } // exist…
Lol. Call sign of a junior developer...
Re: Experience report on a large Python-to-Go translation
#70There would probably be a much longer list of issues if ESR had converted to Rust instead, but the syntax for error returns is quite interesting. Rust and Go both opt not to have exceptions, instead they use error return values. The original Python code using exceptions was: sink = transform3(transform2(transform1(source))) Making that use error return values looks quite verbose in Go, but Rust has syntax specificall…
It's ugly either way but for clarity you can move the error checks into the transformation leaving the call point clean. i.e transform1 returns (result, error) and transform2 accepts (result, error) and short-circuits if err is not nil. It allows the expressive and succinct description of a transformation list but with a bunch of messiness hidden.