Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Generics can make your Go code slower

planetscale.com

Post reply on HN
Print view View original

201–210 of 418 posts

Re: Generics can make your Go code slower

#201

Post by mcronce »

svnpenn wrote: post #183
jdmnd wrote: post #148

Earlier quoted context omitted.

It works for any type that implements the `std::error::Error` trait; which is something you can easily implement for your own types. If you want your errors to be integers for some reason, you can wrap that type in a zero-sized "newtype" wrapper, and implement `Error` for that. The Stack Overflow answer you linked seems to be claiming that it's simply easier to return strings, but I wouldn't say this is a restriction…

> easily implement for your own types have you ever actually done that? I have, its not easy. Please dont try to hand wave away negatives of the Rust type system.

I do it frequently. It is indeed easy

Re: Generics can make your Go code slower

#202

Post by morelisp »

linkdd wrote: post #125
morelisp wrote: post #11

Earlier quoted context omitted.

Ignorance of how your language works is a bad thing. Knowing where performance issues with certain techniques might arise is not premature optimization. Implement with an appropriate level of care, including performance concerns. Not every kind of poor performance appears as a clear spike in a call graph, and even fewer can be fixed without changing any external API.

> Ignorance of how your language works is a bad thing. And I never said anything remotely close to contradict this statement. > Knowing where performance issues with certain techniques might arise is not premature optimization. It is: - Python: should I use a for loop, a list comprehension or the map function? - C++: should I use a std::list, std::vector, ...? - Go: should I use interface{} or generics? The differenc…

You're giving fine advice for well-scoped tasks with minimal design space (well, sort of - using std::list ever is laughable - but if you had said unordered_map vs. map, sure, so I take the broad point). But, some of us have been around the block a few times though, and now need to make sure those spaces are delineated for others in a way that won't force them into a performance corner.

> until you have implemented your algorithm, you don't know what kind of operations (and how often) you will do with your container.. until you have implemented your algorithm, you don't know how often you will have to use generics / reflection, so you can't know what will be the true impact on your code.

I don't mean to brag, but I guess I'm a lot better at planning ahead than you. I don't usually have the whole program written in my head before I start, but I also can't remember any time I had to reach for a hammer as big as reflect and didn't expect to very early on, and most of the time I know what I intend to do to my data!

> This is why you "make it work" and "make it right" before you "make it fast"... This way you have a clear separation between your API and your implementation details.

This is not possible. APIs force performance constraints. Maybe wait until your API works before micro-optimizing it, but also maybe think about how many pointers you're going to have to chase and methods your users will need to implement in the first place because you probably don't get to "optimize" those later without breaking the API. You write about "the bottleneck", but there's not always a single bottleneck distinct from "the API". Sometimes there's a program that's slow because there's a singular part that takes 10 seconds and could take 1 second. But sometimes it's slow because every different bit of it is taking 2ns where it could take 1ns.

Consider the basic read-some-bytes API in Go vs. Python (translated into Go, so the difference is obvious):

    type GoReader interface { Read([]byte) (int, error) }

    type PyReader interface { Read(int) ([]byte, error) }
You're never going to make an API like PyReader anywhere near as fast as GoReader, no matter how much optimization you do!

Re: Generics can make your Go code slower

#203

Post by emtel »

bcrosby95 wrote: post #191
emtel wrote: post #182

Earlier quoted context omitted.

My argument against GC (and which applies similary to JIT-basd runtimes) is that the problems caused by GC pauses have non-local causes. If a piece of code ran slowly because of a GC pause, the cause of the pause is in some sense _the entire rest of the system_. You can't fix the problem with a localized change. Programs in un-managed languages can be slow too, and excessive use of malloc() is a frequent culprit. But…

I don't want to try to bring up an exception that disproves your rule, but what about something like BEAM, where it has per-process (process = lightweight thread) heaps and GC.

I don't know anything about BEAM, but I don't think single-threading of any form really addresses the underlying problem. If you go to allocate something, and the system decides a GC is necessary in order to satisfy your allocation, then the GC has to run before your allocation returns.

Re: Generics can make your Go code slower

#204

Post by tptacek »

stingraycharles wrote: post #66
baq wrote: post #58

Earlier quoted context omitted.

this has been argued ad nauseum a decade ago and it boils down to your definition of 'systems'. at google scale, a system is a mesh of networked programs, not a kernel or low-level bit-banging tool.

By that definition, Java is a systems language as well. I think Go makes a better trade-off than Java, but I struggle to come up with decent examples of projects one could write in Go and not in Java. Most of the “systems” problems that Java is unsuitable for, also apply to Go.

People do huge amounts of systems programming in Java, including in systems that are incredibly performance-sensitive.

Re: Generics can make your Go code slower

#205

Post by cb321 »

ptsneves wrote: post #159
cb321 wrote: post #124

Earlier quoted context omitted.

>until recently Actually, for a long time (almost 20 years, I think), the gcc subsystem gcj let you do AOT compilation of Java to ELF binaries. [1] I think they had to be dynamically linked, but only to a few shared objects (unless you pulled in a lot via native dependencies, but that's kind of "on you"). I don't recall any restrictions on how to use generated code, anymore than gcc-generated object code. So, I don't…

The interesting thing I always heard in the Java world is that AOT was actually a drawback as the virtual machine allowed for just in time optimizations according to hotspots. Actually if I remember correctly the word hotspot itself was even used as a tech trademark. I was always a bit skeptical but given i was never much into Java i just assumed my skepticism was out of ignorance. Now with what I know about profile…

Theory and practice can diverge and it's easy to over-conclude based on either with such complex systems. For example, I have seen gcc PGO make the very same training case used to measure the profile run more slowly. One might think that impossible naively, but maybe it sounds more plausible if I put it differently - "steering the many code generation heuristics with the profile failed in practice in that case". As with almost everything in computer systems, "it all depends...."

Re: Generics can make your Go code slower

#206

Post by TheDong »

svnpenn wrote: post #183
jdmnd wrote: post #148

Earlier quoted context omitted.

It works for any type that implements the `std::error::Error` trait; which is something you can easily implement for your own types. If you want your errors to be integers for some reason, you can wrap that type in a zero-sized "newtype" wrapper, and implement `Error` for that. The Stack Overflow answer you linked seems to be claiming that it's simply easier to return strings, but I wouldn't say this is a restriction…

> easily implement for your own types have you ever actually done that? I have, its not easy. Please dont try to hand wave away negatives of the Rust type system.

> have you ever actually done that? I have, its not easy.

Yes. I do it frequently. "#[derive(Error, Debug)]": https://github.com/dtolnay/thiserror#example

Much easier than implementing the error interface in go.

Rust is powerful enough to allow macros to remove annoying boiler-plate, and so most people using rust will grab one of the error-handling crates that are de-facto standard and remove the minor pain you're talking about.

In go, it's not really possible to do this because the language doesn't provide such macros (i.e. the old third-party github.com/pkg/errors wanted you to implement 'Cause', but couldn't provide sugar like 'this-error' does for it because go is simply less powerful).

I've found implementing errors in go to be much more error-prone and painful than in rust, and that's not to mention every function returning untyped errors, meaning I have no clue what callers should check for and handle new errors I add.

Re: Generics can make your Go code slower

#207

Post by monocasa »

svnpenn wrote: post #101
averagedev wrote: post #72

Earlier quoted context omitted.

I've found Go to be much simpler than Rust, especially syntax wise. However, in Rust you can use the ? operator which propagates errors. In Go you have to check err != nil.

> Rust you can use the ? operator That doesnt work with all types: https://stackoverflow.com/a/65085003

Only automatically printing something when returning it from main doesn't work with all types with the ? operator. And frankly 'handling errors by auto print and exit' is a bit of a code smell anyway, it's not much better than just .unwrap() on everything in main.

Re: Generics can make your Go code slower

#208

Post by steveklabnik »

dahfizz wrote: post #71
svnpenn wrote: post #62

Earlier quoted context omitted.

Typical gatekeeping. I like Go, because it lets me get stuff done. You could say the same about JavaScript, but I think Go is better because of the type system. C, C++ and Rust are faster in many cases, but man are they awful to work with. C and C++ dont really have package management to speak of, its basically "figure it out yourself". I tried Rust a couple of times, but the Result/Option paradigm basically forces y…

> C and C++ dont really have package management to speak of I hear this complaint often, but I consider it a feature of C. You end up with much less third party dependencies, and the libraries you do end up using have been battle tested for decades. I much prefer that to having to install hundreds of packages just to check if a number is even, like in JS.

> You end up with much less third party dependencies,

https://wiki.alopex.li/LetsBeRealAboutDependencies

Re: Generics can make your Go code slower

#209

Post by jjtheblunt »

nvarsj wrote: post #46

I'd argue that golang is inherently not a systems language, with its mandatory GC managed memory. I think it's a poor choice for anything performance or memory sensitive, especially a database. I know people would disagree (hence all the DBs written in golang these days, and Java before it), but I think C/C++/Rust/D are all superior for that kind of application. All of which is to say, I don't think it matters. Use t…

> with its mandatory GC managed memory

is that factual, in the general case?

it seems there exists a category of Go programs for which escape analysis entirely obviates heap allocations, in which case if there is any garbage collection it originates in the statically linked runtime.

Re: Generics can make your Go code slower

#210

Post by fauigerzigerk »

throwaway894345 wrote: post #179
fauigerzigerk wrote: post #153

Earlier quoted context omitted.

>I will assume by "infrastructure code" you mean things like kernels and network stacks. That, and things like database systems, libraries that are used in a lot of other software or language runtimes for higher level languages. >The actual heap footprint of the Linux kernel is pretty small And what would that footprint be if the kernel was written in Java or Go? What would the performance of all those device drivers…

> You can of course write memory efficient code in GC languages by manually managing a bunch of buffers. But I have seen and written quite a bit of that sort of code. It's horribly unsafe and horribly unproductive to write. Go uses buffers pretty idiomatically and they don't seem unsafe or unproductive. Maybe I'm not following your meaning? > If safety was ever good enough reason to use GC languages for systems/infra…

My argument is about the economics of software development more than about any of the large number of interesting technical details we could debate for a very long time.

There are higher level features that cause higher resource consumption. GC is clearly one such feature. No one denies that. So how do we decide where it makes more sense to use these features and where does it make less sense?

What I'm saying is that we should let ourselves be guided by the ratio development_time / running_time. The smaller this ratio, the less sense it makes to use such "resource hogging" features and the more sense it makes to use every opportunity for optimisation.

This is not only true for infrastructure/systems software. This is just one case where that ratio is very small. Another case would be application software that is used by a very large number of people, such as web browsers.

Post reply on HN