Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Hacking Go's runtime with generics

dolthub.com

Post reply on HN
Print view View original

61–70 of 113 posts

Re: Hacking Go's runtime with generics

#61

Post by shakow »

silisili wrote: post #17
voz_ wrote: post #5

I do not understand the appeal of go. > The proposal argues that Golang developers are forced to work around language when writing hash-based data structures like Tries or concurrent hash maps. Developers who want an efficient hash-based data structure have only the builtin map to choose from. When this issue was originally created in 2017, Golang didn't have generics And then the author goes to tell us about how thi…

I've been writing software for 20 years. About half that in Go. Whenever I see someone say it's a bad language because X, I seriously wonder if the asker just writes terrible code, or if there's whole swaths of domains I've not encountered. Or maybe a little of both. I've never once been stumped by a lack of generics, for example, despite how many people claim it's unusable without them.

> I've never once been stumped by a lack of generics

Heve you never written a stack structure (since Go doesn't include one)? Never used a min/max function between numbers (go doesn't either)?

Re: Hacking Go's runtime with generics

#62

Post by czx4f4bd »

maxmcd wrote: post #50
naasking wrote: post #49

Earlier quoted context omitted.

Why can't a database be written in a garbage collected language?

I believe you usually want as much control of memory management as possible as it can be a great place for finding performance improvements and also help in delivering very consistent performance. Automatic memory management is a great aid in reduction of the complexity you have to deal with, but you presumably want to deal with that completely in the context of a database. Seems like this: https://www.cockroachlabs.…

You can allocate memory manually in Go and manage it yourself using Go's `unsafe` package, so there's an escape route once Go's GC became a limiting factor. Here's a blog post about it in the context of Dgraph: https://dgraph.io/blog/post/manual-memory-management-golang-...

Re: Hacking Go's runtime with generics

#63

Post by flagsrule »

naasking wrote: post #52
maxmcd wrote: post #50

Earlier quoted context omitted.

I believe you usually want as much control of memory management as possible as it can be a great place for finding performance improvements and also help in delivering very consistent performance. Automatic memory management is a great aid in reduction of the complexity you have to deal with, but you presumably want to deal with that completely in the context of a database. Seems like this: https://www.cockroachlabs.…

> I believe you usually want as much control of memory management as possible as it can be a great place for finding performance improvements and also help in delivering very consistent performance. Sure, but premature optimization is the root of all evil right? Get something functional, then make it performant by handling the problematic cases. GC doesn't prevent this progression, it's just a significant help to get…

And if the GC ever starts slowing you down, just run a profiler and eliminate the allocations. It's usually as simple as replacing a dependency or using sync.Pool in the hot path.

Re: Hacking Go's runtime with generics

#64

Post by notwokeno »

czx4f4bd wrote: post #62
maxmcd wrote: post #50

Earlier quoted context omitted.

I believe you usually want as much control of memory management as possible as it can be a great place for finding performance improvements and also help in delivering very consistent performance. Automatic memory management is a great aid in reduction of the complexity you have to deal with, but you presumably want to deal with that completely in the context of a database. Seems like this: https://www.cockroachlabs.…

You can allocate memory manually in Go and manage it yourself using Go's `unsafe` package, so there's an escape route once Go's GC became a limiting factor. Here's a blog post about it in the context of Dgraph: https://dgraph.io/blog/post/manual-memory-management-golang-...

The real issue with go is (or was, I haven't messed with it for a few years) its un-optimized goroutine scheduler that makes channels slow if you start using them a lot.

Re: Hacking Go's runtime with generics

#65

Post by ilyt »

silisili wrote: post #17
voz_ wrote: post #5

I do not understand the appeal of go. > The proposal argues that Golang developers are forced to work around language when writing hash-based data structures like Tries or concurrent hash maps. Developers who want an efficient hash-based data structure have only the builtin map to choose from. When this issue was originally created in 2017, Golang didn't have generics And then the author goes to tell us about how thi…

I've been writing software for 20 years. About half that in Go. Whenever I see someone say it's a bad language because X, I seriously wonder if the asker just writes terrible code, or if there's whole swaths of domains I've not encountered. Or maybe a little of both. I've never once been stumped by a lack of generics, for example, despite how many people claim it's unusable without them.

Generics were never required, just their lack was annoying and their presence makes some things just cleaner and easier

For example I deal in ops stuff so pretty common task is "here are desired and current set, generate list of add/delete operations to get from one state to another".

I wrote iteration of that function a bunch of times pre-generics, eventually being just copy-paste-change-types but now I can just write [1] one that compares a slice of types and returns the elements present in left or right and [2] another that can do similar difference on disparate types via conversion function.

Then there is a whole slew of map/reduce and math stuff that now can just have `Map` or `Max` function without worrying about types. Hell I can "just" write "parse all the slice elements in parallel, keeping order [3] once and never worry about it. That's also code that I wrote more than once before generics

* [1] https://github.com/XANi/goneric/blob/master/slice.go#L77

* [2] https://github.com/XANi/goneric/blob/master/slice.go#L106

* [3] https://github.com/XANi/goneric/blob/master/parallel.go#L15

Re: Hacking Go's runtime with generics

#66

Post by fsociety »

naasking wrote: post #52
maxmcd wrote: post #50

Earlier quoted context omitted.

I believe you usually want as much control of memory management as possible as it can be a great place for finding performance improvements and also help in delivering very consistent performance. Automatic memory management is a great aid in reduction of the complexity you have to deal with, but you presumably want to deal with that completely in the context of a database. Seems like this: https://www.cockroachlabs.…

> I believe you usually want as much control of memory management as possible as it can be a great place for finding performance improvements and also help in delivering very consistent performance. Sure, but premature optimization is the root of all evil right? Get something functional, then make it performant by handling the problematic cases. GC doesn't prevent this progression, it's just a significant help to get…

If you end up being wrong you will either have to fight the language or completely rewrite it in another.

It is not premature optimization but an intentional design choice, and a valid one to make.

Re: Hacking Go's runtime with generics

#67

Post by naasking »

fsociety wrote: post #66
naasking wrote: post #52

Earlier quoted context omitted.

> I believe you usually want as much control of memory management as possible as it can be a great place for finding performance improvements and also help in delivering very consistent performance. Sure, but premature optimization is the root of all evil right? Get something functional, then make it performant by handling the problematic cases. GC doesn't prevent this progression, it's just a significant help to get…

If you end up being wrong you will either have to fight the language or completely rewrite it in another. It is not premature optimization but an intentional design choice, and a valid one to make.

> If you end up being wrong you will either have to fight the language or completely rewrite it in another.

This is mostly a myth.

Re: Hacking Go's runtime with generics

#68

Post by skybrian »

Traubenfuchs wrote: post #7

So the reinvented wheel (golang) is now starting to miss / reinvent Java's .hashCode() or an alternative to it?

Sort of, except the way Java did it is awkward because they put a method on Object instead of making it an interface. It's not clear from the API what's really usable as a map key.

I'm not sure making all comparable objects in Go usable as map keys is a good idea either.

Re: Hacking Go's runtime with generics

#69

Post by konart »

shakow wrote: post #54
rob74 wrote: post #40

Earlier quoted context omitted.

True, but can you really work day by day in a language and still not understand its appeal? I mean, not like it, but at least understand why others might choose to use it? And, if you really can't bring yourself to see the positive sides of a language you have to use (and any mainstream language has to have positive sides, otherwise it wouldn't be mainstream), maybe it's really time to consider switching jobs...

> True, but can you really work day by day in a language and still not understand its appeal? Yes. I have been working with Go for close to a year now, and I still don't get why we use it, if not for the inertia of times past – and I still don't get why it was chosen in the first place. If we wanted a fast, down to the metal language, Rust would IMHO be better. If we preferred a higher-level language with a Gc, Java/…

>If we preferred a higher-level language with a Gc, Java/Kotlin/C# would be, IMHO, much better.

Writing code in Java\Kotlin requires quite some investment into the language and tooling (at least that's my Java impression).

With Go - you can start getting the job done from day one most of the time. Some parts of Go may be harder than the others, even challenging maybe, but not as hard as Java.

Lack of OOP (at least in Java's traditional scence) also helps quite a lot (but that depends on project of course)

Re: Hacking Go's runtime with generics

#70

Post by Ingaz »

dj_mc_merlin wrote: post #33

Honest question: isn't Go a bit of a weird choice for a database given the GC? Why not C/C++/Rust?

C/C++/Rust - will not solve anything magically. Anyway you will need to implement some kind of memory management for DB.

Go GC creates problems and they are non-trivial (as all memory management) but it's not so twisted as in JVM.

Examples:

- DGraph: migrated from RocksDB (C++) to Badger (pure go, developed themselves, based on jemalloc)

- CocroachDB - pure go for storage engine

- go 1.20 proposed feature: package `arena` for region-based memory management

Post reply on HN