Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Go Optimization Guide

goperf.dev

Post reply on HN
Print view View original

131–140 of 173 posts

Re: Go Optimization Guide

#131

Post by kbolino »

9rx wrote: post #129
kbolino wrote: post #87

Earlier quoted context omitted.

In particular, struct field alignment matches C (even without cgo) and so any change to the default would break a lot of code.

> struct field alignment matches C (even without cgo) The spec defines alignment for numeric types, but that's about it. There is nothing in the spec about struct layout. That is implementation dependent. If you are relying on a particular implementation, you are decidedly in unsafe territory. > so any change to the default would break a lot of code. The compiler can disable optimization on cgo calls automatically an…

That's good to know. I'm not making use of this assumption, but the purego package (came from ebitengine) does. It looks like they're aware of HostLayout [1] but I'm not sure how many other people have gotten the memo (HostLayout didn't exist before Go 1.23).

[1]: https://github.com/ebitengine/purego/issues/259

Re: Go Optimization Guide

#132

Post by ahmedtd »

nopurpose wrote: post #101
ncruces wrote: post #73

Earlier quoted context omitted.

That's just not true. Pool contents are GCed after two cycles if unused.

What do you mean? Pool content can't be GCed , because there are references to it: pool itself. What people do is what this article suggested, pool.Get/pool.Put, which makes it only grow in size even if load profile changes. App literally accumulated now unwanted garbage in pool and no app I have seen made and attempt to GC it.

From the sync.Pool documentation:

> If the Pool holds the only reference when this happens, the item might be deallocated.

Conceptually, the pool is holding a weak pointer to the items inside it. The GC is free to clean them up if it wants to, when it gets triggered.

Re: Go Optimization Guide

#133

Post by nasretdinov »

donatj wrote: post #81

Unpopular opinion maybe, but sync.Pool is so sharp, dangerous and leaky that I'd avoid using it unless it's your absolute last option. And even then, maybe consider a second server first.

I think in general people understand that sync.Pool introduces essentially an equivalent of unitialised memory (since objects aren't required to be cleaned up before returning them to the pool), and mostly use it for something like []byte, slicing it like buf[0:0] to avoid accidentally reading someone else's memory.

But the instrument itself is really sharp and is indeed kind of last resort

Re: Go Optimization Guide

#134

Post by arp242 »

9rx wrote: post #129
kbolino wrote: post #87

Earlier quoted context omitted.

In particular, struct field alignment matches C (even without cgo) and so any change to the default would break a lot of code.

> struct field alignment matches C (even without cgo) The spec defines alignment for numeric types, but that's about it. There is nothing in the spec about struct layout. That is implementation dependent. If you are relying on a particular implementation, you are decidedly in unsafe territory. > so any change to the default would break a lot of code. The compiler can disable optimization on cgo calls automatically an…

> There is nothing in the spec about struct layout

De-facto a lot of programs rely on it, so whatever the spec says is irrelevant.

Not just for cgo by the way, but also things like binary.Read()/Write().

Re: Go Optimization Guide

#135

Post by sophacles »

tgv wrote: post #79
gwd wrote: post #66

Earlier quoted context omitted.

> Not in the slightest. You know, it's this kind of comment on USENET forums which prompted the creation of StackOverflow. It's not curious and adds nothing to the discussion. I like Go and use it extensively; and I like having the option to fall back to the `any` type. But it's simply a fact that using the `any` type means that certain properties of the program can't be checked at compile time, in the same way that…

Sorry, but comparing Python's total absence of typing to extracting a value from any is quite weird. > certain properties of the program can't be checked at compile time Neither can you check if a number is positive or negative, or if a string is empty or not at compile time, but that doesn't make Go similar to COBOL or Forth. `var v any` declares v to be of the type any, not of any arbitrary type, which is what Pyth…

Python doesn't have "total absense of typing". It doesn't have static typing, so compile time checks are not possible (well historically, there's some psuedo static typing things these days). The fact that you can call `+` on some objects but not others is literally the result of the objects being different types.

A truly typeless language (or maybe more accurately single type for everything language) is ASM, particularly for older CPU designs. You get a word - the bitfield in a register, and can do any operation on it. Is that 64 bits loaded from an array of characters and the programmer intended it to be a string? Cool you can bitwise and it with some other register. Was it a u64, a pointer, a pair of u32s? Same thing - the semantics don't change.

Re: Go Optimization Guide

#137

Post by theThree »

jensneuse wrote: post #40
makeworld wrote: post #13

Earlier quoted context omitted.

Why would Pool increase memory usage?

Let's say you have constantly 1k requests per second and for each request, you need one buffer, each 1 MiB. That means you have 1 GiB in the pool. Without a pool, there's a high likelihood that you're using less. Why? Because in reality, most requests need a 1 MiB buffer but SOME require a 5 MiB buffer. As such, your pool grows over time as you don't have control over the distribution of the size of the pool items. S…

>That means you have 1 GiB in the pool.

This only happen when every request last 1 second.

Re: Go Optimization Guide

#138

Post by sophacles »

pyrale wrote: post #95
tgv wrote: post #50

Earlier quoted context omitted.

You never programmed in Go, I assume? Then you have to understand that the type of `pool.Get()` is `any`, the wildcard type in Go. It is a type, and if you want the underlying value, you have to get it out by asserting the correct type. This cannot be solved with generics. There's no way in Java, Rust or C++ to express this either, unless it is a pool for a single type, in which case Go generics indeed could handle t…

> There's no way in Java, Rust or C++ to express this either You make it look like it's a good thing to be able to express it. There's no way in Java, Rust or C++ to express this, praised be the language designers. As for expressing a pool value that may be multiple things without a horrible any type and an horrible cast, you could make an union type in Rust, or an interface in Java implemented by multiple concrete o…

Rust has an Any type. It's rarely useful, but there are occasionally situations where a heterogeneous collection is the right thing to do. Casting the any type back to actual type is fairly nice though, as the operation returns an Option and you're forced to deal with the case where your cast is wrong.

Re: Go Optimization Guide

#139

Post by 9rx »

arp242 wrote: post #134
9rx wrote: post #129

Earlier quoted context omitted.

> struct field alignment matches C (even without cgo) The spec defines alignment for numeric types, but that's about it. There is nothing in the spec about struct layout. That is implementation dependent. If you are relying on a particular implementation, you are decidedly in unsafe territory. > so any change to the default would break a lot of code. The compiler can disable optimization on cgo calls automatically an…

> There is nothing in the spec about struct layout De-facto a lot of programs rely on it, so whatever the spec says is irrelevant. Not just for cgo by the way, but also things like binary.Read()/Write().

> but also things like binary.Read()/Write().

binary.Read/Write already uses reflect as far as I can tell, so it wouldn't matter in that case. There is an optimization for numeric values in there, which may be what you are thinking of? But they are specified in the spec and that's not what we're talking about anyway (and if an optimization really had to go, for whatever reason, it wouldn't be the end of the world).

Did you mean something else?

Re: Go Optimization Guide

#140

Post by tgv »

pyrale wrote: post #95
tgv wrote: post #50

Earlier quoted context omitted.

You never programmed in Go, I assume? Then you have to understand that the type of `pool.Get()` is `any`, the wildcard type in Go. It is a type, and if you want the underlying value, you have to get it out by asserting the correct type. This cannot be solved with generics. There's no way in Java, Rust or C++ to express this either, unless it is a pool for a single type, in which case Go generics indeed could handle t…

> There's no way in Java, Rust or C++ to express this either You make it look like it's a good thing to be able to express it. There's no way in Java, Rust or C++ to express this, praised be the language designers. As for expressing a pool value that may be multiple things without a horrible any type and an horrible cast, you could make an union type in Rust, or an interface in Java implemented by multiple concrete o…

> You make it look like it's a good thing to be able to express it.

No, just that this pre-generics Go, and backwards compatibility is taken seriously.

Post reply on HN