Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Go has added Valgrind support

go-review.googlesource.com

Post reply on HN
Print view View original

31–40 of 157 posts

Re: Go has added Valgrind support

#31

Post by johncolanduoni »

aleksi wrote: post #24
johncolanduoni wrote: post #16

Earlier quoted context omitted.

Golang has a feature that I love in general but that makes it very easy to keep unintended allocations around. If you have a struct with a simple int field, and you store that somewhere as an *int, the entire struct and anything it points to will be kept alive. This is super useful for short-lived pointers, and super dangerous for long-lived pointers. Most other widely used GCed languages don’t allow the use of arbit…

> If you have a struct with a simple int field, and you store that somewhere as an *int, the entire struct and anything it points to will be kept alive. While Go allows interior pointers, I don't think what you say is true. runtime.KeepAlive was added exactly to prevent GC from collecting the struct when only a field pointer is stored. Take a look at this blog post, for example: https://victoriametrics.com/blog/go-ru…

I don’t believe that’s the case based on the example in the blog post. The fd field in that struct was passed into the later function by value (i.e. as an int, not an *int), so there was no interior pointer in play at all.

Re: Go has added Valgrind support

#32

Post by [deleted] »

aleksi wrote: post #24
johncolanduoni wrote: post #16

Earlier quoted context omitted.

Golang has a feature that I love in general but that makes it very easy to keep unintended allocations around. If you have a struct with a simple int field, and you store that somewhere as an *int, the entire struct and anything it points to will be kept alive. This is super useful for short-lived pointers, and super dangerous for long-lived pointers. Most other widely used GCed languages don’t allow the use of arbit…

> If you have a struct with a simple int field, and you store that somewhere as an *int, the entire struct and anything it points to will be kept alive. While Go allows interior pointers, I don't think what you say is true. runtime.KeepAlive was added exactly to prevent GC from collecting the struct when only a field pointer is stored. Take a look at this blog post, for example: https://victoriametrics.com/blog/go-ru…

[deleted]

Re: Go has added Valgrind support

#33

Post by tsimionescu »

felixge wrote: post #5
defraudbah wrote: post #2

looks very promising, one of the biggest issue in golang for me is profiling and constant memory leaks/pressure. Not sure if there is an alternative of what people use now

I'd love to hear more! What kind of profiling issues are you running into? I'm assuming the inuse memory profiles are sometimes not good enough to track down leaks since they only show the allocation stack traces? Have you tried goref [1]?. What kind of memory pressure issues are you dealing with? [1] https://github.com/cloudwego/goref Disclaimer: I work on continuous profiling for Datadog and contribute to the profi…

I for one am still mystified how it's possible that a GC language can't expose the GC roots in a memory profile. I've lost so many hours of my life manually trying to figure out what might be keeping some objects live, information the GC figures out every single time it runs...

Re: Go has added Valgrind support

#34

Post by esbranson »

amelius wrote: post #3

It only works if every package tests with it. Otherwise the relevant warnings get swamped by a huge amount by irrelevant warnings. This is why running Valgrind on Python code does not work.

Simplification of overwhelming information sounds like a good use case for local LLMs. So I agree with other comments that toolchains are better positioned to include batteries like Valgrind.

Re: Go has added Valgrind support

#35

Post by defraudbah »

felixge wrote: post #5
defraudbah wrote: post #2

looks very promising, one of the biggest issue in golang for me is profiling and constant memory leaks/pressure. Not sure if there is an alternative of what people use now

I'd love to hear more! What kind of profiling issues are you running into? I'm assuming the inuse memory profiles are sometimes not good enough to track down leaks since they only show the allocation stack traces? Have you tried goref [1]?. What kind of memory pressure issues are you dealing with? [1] https://github.com/cloudwego/goref Disclaimer: I work on continuous profiling for Datadog and contribute to the profi…

no, haven't heard of goref yet but will give it a shot!

usually I go with pprof, like basic stuff and it helps. I would NOT say memory leak is the biggest or most common issue I see, however as time goes and services become more complicated what I often see in the metrics is how RAM gets eaten and does not get freed as time goes, so the app eats more and more memory as time goes and only restart helps.

It's hard to call it memory leak in "original meaning of memory leak" but the memory does not get cleaned up because the choices I made and I want to understand how to make it better.

Thanks for the tool!

Re: Go has added Valgrind support

#36

Post by SkySkimmer »

johncolanduoni wrote: post #19
pjmlp wrote: post #7

Earlier quoted context omitted.

Ideally, they would have learnt from other languages, and offered explicit control over what goes into the stack instead of relying into escape analysis alone. As it is, the only way to currently handle that is with " -gcflags -m=3" or using something like VSCode Go plugin, via "ui.codelenses" and "ui.diagnostic.annotations" configurations.

I sometimes dream of a GCed language with a non-escaping pointer type. However to make it really useful (i.e. let you put it inside other non-escaping structs) you need something on the scale of the Rust borrow checker, which means adding a lot of complexity.

https://oxcaml.org/documentation/stack-allocation/intro/ ?

Re: Go has added Valgrind support

#37

Post by defraudbah »

Thaxll wrote: post #8
defraudbah wrote: post #2

looks very promising, one of the biggest issue in golang for me is profiling and constant memory leaks/pressure. Not sure if there is an alternative of what people use now

pprof is pretty good, what do you need?

yes, that's what I use, just wonder if there are alternatives. I am not sure how valgrind compares to it or the goref tool mentioned above, just asking around, does not hurt.

Re: Go has added Valgrind support

#38

Post by Thaxll »

amelius wrote: post #3

It only works if every package tests with it. Otherwise the relevant warnings get swamped by a huge amount by irrelevant warnings. This is why running Valgrind on Python code does not work.

What do you mean if every package tests with it in the context of Go?

Re: Go has added Valgrind support

#39

Post by gdbsjjdn »

giancarlostoro wrote: post #25
dijit wrote: post #17

Earlier quoted context omitted.

There’s ways, GC isn’t perfect . A common one I see fairly often is opening a big file, creating a “new slice” on a subset of the file and then using the “new slice” and expecting the old large object to be dropped. Except, the “new slice” is just a reference into the larger slice, so its never marked unused.

Interesting, I always thought of slices as stand-alone, I wonder if its the same in Python?

A go slice is a wrapper around a normal array. When you take sub-slices those also point to the original array. There's a possible optimization to avoid this footgun where they could reallocate a smaller array if only subslices are reachable (similar to how they reallocate if a slice grows beyond the size of the underlying array).

Re: Go has added Valgrind support

#40

Post by amelius »

sauercrowd wrote: post #29
giancarlostoro wrote: post #23

Earlier quoted context omitted.

> it worked fine after spending an hour making a suppressions file. So you are confirming the problem, but treating it as if ignoring it is the solution for all?

it's a rejection of the thesis that it "does not work". It does, but it requires investing into a suppression file.

Yeah I tried that on a PySide6 application.

Trust me, it does not work.

Post reply on HN