Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Finding and fixing Ghostty's largest memory leak

mitchellh.com

Post reply on HN
Print view View original

131–140 of 152 posts

Re: Finding and fixing Ghostty's largest memory leak

#131

Post by vacuity »

uecker wrote: post #108
tialaramex wrote: post #107

Earlier quoted context omitted.

With Rust you always could unsafely do whatever went wrong in somebody's C or Zig or whatever, but the question is whether you would. Rust's technical design reinforces a culture where the answer is usually "No". I don't find the claim that weird low level mmap tricks here are perf critical at all persuasive. The page recycling makes sense - I can see why that's helping performance, but the bare metal mmap calls smel…

Also in C or Zig you do not need to create your own memory management using mmap. Whether this is necessary in this case or not is a different question. In the end, if the Rust advantage is that "Rust's technical design reinforces a culture" where one tries to avoid this, then this is a rather weak argument. We will see how this turns out in the long run though.

Of course, culture and technical design are both important for any language, but be specific. Despite the prevalence of tools that improve C's safety, writing C safely generally requires a culture of using those tools and other techniques. For better or worse, Rust's borrow checker is a clear demonstration of where Rust lies on the safety-freedom spectrum.

Re: Finding and fixing Ghostty's largest memory leak

#132

Post by andrewaylett »

Let me see if I can understand this properly:

There's a linear buffer of pages, most of which come from the pool. It's not clear to me under what conditions these are returned to the pool? Is it when the specific session terminates?

When a non-standard page reaches the point of being recycled, it'll instead be re-added to the list but with a standard size. That effectively leaks the extra space above the standard size. But when the buffer is released (because the session ends?) the pool is also released, which releases all the standard sized pages but leaks the custom-sized ones?

Which suggests that the issue may be even rarer than it initially looked to me: I tend to open a small number of sessions and then use them continuously, rather than starting new sessions during the lifetime of the process. If I never terminated a session, I would never fully leak the memory?

Re: Finding and fixing Ghostty's largest memory leak

#135

Post by barnabee »

cbmuser wrote: post #94
yoyohello13 wrote: post #56

Earlier quoted context omitted.

A lot of developers use the terminal as their primary interaction with the computer. Nvim, tmux, etc. Having it be fast is an extreme quality of life improvement. For devs who only ever use the terminal integrated into their ide then it’s probably less important.

Can you elaborate on that a bit, please? I have never found myself in the situation where my terminal emulator would be too slow and I‘m using it for the majority of my day-to-day work. I honestly never ran into a situation where I would habe blamed the terminal emulator for being too slow.

I think this kind of thing just bothers some people and not others.

I first started to understand and notice update rates and responsiveness as a gamer playing 1st person shooters.

I hate (ok, I find it a bit jarring) the jerky scrolling of a phone in battery save mode limited to 60(?) FPS. It’s so obviously not connected to your touch anymore.

In terminals it’s things like the responsiveness fuzzy finders and scrolling that I really notice.

I turn off animations everywhere I can.

It’s not impossible to use something slower, but when everything feels instant it’s just much more pleasant, smoother, and feels more productive as a result of the computer working at whatever speed my brain does.

Re: Finding and fixing Ghostty's largest memory leak

#136

Post by uecker »

tialaramex wrote: post #114
uecker wrote: post #108

Earlier quoted context omitted.

Also in C or Zig you do not need to create your own memory management using mmap. Whether this is necessary in this case or not is a different question. In the end, if the Rust advantage is that "Rust's technical design reinforces a culture" where one tries to avoid this, then this is a rather weak argument. We will see how this turns out in the long run though.

> We will see how this turns out in the long run though. Rust 1.0 was in 2015. This is the long run. And I disagree that safety culture is a "weak argument". It's foundational, this is where you must start, adding it afterwards is a Herculean task, so no surprise that people aren't really trying.

I am not saying that safety culture is irrelevant, not at all. I am saying that if the advantage of Rust is the culture that emphasizes safety (or rather memory safety, if the Rust community cared about safety in general cargo would not exist in this form) then that is a weak argument.

I don't think 10 years ago there was a lot of Rust used, so I am not sure how relevant it is that 1.0 was released at this time.

Re: Finding and fixing Ghostty's largest memory leak

#137

Post by uecker »

vlovich123 wrote: post #124
uecker wrote: post #108

Earlier quoted context omitted.

Also in C or Zig you do not need to create your own memory management using mmap. Whether this is necessary in this case or not is a different question. In the end, if the Rust advantage is that "Rust's technical design reinforces a culture" where one tries to avoid this, then this is a rather weak argument. We will see how this turns out in the long run though.

The long run has already spoken. Go look at the reports out of Microsoft and Android. It’s screamingly clear that the philosophy of Rust that most code can be written in safe with small bits in unsafe is inherently safer. The defect rate plummets by one or two orders of magnitude if I recall correctly. C is an absolute failure (since it’s the baseline) and Zig has no similar adoption studies. You could argue it will…

I don't put too much wait on the self-reporting by Microsoft or Google. I agree though that the strategy to write safe bits and abstractions is good. What I know not to be true is the idea that similar strategies would not work also in C.

Re: Finding and fixing Ghostty's largest memory leak

#138

Post by tialaramex »

vlovich123 wrote: post #123
tialaramex wrote: post #107

Earlier quoted context omitted.

With Rust you always could unsafely do whatever went wrong in somebody's C or Zig or whatever, but the question is whether you would. Rust's technical design reinforces a culture where the answer is usually "No". I don't find the claim that weird low level mmap tricks here are perf critical at all persuasive. The page recycling makes sense - I can see why that's helping performance, but the bare metal mmap calls smel…

I think we can agree that Mitchell knows what he’s doing and isn’t playing around with mmap just because. It’s probably quite important to ensure a low memory footprint. But mmap in rust is not extra risky in some weird mystical way. It’s just a normal FFI function to get a pointer back and you can trivially build safe abstractions around it to ensure the lifetime of a slice doesn’t exceed the lifetime of the underly…

I actually don't think I agree about mmap. Reading around it seems as though Mitchell had clever ideas for abusing mmap and those didn't work out. Now he's got mmap for the pages and it works so why replace it, but that does not mean you need mmap to deliver this performance and in fact I'd be extremely surprised if that were true as somebody who spent about a decade of his life mostly writing close-to-metal database code in C using mmap...

If you want a whole lot of bytes and you ask your allocator, do you know what almost any popular general purpose allocator will do on a vaguely decent modern Unix? Call mmap to get them for you. So at most you're cutting out a few CPU instructions worth of middle man.

Re: Finding and fixing Ghostty's largest memory leak

#139

Post by vlovich123 »

uecker wrote: post #137
vlovich123 wrote: post #124

Earlier quoted context omitted.

The long run has already spoken. Go look at the reports out of Microsoft and Android. It’s screamingly clear that the philosophy of Rust that most code can be written in safe with small bits in unsafe is inherently safer. The defect rate plummets by one or two orders of magnitude if I recall correctly. C is an absolute failure (since it’s the baseline) and Zig has no similar adoption studies. You could argue it will…

I don't put too much wait on the self-reporting by Microsoft or Google. I agree though that the strategy to write safe bits and abstractions is good. What I know not to be true is the idea that similar strategies would not work also in C.

> What I know not to be true is the idea that similar strategies would not work also in C.

Is your argument that developers at MS and Google haven’t been trying to employ these strategies for existing C codebases? It’s a bold position to take and one I’d say devoid of evidence; all the evidence suggests it’s really hard to reason about ownership in complex systems and abstractions only help you do so error free up to a very limited point.

Re: Finding and fixing Ghostty's largest memory leak

#140

Post by vlovich123 »

uecker wrote: post #136
tialaramex wrote: post #114

Earlier quoted context omitted.

> We will see how this turns out in the long run though. Rust 1.0 was in 2015. This is the long run. And I disagree that safety culture is a "weak argument". It's foundational, this is where you must start, adding it afterwards is a Herculean task, so no surprise that people aren't really trying.

I am not saying that safety culture is irrelevant, not at all. I am saying that if the advantage of Rust is the culture that emphasizes safety (or rather memory safety, if the Rust community cared about safety in general cargo would not exist in this form) then that is a weak argument. I don't think 10 years ago there was a lot of Rust used, so I am not sure how relevant it is that 1.0 was released at this time.

The culture of Rust is pretty uniform both in terms of convention (lots of good examples to learn from) and automated tooling (eg cargo clippy can fix many constructs into cleaner versions).

But sure, ultimately any code you see is limited by the talent of the author. However the safety of that code is not - it’s limited by how many unsafe blocks they wrote which you can actually grep for.

Post reply on HN