Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Wild performance tricks

davidlattimore.github.io

Post reply on HN
Print view View original

41–50 of 86 posts

Re: Wild performance tricks

#41

Post by haileys »

jstimpfle wrote: post #39
haileys wrote: post #37

Earlier quoted context omitted.

This is like saying there’s no point having unprivileged users if you’re going to install sudo anyway. The point is to escalate capability only when you need it, and you think carefully about it when you do. This prevents accidental mistakes having catastrophic outcomes everywhere else.

I think sudo is a great example. It's not much more secure than just logging in at root. It doesn't really protect malicious attackers in practice. And it's more of an annoyance than it protects against accidental mistakes in practice.

Unsafe isn’t a security feature per se. I think this is where a lot of the misunderstanding comes from.

It’s a speed bump that makes you pause to think, and tells reviewers to look extra closely. It also gives you a clear boundary to reason about: it must be impossible for safe callers to trigger UB in your unsafe code.

Re: Wild performance tricks

#42

Post by jstimpfle »

dwattttt wrote: post #33
jstimpfle wrote: post #9

Earlier quoted context omitted.

In C I wouldn't use such a fluffy high-level approach in the first place. I wouldn't use contiguous unbounded vec-slices. And no, I wouldn't attempt trickery with overwriting input buffers. That's a bad inflexible approach that will bite at the next refactor. Instead, I would first make sure there's a way to cheaply allocate fixed size buffers (like 4 K buffers or whatever) and stream into those. Memory should be use…

> straightforward, efficient architecture and bug-free code The grace with which C handles projects of high complexity disagrees. You get a simple implementation only by ignoring edge cases or improvements that increase complexity.

The idea that a language can handle any complexity for you is an illusion. A language can automate a lot of the boring and repetitive small scale work. And it can have some of the things you would have otherwise coded yourself as built-ins. However you still have to deal with the complexity caused by buying into these built-ins. The larger a project gets the more likely the built-ins are to get in the way, and the more likely you are to rewrite these features or sub-systems yourself.

I'd say, more fully featured languages are most useful for the simpler side of projects (granted some of them can scale quite a way up with proficient use).

Now go research how some of the most complex, flexible, and efficient pieces of software are written.

Re: Wild performance tricks

#43

Post by AnyTimeTraveler »

quotemstr wrote: post #34
0x1ceb00da wrote: post #22

Earlier quoted context omitted.

> Great example of Rust being built such that you have to deal with error returns and think about C++-style exception safety. Not really. Panics are supposed to be used in super exceptional situations, where the only course of action is to abort the whole unit of work you're doing and throw away all the resources. However you do have to be careful in critical code because things like integer overflow can also raise a…

> be careful in critical code because things like integer overflow can also raise a panic So you can basically panic anywhere. I understand people have looked at no-panic markers (like C++ noexcept) but the proposals haven't gone anywhere. Consequently, you need to maintain the basic exception safety guarantee [1] at all times. In safe Rust, the compiler enforces this level of safety in most cases on its own, but the…

Rust's number types have functions like "wrapping_add" or "overflowing_add", which do not panic when overflowing and instead explicitly wrap around or return a result that must be checked.

You can easily write code that does not contain any possible panic points, if you want.

Re: Wild performance tricks

#44

Post by vlovich123 »

0x1ceb00da wrote: post #22
quotemstr wrote: post #11

> Even if it were stable, it only works with slices of primitive types, so we’d have to lose our newtypes (SymbolId etc). That's weird. I'd expect it to work with _any_ type, primitive or not, newtype or not, with a sufficiently simple memory layout, the rough equivalent of what C++ calls a "standard-layout type" or (formerly) a "POD". I don't like magical stdlibs and I don't like user types being less powerful than…

> Great example of Rust being built such that you have to deal with error returns and think about C++-style exception safety. Not really. Panics are supposed to be used in super exceptional situations, where the only course of action is to abort the whole unit of work you're doing and throw away all the resources. However you do have to be careful in critical code because things like integer overflow can also raise a…

> However you do have to be careful in critical code because things like integer overflow can also raise a panic.

This is incorrect. Only in debug builds does it raise a panic. In release Rust has to make the performance tradeoff that C++ does and defines signed integer math to wrap 2’s complement. Only in debug will signed overflow panic. Unsigned math never panics - it’s always going to overflow 2’s complement.

Re: Wild performance tricks

#45

Post by jstimpfle »

kibwen wrote: post #16
jstimpfle wrote: post #9

Earlier quoted context omitted.

In C I wouldn't use such a fluffy high-level approach in the first place. I wouldn't use contiguous unbounded vec-slices. And no, I wouldn't attempt trickery with overwriting input buffers. That's a bad inflexible approach that will bite at the next refactor. Instead, I would first make sure there's a way to cheaply allocate fixed size buffers (like 4 K buffers or whatever) and stream into those. Memory should be use…

> In C I wouldn't use such a fluffy high-level approach in the first place. Sure, though that's because C has abstraction like Mars has a breathable atmosphere. > This approach leads to straightforward, efficient architecture and bug-free code. It's also much better for concurrency/parallelism. This claim is wild considering that Rust code is more bug-free than C code while being just as efficient, while keeping in m…

I'm not even sure what it means for a language to "have" abstractions. Abstractions are created by competent software engineers, according to the requirements. A language can have features that make creating certain kinds of abstractions easier -- for example type-abstractions. I've stopped thinking that type abstractions are all that important. Somehow creating those always leads to decision paralysis and scope creep, and using those always leads to layers of bloat and less than straightforward programs.

Re: Wild performance tricks

#46

Post by jstimpfle »

vlovich123 wrote: post #40
jstimpfle wrote: post #9

Earlier quoted context omitted.

In C I wouldn't use such a fluffy high-level approach in the first place. I wouldn't use contiguous unbounded vec-slices. And no, I wouldn't attempt trickery with overwriting input buffers. That's a bad inflexible approach that will bite at the next refactor. Instead, I would first make sure there's a way to cheaply allocate fixed size buffers (like 4 K buffers or whatever) and stream into those. Memory should be use…

Have you written a linker before? It sounds like you’re describing I/O but that isn’t the work being done here by the linker.

I've considered writing one. Why do you think what I'm describing here only applies to I/O (like syscalls)? And, more abstractly speaking -- isn't everything an I/O (like input/output) problem?

Re: Wild performance tricks

#47

Post by sweetjuly »

ashvardanian wrote: post #25

I’d strongly caution against many of those “performance tricks.” Spawning an asynchronous task on a separate thread, often with a heap-allocated handle, solely to deallocate a local object is a dubious pattern — especially given how typical allocators behave under the hood. I frequently encounter use-cases akin to the “Sharded Vec Writer” idea, and I agree it can be valuable. But if performance is a genuine requireme…

> especially given how typical allocators behave under the hood.

To say more about it: nearly any modern high performance allocator will maintain a local (private) cache of freed chunks.

This is useful, for example, if you're allocating and deallocating about the same amount of memory/chunk size over and over again since it means you can avoid entering the global part of the allocator (which generally requires locking, etc.).

If you make an allocation while the cache is empty, you have to go to the global allocator to refill your cache (usually with several chunks). Similarly, if you free and find your local cache is full, you will need to return some memory to the global allocator (usually you drain several chunks from your cache at once so that you don't hit this condition constantly).

If you are almost always allocating on one thread and deallocating on another, you end up increasing contention in the allocator as you will (likely) end up filling/draining from the global allocator far more often than if you kept in on just one CPU. Depending on your specific application, maybe this performance loss is inconsequential compared to the value of not having to actually call free on some critical path, but it's a choice you should think carefully about and profile for.

Re: Wild performance tricks

#48

Post by quotemstr »

vlovich123 wrote: post #44
0x1ceb00da wrote: post #22

Earlier quoted context omitted.

> Great example of Rust being built such that you have to deal with error returns and think about C++-style exception safety. Not really. Panics are supposed to be used in super exceptional situations, where the only course of action is to abort the whole unit of work you're doing and throw away all the resources. However you do have to be careful in critical code because things like integer overflow can also raise a…

> However you do have to be careful in critical code because things like integer overflow can also raise a panic. This is incorrect. Only in debug builds does it raise a panic. In release Rust has to make the performance tradeoff that C++ does and defines signed integer math to wrap 2’s complement. Only in debug will signed overflow panic. Unsigned math never panics - it’s always going to overflow 2’s complement.

> Only in debug builds does it raise a panic.

Correctness in debug builds is important, isn't it?

That said, panic on integer overflow in debug builds is unfortunate behavior. Overflow should cause an abort, not a panic.

> make the performance tradeoff that C++ does and defines signed integer math to wrap 2’s complement

In C++, signed overflow is undefined behavior, not wraparound. This property is useful to the optimizer for things like inferring loop bounds. The optimizer has less flexibility in equivalent Rust code.

Re: Wild performance tricks

#49

Post by quotemstr »

AnyTimeTraveler wrote: post #43
quotemstr wrote: post #34

Earlier quoted context omitted.

> be careful in critical code because things like integer overflow can also raise a panic So you can basically panic anywhere. I understand people have looked at no-panic markers (like C++ noexcept) but the proposals haven't gone anywhere. Consequently, you need to maintain the basic exception safety guarantee [1] at all times. In safe Rust, the compiler enforces this level of safety in most cases on its own, but the…

Rust's number types have functions like "wrapping_add" or "overflowing_add", which do not panic when overflowing and instead explicitly wrap around or return a result that must be checked. You can easily write code that does not contain any possible panic points, if you want.

I don't think it's quite as easy to guarantee panic freedom as you think.

For example: do logging frameworks guarantee no-panic behavior? People can add logging statements practically anywhere, especially in a large team that maintains a codebase over significant time. One innocuous-looking debug log added to a section of code that's temporarily violated invariants can end up putting the whole program into a state, post-panic, in which those invariants no longer hold.

A lot of experience tells us that this happens in practice in C++, Java, Python, and other excpeption-ful languages. Maybe it happens less in Rust, but I'd be shocked if this class of bugs were absent.

Note that I'm talking about both safe and unsafe code. A safe section of code that panics unexpectedly might preserve memory safety invariants but hork the higher-level logical invariants of your application. You can end up with security vulnerabilities this way too.

Imagine an attacker who can force a panic in a network service, aborting his request but not killing the server, such that the panic on his next request grants him some kind of access he shouldn't have had due to the panic leaving the program in a bad state.

I'm not seeing Rust people take this problem as seriously as I think is warranted.

Re: Wild performance tricks

#50

Post by jstimpfle »

haileys wrote: post #41
jstimpfle wrote: post #39

Earlier quoted context omitted.

I think sudo is a great example. It's not much more secure than just logging in at root. It doesn't really protect malicious attackers in practice. And it's more of an annoyance than it protects against accidental mistakes in practice.

Unsafe isn’t a security feature per se. I think this is where a lot of the misunderstanding comes from. It’s a speed bump that makes you pause to think, and tells reviewers to look extra closely. It also gives you a clear boundary to reason about: it must be impossible for safe callers to trigger UB in your unsafe code.

That's my point; I think after a while you instinctly repeat a command with sudo tacked on (see XKCD), and I wonder if I'm any safer from myself like that?

I'm doubtful that those boundaries that you mention really work so great. I imagine that in practice you can easily trigger faulty behaviours in unsafe code from within safe code. Practical type systems are barely powerful enough to let you inject a proof of valid-state into the unsafe-call. Making a contract at the safe/unsafe boundary statically enforceable (I'm not doubting people do manage to do it in practice but...) probably requires a mountain of unessential complexity and/or runtime checks and less than optimal algorithms & data structures.

Post reply on HN