Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How expensive is integer-overflow trapping in C++?

lemire.me

Post reply on HN
Print view View original

91–100 of 198 posts

Re: How expensive is integer-overflow trapping in C++?

#91

Post by pjc50 »

amelius wrote: post #69
quietbritishjim wrote: post #59

> adding two large integers can result in an integer that cannot be represented in the integer type. We often refer to such error conditions as overflows. There's a subtlety here that's missed in the article: In C and C++ this is only called "overflow" for signed integers. That's because that word specifically refers to the case where the behaviour is undefined, and for unsigned integers the result is always well def…

We probably need different integer types to express the differences. Perhaps whether or not we check for overflow can be encoded in the type of the integer.

Yes. For some time I've wondered about doing a programming language for conservative arithmetic, in which you don't ask for "integer" but specify "32-bit signed, exception on overflow" or "16-bit, do not exception immediately but set a flag which I can check at the end of the function" or "8-bit, use saturation arithmetic".

The exact opposite of the scripting language "everything is just a thing and you can add numbers to strings or whatever idk".

Saturation arithmetic is useful in e.g. audio codecs as it corresponds to "clipping"; there's hardware support for it in some SIMD instruction sets and I think it's required for some compression standards. But compilers won't generate it because there's no way to easily represent it in most languages other than hope for pattern-matching in the optimizer.

Would also be a good place for "please compile this function with constant execution time regardless of inputs and fail if you can't do that", useful for crypto.

Re: How expensive is integer-overflow trapping in C++?

#92

Post by wahern »

zeotroph wrote: post #89
wahern wrote: post #47

Earlier quoted context omitted.

This reminds me of Rust unconditionally ignoring SIGPIPE. Except in this case there's a non-trivial amount of pre-existing (and even future) code that assumes a process will be terminated if it attempts to write to a closed pipe. One of Rust's selling points is ease of FFI, so it's largely irrelevant that Rust APIs wouldn't let you ignore write errors. Rust breaks the environment for libraries that rely on the behavi…

Yes, quite annoying, and I've been bitten by that before, but you are supposed to check the return of each write(). And I think the motivation is to write platform-agnostic code by default. SIGPIPE is only available on Unix, so you have to explicitly opt-in.

The original impetus seems to have been for the now defunct green threading concurrency approach: https://github.com/rust-lang/rust/pull/13158/files

One of the post hoc rationales that seems to have cropped up is as you mentioned. But that logic only makes sense if you expect platform-specific code to be rare. I doubt that's the case: from my limited experience there seems to be quite a lot of Rust code written specifically for the Unix environment. Rust is a "systems" language after all; writing platform-specific applications, not simply platform-specific modules or wrappers, is to be expected.

It's a legitimate glitch in Rust. It happens. I don't understand the urge for people to defend Rust here. It violates well accepted norms in the world of systems programming--don't f'up the global environment without being asked. Indeed, part of the reason green threading failed was because it required too much magic interposed between the application and the native platform environment. Rust wasn't trying to go the route of Go.

Re: How expensive is integer-overflow trapping in C++?

#93

Post by mhh__ »

asveikau wrote: post #8

Aborting the program on integer overflow seems so drastic. Say you do that in a server program. Then convincing it to overflow an integer somewhere (possibly somewhere where it is harmless) causes denial of service and all unrelated connections in the process die? I guess languages with support for exceptions have a more natural action for this, that could be caught somewhere in a server's processing loop to not affe…

That's the tradeoff at play here. Think of it like debugging a python program, it might fail hours after you start it but if it does fail it's failed for a reason (uncaught overflows can be very very not good)

Re: How expensive is integer-overflow trapping in C++?

#94

Post by mhh__ »

amelius wrote: post #74
asveikau wrote: post #8

Aborting the program on integer overflow seems so drastic. Say you do that in a server program. Then convincing it to overflow an integer somewhere (possibly somewhere where it is harmless) causes denial of service and all unrelated connections in the process die? I guess languages with support for exceptions have a more natural action for this, that could be caught somewhere in a server's processing loop to not affe…

Perhaps we can have both. E.g. __checked__ int a; // aborts on overflow __throwing_checked__ int b; // throws exception on overflow You could have the same syntax for unsigned ints.

D has checkedint which does this cleanly in a template

Re: How expensive is integer-overflow trapping in C++?

#95

Post by mhh__ »

xxs wrote: post #19

Writing (compiler) microbenchmarks is notoriously hard. Overall no benchmark should run fewer than two mins or so to account for context switching noises at least. It should run on fixed CPU frequency (no turbo boost, etc), fixed CPU core(s) too. It should be aware of L1/L2 cache sizes - data crossing L1/L2 boundaries on different processors results in disproportional results. The difference in compilers is too high…

I think unless you own the entire machine almost all microarchitecture level benchmarking is playing with fire because of all the things you've mentioned. Even if you use perf_event to get as low as the OS allows you to account for things that'll slow you down there's still overhead associated with that AFAIK

Also, code disassembly is a fairly awful way of assessing performance unless you mean for the purposes of the article

Re: How expensive is integer-overflow trapping in C++?

#96

Post by xxs »

mhh__ wrote: post #95
xxs wrote: post #19

Writing (compiler) microbenchmarks is notoriously hard. Overall no benchmark should run fewer than two mins or so to account for context switching noises at least. It should run on fixed CPU frequency (no turbo boost, etc), fixed CPU core(s) too. It should be aware of L1/L2 cache sizes - data crossing L1/L2 boundaries on different processors results in disproportional results. The difference in compilers is too high…

I think unless you own the entire machine almost all microarchitecture level benchmarking is playing with fire because of all the things you've mentioned. Even if you use perf_event to get as low as the OS allows you to account for things that'll slow you down there's still overhead associated with that AFAIK Also, code disassembly is a fairly awful way of assessing performance unless you mean for the purposes of the…

Article in this case, hence the new paragraph. I used to routinely do for Java... with -XX:+PrintAssembly to ensure certain bound checks were eliminated by enforcing the index fell within the lattice [0-array.length). A simple test that always succeeded improved zip compression by 15% for instance.

Not going in details but verifying results of microbenchmarks with assembly is quite useful - of course usually limited to very hot spots. Of course about owning the hardware and doing on bare metal with fixed freq (core/uncore) + set voltage(s). Removing as many variables as possible is important just as knowing what to test and how it gets affected by the hardware.

Re: How expensive is integer-overflow trapping in C++?

#97

Post by higerordermap »

lmm wrote: post #62
asveikau wrote: post #8

Aborting the program on integer overflow seems so drastic. Say you do that in a server program. Then convincing it to overflow an integer somewhere (possibly somewhere where it is harmless) causes denial of service and all unrelated connections in the process die? I guess languages with support for exceptions have a more natural action for this, that could be caught somewhere in a server's processing loop to not affe…

> Aborting the program on integer overflow seems so drastic. Say you do that in a server program. Then convincing it to overflow an integer somewhere (possibly somewhere where it is harmless) causes denial of service and all unrelated connections in the process die? If you don't abort then you probably have an arbitrary code execution vulnerability. Overflow is undefined behaviour in C++, your code is in an unanticip…

He is saying it should raise an exception which can be caught somewhere up the stack.

Re: How expensive is integer-overflow trapping in C++?

#98

Post by xxs »

rocqua wrote: post #72
xxs wrote: post #19

Writing (compiler) microbenchmarks is notoriously hard. Overall no benchmark should run fewer than two mins or so to account for context switching noises at least. It should run on fixed CPU frequency (no turbo boost, etc), fixed CPU core(s) too. It should be aware of L1/L2 cache sizes - data crossing L1/L2 boundaries on different processors results in disproportional results. The difference in compilers is too high…

At what performance differential are those factors relevant? For a 5% difference I would share your sceptisism. However, the effect size is a 4x difference (3x slowdown vs 12x slowdown). To me, it seems very unlikely to be a result of inconsistent benchmarking. However, I am not an expert on the subject.

The test is too short to measure anything of use. Also you want the same code run multiple times and the results collected, at the least you'd like standard std dev between tests.

Re: How expensive is integer-overflow trapping in C++?

#99

Post by bregma »

masklinn wrote: post #78
quietbritishjim wrote: post #59

> adding two large integers can result in an integer that cannot be represented in the integer type. We often refer to such error conditions as overflows. There's a subtlety here that's missed in the article: In C and C++ this is only called "overflow" for signed integers. That's because that word specifically refers to the case where the behaviour is undefined, and for unsigned integers the result is always well def…

> Is it just about signed numbers or not? No. You can plug an `UInt8(255)` in an online playground and try to increment it, it’ll crash unless you use an overflowing operator. > Also, what other languages are there that are "like Swift"? In debug mode, Rust will panic on overflow (regardless of signing). You can also enable overflow-checks in release mode.

Huh. That would make classic LCG PRNGs pretty useless. I guess it's "safer" in that it prevents rare dangerous edge cases at the expense of common everyday safe operations, but that's a tradeoff.

Re: How expensive is integer-overflow trapping in C++?

#100

Post by galad87 »

alblue wrote: post #22
asveikau wrote: post #8

Aborting the program on integer overflow seems so drastic. Say you do that in a server program. Then convincing it to overflow an integer somewhere (possibly somewhere where it is harmless) causes denial of service and all unrelated connections in the process die? I guess languages with support for exceptions have a more natural action for this, that could be caught somewhere in a server's processing loop to not affe…

This is essentially Swift’s problem on the server. If you have a service running and serving hundreds of clients, then any one of those threads can cause a trivial panic if there’s an overflow and take the entire service down. As a result, a number of use cases for swift on the server actually use heavyweight out of process threads (i.e. one process per client) so that the failure of one of them doesn’t take down the…

There is a compiler option to disable overflow/underflow checks, so unless the overflow is in some library outside your control, it's not actually an issue.
Post reply on HN