Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Python rounds float values by converting them to string and then back

github.com

Post reply on HN
Print view View original

71–80 of 152 posts

Re: Python rounds float values by converting them to string and then back

#71

Post by ericfrederich »

stefan_ wrote: post #43
fs111 wrote: post #6

Apples libc used to shell-out to perl in a function: https://github.com/Apple-FOSS-Mirror/Libc/blob/2ca2ae7464771...

I thought this is what the Unix philosophy is supposed to be all about. (Realistically, calling wordexp should just abort the program. Now I actually want to make a hacked up musl that aborts in all the various "libc functions no one should ever use" and see how far I get into a Ubuntu boot..)

/* wordexp is also rife with security "challenges", unless you pass it WRDE_NOCMD it must support subshell expansion, and even if you don't beause it has to support so much of the standard shell (all the odd little variable expansion options for example) it is hard to do without a subshell). It is probbably just plan a Bad Idea to call in anything setuid, or executing remotely. */

Re: Python rounds float values by converting them to string and then back

#73

Post by wjmao88 »

SideQuark wrote: post #55
coldtea wrote: post #14

Seems to be one of the best ways to go about it. From the comment in protobuf source (which does the same thing as Python), mentioned in the Twitter thread: (...) An arguably better strategy would be to use the algorithm described in "How to Print Floating-Point Numbers Accurately" by Steele & White, e.g. as implemented by David M. Gay's dtoa(). It turns out, however, that the following implementation is about as fas…

>Seems to be one of the best ways to go about it. The C/C++ standards do not require formatting to round correctly or even be portable. I recently had an issue where a developer used this method to round floats for display, and there were differences on PC and on Mac. It literally rounded something like 18.25 to 18.2 on one platform and 18.3 on the other. This led to all sorts of other bugs as some parts of the progr…

If you want anything approaching consistency or predictability, don't round your numbers in transit and only for display.

Re: Python rounds float values by converting them to string and then back

#74

Post by dagw »

ericfrederich wrote: post #62

I was looking once at Python and Redis and how numbers get stored. I remember Python would in the end send Redis some strings. I dove pretty deep and found that Python floats when turned into a string and then back are exactly the same float. I remember even writing a program that tested every possible floating point number (must have only been 32 bit). I think I used ctypes and interpreted every binary combination o…

A lot of them were NaN.

I seem to recall that ~0.5% of the IEEE 32 bit float space is NaN.

Re: Python rounds float values by converting them to string and then back

#75

Post by bregma »

TekMol wrote: post #15
latchkey wrote: post #4

https://0.30000000000000004.com/

Computers can only natively store integers, so they need some way of representing decimal numbers. Really? How do computers "natively" store integers?

Maybe you need to parse "decimal numbers" as "fractional numbers" rather than "expressed using a radix of 10".

Perhaps it would be better phrased as "computers can only store whole numbers, so they need some way to represent other information, including integers, rational numbers and approximations to complex numbers."

Re: Python rounds float values by converting them to string and then back

#76

Post by xienze »

stefan_ wrote: post #43
fs111 wrote: post #6

Apples libc used to shell-out to perl in a function: https://github.com/Apple-FOSS-Mirror/Libc/blob/2ca2ae7464771...

I thought this is what the Unix philosophy is supposed to be all about. (Realistically, calling wordexp should just abort the program. Now I actually want to make a hacked up musl that aborts in all the various "libc functions no one should ever use" and see how far I get into a Ubuntu boot..)

> I thought this is what the Unix philosophy is supposed to be all about.

Perhaps from the perspective of an end user running things from a shell. Generally speaking though, shelling out from within a program is not ideal.

Re: Python rounds float values by converting them to string and then back

#77

Post by fernandotakai »

mkl wrote: post #37
zsrxx wrote: post #27

Earlier quoted context omitted.

It's Python, it's not like anybody is going to complain about performance, because nobody expects any.

Many people complain about performance, and enormous amounts of effort have been spent on improving it, successfully. People do expect good performance, and can achieve it in many useful cases.

i mean, if you are worried about python's rounding number performance, you are probably using numpy already (which has a different implementation that is faster).

Re: Python rounds float values by converting them to string and then back

#78

Post by ImNotTheNSA »

eesmith wrote: post #57
SideQuark wrote: post #55

Earlier quoted context omitted.

>Seems to be one of the best ways to go about it. The C/C++ standards do not require formatting to round correctly or even be portable. I recently had an issue where a developer used this method to round floats for display, and there were differences on PC and on Mac. It literally rounded something like 18.25 to 18.2 on one platform and 18.3 on the other. This led to all sorts of other bugs as some parts of the progr…

> The C/C++ standards do not require formatting to round correctly or even be portable. The linked-to method uses PyOS_snprintf(). Its documentation at https://docs.python.org/3/c-api/conversion.html says: """PyOS_snprintf() and PyOS_vsnprintf() wrap the Standard C library functions snprintf() and vsnprintf(). Their purpose is to guarantee consistent behavior in corner cases, which the Standard C functions do not."""

response was to the fact that the comment said the method of format strings was one of “the best ways to go about it”

It’s obvious that PyOS_snprintf is not a standard library function

Re: Python rounds float values by converting them to string and then back

#79

Post by raphlinus »

analog31 wrote: post #51

My quick impression is that the choice of a rounding algorithm is relative to the purpose that it serves. For instance, floor(x + 0.5) is good enough in many applications. In some cases, rounding is performed for the primary purpose of displaying a number as a string, in which case it can't be any less complicated than the string conversion function itself.

Fun fact: floor(x + 0.5) rounds 0.49999997 to 1.0 (this is 32 bit floats, the same principle applies to 64). Most libraries have slower than ideal round conversion because of historical dross; modern chips have a very fast SIMD round instruction but its behavior doesn't exactly match libc round. See https://github.com/rust-lang/rust/issues/55107 for a deeper discussion.

Re: Python rounds float values by converting them to string and then back

#80

Post by ImNotTheNSA »

peterwwillis wrote: post #72
fs111 wrote: post #6

Apples libc used to shell-out to perl in a function: https://github.com/Apple-FOSS-Mirror/Libc/blob/2ca2ae7464771...

What if Perl uses libc ?????

then it calls the Perl implementation... is there some nuance of perl that would cause a problem with that?
Post reply on HN