Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Python rounds float values by converting them to string and then back

github.com

Post reply on HN
Print view View original

101–110 of 152 posts

Re: Python rounds float values by converting them to string and then back

#101

Post by mark-r »

yjftsjthsd-h wrote: post #10
ChrisSD wrote: post #5

Maybe I'm missing something but what's wrong with rounding floats this way?

I don't know that it is "wrong", just unexpected. I suspect most people expect all math functions to be purely implemented in numerical terms, so finding string manipulation is surprising/interesting.

There are many corner cases involved with rounding, and the folks who did the string conversion had to put a lot of effort into handling all of them. It makes sense to piggyback on their efforts, even if it isn't the most efficient way 99% of the time.

Re: Python rounds float values by converting them to string and then back

#102

Post by Shorel »

mkl wrote: post #37
zsrxx wrote: post #27

Earlier quoted context omitted.

It's Python, it's not like anybody is going to complain about performance, because nobody expects any.

Many people complain about performance, and enormous amounts of effort have been spent on improving it, successfully. People do expect good performance, and can achieve it in many useful cases.

I used to complain about performance, as I remember the abysmally bad performance of Zope (Python web framework) back in 2003 -2004.

It made me write off that language for about a decade.

Benchmarking Python3 code reveals that there have been some quite noticeable improvements.

Re: Python rounds float values by converting them to string and then back

#103

Post by eesmith »

ImNotTheNSA wrote: post #78
eesmith wrote: post #57

Earlier quoted context omitted.

> The C/C++ standards do not require formatting to round correctly or even be portable. The linked-to method uses PyOS_snprintf(). Its documentation at https://docs.python.org/3/c-api/conversion.html says: """PyOS_snprintf() and PyOS_vsnprintf() wrap the Standard C library functions snprintf() and vsnprintf(). Their purpose is to guarantee consistent behavior in corner cases, which the Standard C functions do not."""

response was to the fact that the comment said the method of format strings was one of “the best ways to go about it” It’s obvious that PyOS_snprintf is not a standard library function

Sure, PyOS_snprintf isn't a standard library function, but it's a thin wrapper to snprintf, which is. Python/mysnprintf.c is the location of the:

   snprintf() wrappers.  If the platform has vsnprintf, we use it, else we
   emulate it in a half-hearted way.  Even if the platform has it, we wrap
   it because platforms differ in what vsnprintf does in case the buffer
   is too small:
It mentions that one corner cases what happens when the buffer is too small. Not rounding issues.

The "the best ways to go about it" comment links to the protobuf code, which also uses snprintf.

The (what I think is the) relevant C99 spec at http://www.open-std.org/jtc1/sc22/WG14/www/docs/n1256.pdf says:

> For e, E, f, F, g, and G conversions, if the number of significant decimal digits is at most DECIMAL_DIG, then the result should be correctly rounded. If the number of significant decimal digits is more than DECIMAL_DIG but the source value is exactly representable with DECIMAL_DIG digits, then the result should be an exact representation with trailing zeros. Otherwise, the source value is bounded by two adjacent decimal strings LSo either 1) "The C/C++ standards do not require formatting to round correctly or even be portable.", in which case Python and protobuf are doing it wrong and somehow this issue was never detected, or 2) The C/C++ standards do require correct rounding, but the case described by ChrisLomont didn't quite meet the spec requirements to get precision and rounding modes to match across platforms. Or 3), I don't know what I'm talking about.

Re: Python rounds float values by converting them to string and then back

#104

Post by jancsika »

A bit on topic...

Is there a phrase for the ratio between the frequency of an apparent archetype of a bug/feature and the real-world occurrences of said bug/feature? If not then perhaps the "Fudderson-Hypeman ratio" in honor of its namesakes.

For example, I'm sure every C programmer on here has their favored way to quickly demo what bugs may come from C's null-delimited strings. But even though C programmers are quick to cite that deficiency, I'd bet there's a greater occurrence of C string bugs in the wild. Thus we get a relatively low Fudderson-Hypeman ratio.

On the other hand: "0.1 + 0.2 != 0.3"? I'm just thinking back through the mailing list and issue tracker for a realtime DSP environment that uses single-precision floats exclusively as the numeric data type. My first approximation is that there are significantly more didactic quotes of that example than reports of problems due to the class of bugs that archetype represents.

Does anyone have some real-world data to trump my rank speculation? (Keep in mind that simply replying with more didactic examples will raise the Fudderson-Hypeman ratio.)

Re: Python rounds float values by converting them to string and then back

#105

Post by f00zz »

tehlike wrote: post #82
benj111 wrote: post #45

Earlier quoted context omitted.

Which raises the question what libc functions perl calls... And imagine the debug errors: >perl error X "But I'm just calling libc ?!?"

Or unintended stackoverflows.

in this case, unintended fork bombs

Re: Python rounds float values by converting them to string and then back

#106

Post by kzrdude »

yjftsjthsd-h wrote: post #10
ChrisSD wrote: post #5

Maybe I'm missing something but what's wrong with rounding floats this way?

I don't know that it is "wrong", just unexpected. I suspect most people expect all math functions to be purely implemented in numerical terms, so finding string manipulation is surprising/interesting.

And then strings are text implemented in numerical terms

Re: Python rounds float values by converting them to string and then back

#107

Post by chronolitus »

mark-r wrote: post #95
moring wrote: post #33

Earlier quoted context omitted.

Somewhat offtopic, but is there a reason some many explanations of this issue lump together the fundamental principle of how numbers are represented (integers vs. fractions vs. exact reals (technically impossible) vs. IEEE 754) and the base (decimal vs. binary)? Every time I read something like the explanation on that site, I wonder if I would understand it if I didn't knew it already.

The difference between decimal and binary is essential to understanding the problem. Just as there's no elegant way to represent 1/3 in base 10, there's no elegant way to represent 1/10 in base 2.

your comment led me to wonder why we commonly represent real numbers as floating-point in computing, and not, for example, as fractions.

https://retrocomputing.stackexchange.com/questions/7810/why-...

Re: Python rounds float values by converting them to string and then back

#108

Post by duckerude »

science404 wrote: post #69

Misleading title is misleading... CPython rounds float values by converting them to string and then back

PyPy does it too: https://bitbucket.org/pypy/pypy/src/2fc0a29748362f2a4b99ab57...

Jython instead uses BigDecimal::doubleValue: https://github.com/jythontools/jython/blob/b9ff520f4f6523120...

But as another comment noted, BigDecimal::doubleValue can pull a similar trick: https://news.ycombinator.com/item?id=20818586

Re: Python rounds float values by converting them to string and then back

#109

Post by Vindicis »

bhouston wrote: post #35

In my experience there are few things slower that float to string and string to float. And it seems so unnecessary. I always implemented round to a specific digit based on the built-in roundss/roundsd functions which are native x86-64 assembler instructions (i.e. https://www.felixcloutier.com/x86/roundsd ). I do not understand why this would not be preferable to the string method. float round( float x, int digits, in…

I don't think converting is slow by itself depending on what you need done. I have a function in my code I wrote to convert strings of floats/doubles to a rounded string of however many digits you want(for storing financial data that was muxed with multiple streams), and converting 1.59688139452f to a string, and then rounding that string to the 5th decimal place took 8.759 seconds for 10 million iterations (87.5 nanoseconds/iteration). Granted, it was written for my specific use case, so I don't need to handle the various edge cases/formats. But, little is inherently slow if you take the time to write a solution for your own need.
Post reply on HN