What even is a JSON number? • Hacker News Board

Re: What even is a JSON number?

#21

Post by bterlson » Mon, Apr 01, 2024, 9:25 PM UTC

frizlab wrote: post #11

It’s missing Swift tests, but otherwise it’s a great post.

If you would like to contribute Swift tests, I would be happy to take it! You can send a PR into this document, updating the data tables and adding a code sample at the end: https://github.com/bterlson/blog/blob/main/content/blog/what.... No need to test openapi-tools swift codegen unless you really want to!

Re: What even is a JSON number?

#22

Post by nedt » Mon, Apr 01, 2024, 9:26 PM UTC

whalesalad wrote: post #12

I tend to end up encoding everything as an integer (multiply by 1000, 10000 etc) and then turn it back into a float/decimal on decode. For instance if I am building a system dealing with dollar amounts I will store cent amounts everywhere, communicate cent amounts over the wire, etc. then treat it as a presentation concern to render it as a dollar amount.

So basically you use fixpoint numbers. Especially for currency that’s a very good idea anyway, because of rounding errors, even more so in IEEE 754

Re: What even is a JSON number?

#23

Post by enriquto » Mon, Apr 01, 2024, 9:26 PM UTC

msm_ wrote: post #15

erik_seaberg wrote: post #13

It's weird that any parser that loses digits is tolerated. A parser that forces strings into uppercase US-ASCII never would be.

That's true for every floating point number in every programming language you have ever used, though. $ python3 Python 3.10.13 (main, Aug 24 2023, 12:59:26) [GCC 12.2.0] on linux Type "help", "copyright", "credits" or "license" for more information. >>> 100000.000000000017 100000.00000000001

> That's true for every floating point number in every programming language you have ever used, though.

Alright, if "you" have only ever used python. In C, for example, we have hexadecimal floating point literals that represent all floats and doubles exactly (including infinities and nans that make the json parser fail miserably).

Re: What even is a JSON number?

#24

Post by cerved » Mon, Apr 01, 2024, 9:31 PM UTC

Izkata wrote: post #20

msm_ wrote: post #15

Earlier quoted context omitted.

That's true for every floating point number in every programming language you have ever used, though. $ python3 Python 3.10.13 (main, Aug 24 2023, 12:59:26) [GCC 12.2.0] on linux Type "help", "copyright", "credits" or "license" for more information. >>> 100000.000000000017 100000.00000000001

This is why Decimal exists: Python 3.8.10 (default, Nov 22 2023, 10:22:35) [GCC 9.4.0] on linux Type "help", "copyright", "credits" or "license" for more information. >>> from decimal import Decimal >>> Decimal('100000.000000000017') Decimal('100000.000000000017') For example: >>> import json >>> json.loads('{"a": 100000.000000000017}') {'a': 100000.00000000001} >>> json.loads('{"a": 100000.000000000017}', parse_floa…

but serializing/deserializing decimal using the json module is futile

Re: What even is a JSON number?

#25

Post by bterlson » Mon, Apr 01, 2024, 9:32 PM UTC

nedt wrote: post #22

whalesalad wrote: post #12

I tend to end up encoding everything as an integer (multiply by 1000, 10000 etc) and then turn it back into a float/decimal on decode. For instance if I am building a system dealing with dollar amounts I will store cent amounts everywhere, communicate cent amounts over the wire, etc. then treat it as a presentation concern to render it as a dollar amount.

So basically you use fixpoint numbers. Especially for currency that’s a very good idea anyway, because of rounding errors, even more so in IEEE 754

Pedantically, IEEE 754 defines decimal floating point formats (like decimal128) which are appropriate for representing currency. Representing currency in non-integer values in any of the binary floating point formats is indeed a recipe for disaster though.

Re: What even is a JSON number?

#26

Post by dhosek » Mon, Apr 01, 2024, 9:36 PM UTC

Izkata wrote: post #20

msm_ wrote: post #15

Earlier quoted context omitted.

That's true for every floating point number in every programming language you have ever used, though. $ python3 Python 3.10.13 (main, Aug 24 2023, 12:59:26) [GCC 12.2.0] on linux Type "help", "copyright", "credits" or "license" for more information. >>> 100000.000000000017 100000.00000000001

This is why Decimal exists: Python 3.8.10 (default, Nov 22 2023, 10:22:35) [GCC 9.4.0] on linux Type "help", "copyright", "credits" or "license" for more information. >>> from decimal import Decimal >>> Decimal('100000.000000000017') Decimal('100000.000000000017') For example: >>> import json >>> json.loads('{"a": 100000.000000000017}') {'a': 100000.00000000001} >>> json.loads('{"a": 100000.000000000017}', parse_floa…

And not every programming language offers a Decimal type and on most of those, there’s usually a performance penalty associated with it not to mention issues of interoperability and developer knowledge of its existence. For financial calculations, usually using integers with an implicit decimal offset (e.g., US currency amounts being expressed in cents rather than dollars), while other contexts will often determine that the inherent inaccuracy of IEEE floating types is a non-issue. The biggest potential problem lies in treating values that act kind of like numbers and look like numbers as numbers, e.g., Dewey Decimal classification numbers or the topic in a Library of Congress classification.¹

⸻

1. This is a bit on my mind lately as I discovered that LibraryThing’s sort by LoC classification seems to be broken so I exported my library (discovering that they export as ISO8859-1 with no option for UTF-8) and wrote a custom sorter for LOC classification codes for use in finally arranging the books on my shelves after my move last year.

Re: What even is a JSON number?

#27

Post by legobmw99 » Mon, Apr 01, 2024, 9:38 PM UTC

enriquto wrote: post #23

msm_ wrote: post #15

Earlier quoted context omitted.

That's true for every floating point number in every programming language you have ever used, though. $ python3 Python 3.10.13 (main, Aug 24 2023, 12:59:26) [GCC 12.2.0] on linux Type "help", "copyright", "credits" or "license" for more information. >>> 100000.000000000017 100000.00000000001

> That's true for every floating point number in every programming language you have ever used, though. Alright, if "you" have only ever used python. In C, for example, we have hexadecimal floating point literals that represent all floats and doubles exactly (including infinities and nans that make the json parser fail miserably).

If you use the same syntax as OP, C’s parser will also round that literal. The existence of a hex literal for floats is something orthogonal

Re: What even is a JSON number?

#28

Post by kccqzy » Mon, Apr 01, 2024, 9:46 PM UTC

I'll add that for Haskell, the library everyone uses for JSON parses numbers into Scientific types with almost unlimited size and precision. I say almost unlimited because they use a decimal coefficient-and-exponent representation where the exponent is a 64-bit integer.

The documentation is quite paranoid that if you are dealing with untrusted inputs, you could parse two JSON numbers from the untrusted source fine and then performing an addition on them could cause your memory to fill up. Exciting new DoS vector.

Of course in practice people end up parsing them into custom types with 64-bit integers, so this is only a problem if you are manipulating JSON directly which is very rare in Haskell.

Re: What even is a JSON number?

#29

Post by codetrotter » Mon, Apr 01, 2024, 9:48 PM UTC

enriquto wrote: post #23

msm_ wrote: post #15

Earlier quoted context omitted.

That's true for every floating point number in every programming language you have ever used, though. $ python3 Python 3.10.13 (main, Aug 24 2023, 12:59:26) [GCC 12.2.0] on linux Type "help", "copyright", "credits" or "license" for more information. >>> 100000.000000000017 100000.00000000001

> That's true for every floating point number in every programming language you have ever used, though. Alright, if "you" have only ever used python. In C, for example, we have hexadecimal floating point literals that represent all floats and doubles exactly (including infinities and nans that make the json parser fail miserably).

> we have hexadecimal floating point literals that represent all floats and doubles exactly

How do you do that?

A couple of resources I found but which I’m not sure if are about exactly what you speak of

https://stackoverflow.com/questions/65480947/is-ieee-754-rep...

https://gcc.gnu.org/onlinedocs/gcc/Hex-Floats.html

Furthermore, what exactly do you mean by “all floats and doubles exactly”?

Re: What even is a JSON number?

#30

Post by egwor » Mon, Apr 01, 2024, 9:51 PM UTC

I think the thing folk miss is when there’s an error like divide by zero, or the calculation would return NaN. I feel like this is the main gap/concern with using JSON and it seems to be rarely discussed.