Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What even is a JSON number?

blog.trl.sn

Post reply on HN
Print view View original

141–150 of 151 posts

Re: What even is a JSON number?

#141

Post by js2 »

jbboehr wrote: post #93
bterlson wrote: post #57

Earlier quoted context omitted.

I'm just a JS guy trying to understand the world around me and documenting what I find, not trying to be discourteous (or even courteous). I'll add the note about Python, thanks for calling it out. FWIW JS does not have a similar capability so I can't add a note there.

> FWIW JS does not have a similar capability so I can't add a note there This example on MDN seems to indicate that you can, am I misunderstanding it? const bigJSON = '{"gross_gdp": 12345678901234567890}'; const bigObj = JSON.parse(bigJSON, (key, value, context) => { if (key === "gross_gdp") { // Ignore the value because it has already lost precision return BigInt(context.source); } return value; }); [0]: https://dev…

> am I misunderstanding it?

The optional `context` parameter is a tc39 proposal. The feature compatibility matrix on the bottom of the MDN page is really confusing because it's showing only when `JSON.parse` was added, not whether the optional `context` parameter is supported.

I've confirmed it's available in:

  - V8 11.4.31
  - Node 20.12.0 (with `--harmony`)
  - Node 21.7.1 (without requiring `--harmony`)
  - Chrome 123.0.6312.107
But not available in:

  Firefox 124.0.2 (ironically) 
  Safari 17.3.1
The original blog post linked to the proposal:

https://tc39.es/proposal-json-parse-with-source/

https://github.com/tc39/proposal-json-parse-with-source

This issue links to the various browser engine tracking bugs:

https://github.com/tc39/proposal-json-parse-with-source/issu...

Which are:

• Chrome/V8: https://bugs.chromium.org/p/v8/issues/detail?id=12955

• Firefox/SpiderMonkey: https://bugzilla.mozilla.org/show_bug.cgi?id=1658310

• Safari/JavaScriptCore: https://bugs.webkit.org/show_bug.cgi?id=248031

Re: What even is a JSON number?

#142

Post by aleph_minus_one »

skrebbel wrote: post #98
justinpombrio wrote: post #85

Earlier quoted context omitted.

Isn't the lesson only that ids shouldn't be floats ? If they were integers everything would be fine, but JS numbers aren't integers, even if they look like them sometimes.

Nah, the lesson is broader than that, cause numbers as IDs have a whole bunch of problems and this is just one of them. Eg Twitter has incrementing number IDs and back when they had this whole ecosystem of 3rd party twitter apps (that they have since ruined), half the apps failed when the IDs became too large to fit into a 32-bit int. If it looks like a number, and it quacks like a number, sooner or later people are…

> If it looks like a number, and it quacks like a number, sooner or later people are going to treat it like a number.

Which is perfectly fine; just don't treat it like an int32. :-)

Re: What even is a JSON number?

#143

Post by akvadrako »

zerocrates wrote: post #137
yawaramin wrote: post #136

Earlier quoted context omitted.

You store it as an integer, but as we just saw in the OP, for general interop with any system that parses JSON you have to assume that it will be parsed as a double. So to avoid precision loss you are going to have to store it as a string anyway. At that point it's upto you whether you want to reinvent the wheel and implement all the required arithmetic operations for your new fixed-point type. Or you could just use…

In dollars, what do you get up to with a double of cents without precision loss? It's in the trillions, I figure? So a very large space of applications where smallest-denomination-as-JSON-number is going to be fine.

Prices can certainly have more decimals that cents.

If you just store cents you can't represent them. You either have to guess at the beginning the smallest unit or store the precision along with it.

Just use strings, it's much simpler.

Re: What even is a JSON number?

#144

Post by kbolino »

hinkley wrote: post #139
kbolino wrote: post #138

Earlier quoted context omitted.

With 10 times the memory usage and 100 times the compute power, maybe you could replace floats with something that behaves more like real numbers and covers mostly the same range. But the resulting type is still going to have its own limitations and sharp edges. Floats are not the right tool for every job but they are quite good at the jobs they are right for. Learning how they work is more useful than lamenting thei…

It’s about 2.5 times as much memory, if you do base 10. 65k is a little less than 5 bytes to represent 2. But floats are not the right representation for values that need to exactly match, like an ID, to be sure. If I’m off by half a cent it’s annoying. If I’m off by half a row I get nothing. The thing is that almost all of the problems we had in my initial story came from choosing system defaults. All except the PK…

With densely packed decimals (3 digits in 10 bits), you can reduce the space overhead to as little as 2.4% (1024/1000). The IEEE has even standardized various base-10 floating-point formats (e.g. decimal64). I'd suspect that with dedicated hardware, you could bring down the compute difference to 2-3x binary FP.

However I read the post I responded to as decrying all floating-point formats, regardless of base. That leaves only fixed-point (fancy integers) and rationals. To represent numbers with the same range as double precision, you'd need about 2048 bits for either alternative. And rational arithmetic is really slow due to heavy reliance on the GCD operation.

Re: What even is a JSON number?

#145

Post by js2 »

js2 wrote: post #141
jbboehr wrote: post #93

Earlier quoted context omitted.

> FWIW JS does not have a similar capability so I can't add a note there This example on MDN seems to indicate that you can, am I misunderstanding it? const bigJSON = '{"gross_gdp": 12345678901234567890}'; const bigObj = JSON.parse(bigJSON, (key, value, context) => { if (key === "gross_gdp") { // Ignore the value because it has already lost precision return BigInt(context.source); } return value; }); [0]: https://dev…

> am I misunderstanding it? The optional `context` parameter is a tc39 proposal. The feature compatibility matrix on the bottom of the MDN page is really confusing because it's showing only when `JSON.parse` was added, not whether the optional `context` parameter is supported. I've confirmed it's available in: - V8 11.4.31 - Node 20.12.0 (with `--harmony`) - Node 21.7.1 (without requiring `--harmony`) - Chrome 123.0.…

The MDN PR documenting the optional `context` parameter was merged just 2 weeks ago:

https://github.com/mdn/content/pull/32697/files

Re: What even is a JSON number?

#146

Post by eternityforest »

I like to think of floating point values as noisy analog voltages, with the extra propery that they can store small integers perfectly, and they can be copied within code but not round trip serialized and deserializer without noise.

They're not really noisy, but if an application would work with some random noise added, it will probably work with floats, and if it wouldn't work with noise added, it's probably easier to just not use floats and expext people to reason about IEEE details, while risking subtle bugs if different float representations get mixed.

Of course I'm not doing a lot of high performance algorithms, I would imagine in some applications you really do need to reason about floats.

Re: What even is a JSON number?

#147

Post by kccqzy »

tome wrote: post #140
kccqzy wrote: post #134

Earlier quoted context omitted.

Nope you didn't understand the situation correctly. First, almost nobody directly parses from a string to JSON AST: people almost always parse into a custom type using either Template Haskell or generics. Second, parsing isn't the issue; doing arithmetic on the number is the issue.

Surely the generics approach would go via an aeson Value as an intermediate format, and thus possibly store an unbounded Scientific.

Storing it isn't the problem.

Re: What even is a JSON number?

#148

Post by hinkley »

kbolino wrote: post #144
hinkley wrote: post #139

Earlier quoted context omitted.

It’s about 2.5 times as much memory, if you do base 10. 65k is a little less than 5 bytes to represent 2. But floats are not the right representation for values that need to exactly match, like an ID, to be sure. If I’m off by half a cent it’s annoying. If I’m off by half a row I get nothing. The thing is that almost all of the problems we had in my initial story came from choosing system defaults. All except the PK…

With densely packed decimals (3 digits in 10 bits), you can reduce the space overhead to as little as 2.4% (1024/1000). The IEEE has even standardized various base-10 floating-point formats (e.g. decimal64). I'd suspect that with dedicated hardware, you could bring down the compute difference to 2-3x binary FP. However I read the post I responded to as decrying all floating-point formats, regardless of base. That lea…

I was speaking in the context of JSON, where all numbers are decimal.

Re: What even is a JSON number?

#149

Post by kag0 »

yawaramin wrote: post #76

Long story short: don't use JSON numbers to represent money or monetary rates. Always use decimals encoded as string. It's surprising how many APIs fall short of this basic bar.

Yeah I disagree.

As the article said

> RFC 8259 raises the important point that ultimately implementations decide what a JSON number is.

Any implementation dealing with money or monetary rates should know that it needs to deal with precision and act accordingly. If you want to use JavaScript to work with money, you need to get a library that allows you to represent high precision numbers. It's not unreasonable to also expect that you get a JSON parsing library that supports the same.

oh, TIL that you can support large numbers with the default JavaScript JSON library https://developer.mozilla.org/en-US/docs/Web/JavaScript/Refe...

Re: What even is a JSON number?

#150

Post by Reubensson »

beryilma wrote: post #49
01HNNWZ0MV43FF wrote: post #40

Earlier quoted context omitted.

Makes sense for dollars, but for anything like graphics or physics I'd consider a power of two like 1,024 as the fixed-point factor instead. My intuition tells me that "x * 1000 / 1000 == x" might not be true for all numbers if you're using floats.

A sure sign of an inexperienced programmer in numerical computing is when they check for equality to zero of a floating-point number as if (x == 0) ... instead of something like if (abs(x) where eps is a suitably defined small number.

I would guess even most of time people using epsilon don't understand it. Its not like there is universal constant error with floating point numbers. I feel that saying just use epsilon is not much better than x == 0 and could be harder to find bugs if it sometimes works and othertimes does not.
Post reply on HN