Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Fixing-JSON Conversation

tbray.org

Post reply on HN
Print view View original

41–50 of 55 posts

Re: The Fixing-JSON Conversation

#41

Post by wtbob »

I think the idea about removing whitespace is kinda hilarious, because that would mean one could write:

    ["a" "b" "c"]
or:

    {"a": "foo" "b": "bar" "c": "baz"}
The advantage over:

    (a b c)
or:

    (a foo b bar c baz)
or:

    ((a foo) (b bar) (c baz))
or:

    ((a . foo) (b . bar) (c . baz))
seems … non-existent.

Re: The Fixing-JSON Conversation

#43

Post by masklinn »

realharo wrote: post #21
coldtea wrote: post #19

Earlier quoted context omitted.

> that if you find yourself working with offsets and DST values, it's a sign that you're most likely doing something wrong. The video is wrong then. If you're writing an application that deals with times (e.g. stores and queries events with specific timestamps) and you don't take offsets and DST values into account, you get all kinds of weird edge cases.

No, the point is that you should not be doing those thing manually , i.e. you should never be adding integer offsets to something or doing similar operations. Instead, all the rules for conversions between times are already stored in the timezone database, so all you should do is something like ToAbsolute(CivilTime, TimeZone) and the reverse. He also mentions (at 26:50) proper ways of dealing with repeating and non-e…

> Instead, all the rules for conversions between times are already stored in the timezone database, so all you should do is something like

That only works if you're completely detached from the user and don't care for them. Example:

On January 1st 2011, a Samoan user living in Samoa (timezone Pacific/Apia) records an event for January 1st 2012. You convert January 1st 2012 09:00:00 to UTC, storing 2011-01-01T20:00:00.

On January 2nd 2012 at 10AM, you remind your user that they had an event set.

Because in May 2011 Samoa announced they were going to skip a local day and move across the international date line. So 2011-12-30T09:00:00 UTC was 2011-12-29T23:00:00 Pacific/Apia, but 2011-12-30T10:00:00 UTC was 2011-12-31T00:00:00 Pacific/Apia.

And as far as your user is concerned, they told you to ping them on January 1st at 9AM and you pinged a day late. Just because you store absolute datetimes doesn't mean you won't fuck up, and when the data is user-provided, chances are as good that that decision will be the one fuckup.

Re: The Fixing-JSON Conversation

#44

Post by fiatjaf »

gengkev wrote: post #37
fiatjaf wrote: post #35

“Just use X” · For val­ues of X in­clud­ing Hj­son, Ama­zon Ion, edn, Tran­sit, YAML, and TOML. ¶ Nah, most of them are way, way rich­er than JSON, of­ten with fully-worked-out type sys­tems and Con­cep­tu­al Tu­to­ri­als and so on. What? MOST OF THEM? YAML is not, Hjson is not, TOML is not.

YAML... really? Looking at the examples in the Wikipedia article ( https://en.wikipedia.org/wiki/YAML ) gives me a headache. Fortunately, most actual YAML files I've seen are not that complicated.

https://en.wikipedia.org/wiki/JSON#YAML_sample

Re: The Fixing-JSON Conversation

#45

Post by DonHopkins »

masklinn wrote: post #40
DonHopkins wrote: post #31

Earlier quoted context omitted.

If you're going to design something like that, it's wise to make sure it can be represented as valid JSON. Since JSON objects don't support order or repeated keys, that syntax can't be represented, edited or processed by the rich ecosystem of JSON tools. Most decent JSON editors will show that text with squigly red underlines. It's not worth giving up interoperability, and having to make yet another new set of tools…

> It would be great to have a Genshi-like templating language for JSON, tightly integrated with JavaScript the same way Genshi is integrated with Python. But… that already exists. It's called "Python". Just define your data structure using bog-standard Python and serialise it.

What I mean is using JSON as the syntax, and JavaScript as the expression and scripting language.

Zope has this supposedly "restricted subset" of Python that you're allowed to use as expressions and scripts, but it's missing important features and isn't meaningfully safe from a paranoid security perspective, and you end up having to drop down to lower level Zope external methods to write real Python code, which is very inconvenient.

If you trust someone enough to give them access to editing templates with "restricted python expressions", then you can probably trust them enough to use real Python expressions. You'd be unwise to give somebody you don't actually trust access to even a "restricted subset" of Python running on your server. That's just asking for trouble.

Re: The Fixing-JSON Conversation

#46

Post by pwfisher »

thaumasiotes wrote: post #12
thymelord wrote: post #3

Timestamps are so complicated once you factor in timezones and daylight savings that it doesn't belong in JSON. Time zones are not static. They can change from country to country, or even states within countries. Ditto for when daylight savings is enacted during the year - even changing over the years. There is no rhyme or reason to any of this. The data for this has to be stored in tables and time zone meanings can…

> (Speaking of leap seconds, who thought seconds going from 0 - 60 rather than 0 - 59 was a good idea?) Who thought February going from 1-29 instead of 1-28 was a good idea? I don't understand why everyone seems to believe that the phenomena are so inherently different.

Because other months went to 30 or 31 already, so there was room for it in calendars and people never assumed months are always the same length.

60 seconds in a minute, OTOH, is a very strong assumption in many systems and people's minds.

Re: The Fixing-JSON Conversation

#47

Post by realharo »

masklinn wrote: post #43
realharo wrote: post #21

Earlier quoted context omitted.

No, the point is that you should not be doing those thing manually , i.e. you should never be adding integer offsets to something or doing similar operations. Instead, all the rules for conversions between times are already stored in the timezone database, so all you should do is something like ToAbsolute(CivilTime, TimeZone) and the reverse. He also mentions (at 26:50) proper ways of dealing with repeating and non-e…

> Instead, all the rules for conversions between times are already stored in the timezone database, so all you should do is something like That only works if you're completely detached from the user and don't care for them. Example: On January 1st 2011, a Samoan user living in Samoa (timezone Pacific/Apia) records an event for January 1st 2012. You convert January 1st 2012 09:00:00 to UTC, storing 2011-01-01T20:00:00…

That still doesn't mean that you should do any of those operations by hand (which is what the previous comment was about).

But sure, if you do scheduling for future times, you do need to be aware of such possibilities and store the future times as civil times (and have some sane way of handling non-existing/repeating times - but again, most of the time the system/library will do this for you).

Then even if the user is flying across the world, they can still get the alert at the right time wherever they are (assuming the device updates local timezone based on location).

I never said that you should only store absolute times - only that they are a separate data type and you shouldn't mix them or try to convert them by hand.

Re: The Fixing-JSON Conversation

#48

Post by thymelord »

outsidetheparty wrote: post #34
niftich wrote: post #2

He summarized the most upvoted posts from the last thread [1] really well. [1] https://news.ycombinator.com/item?id=12328088 Regarding datetimes, it's worth pointing out the conversation that TOML had about it. It's a pretty long read [2][3][4][5] with lots of points raised for and against, but it also shows some of the process of how consensus was eventually forged: through trial-and-error, some enlightening realiza…

> He summarized the most upvoted posts from the last thread [1] really well. I feel like he glossed right past the objections to the biggest and (to my mind) most destructive proposed change, the commas-to-whitespace thing; in fact doubles down on it (let's just declare that commas are whitespace! That surely won't confuse anyone!)

I've written a few JSON parsers over the years that treat commas as whitespace. The grammar is simpler and the parser is faster as result. As long as one always emits standards-compliant JSON there's no problem.

Had the JSON standard supported ECMAScript array holes [1,,,2,,3] this grammar shortcut would not have been possible. But luckily that's not the case.

Re: The Fixing-JSON Conversation

#49

Post by thymelord »

espadrine wrote: post #22
thymelord wrote: post #3

Timestamps are so complicated once you factor in timezones and daylight savings that it doesn't belong in JSON. Time zones are not static. They can change from country to country, or even states within countries. Ditto for when daylight savings is enacted during the year - even changing over the years. There is no rhyme or reason to any of this. The data for this has to be stored in tables and time zone meanings can…

> The only reliable time stamp is UTC without leap seconds. That doesn't make a lot of sense, as UTC does have leap seconds. It is similar to saying that the fastest car has no wheels, when you really mean that the fastest vehicle is a rocket. TAI is the most reliable and easiest to work with. It relies on atomic clock seconds at sea level. https://en.wikipedia.org/wiki/International_Atomic_Time However, it already h…

> That doesn't make a lot of sense, as UTC does have leap seconds.

Pedantic much? Posting in technical forums is such a bother. People pick at any unimportant detail.

Reworded just for you:

The only reliable time stamp would be something like UTC but without leap seconds.

Re: The Fixing-JSON Conversation

#50

Post by andrewaylett »

realharo wrote: post #21
coldtea wrote: post #19

Earlier quoted context omitted.

> that if you find yourself working with offsets and DST values, it's a sign that you're most likely doing something wrong. The video is wrong then. If you're writing an application that deals with times (e.g. stores and queries events with specific timestamps) and you don't take offsets and DST values into account, you get all kinds of weird edge cases.

No, the point is that you should not be doing those thing manually , i.e. you should never be adding integer offsets to something or doing similar operations. Instead, all the rules for conversions between times are already stored in the timezone database, so all you should do is something like ToAbsolute(CivilTime, TimeZone) and the reverse. He also mentions (at 26:50) proper ways of dealing with repeating and non-e…

+1 serialise things without the processing complexities.

In general, an event is either scheduled in the future relative to a specific physical location, in which case you want to record it with a symbolic timezone reference (in case the TZ DB changes in the meantime), or it's timed absolutely, in which case you want to record it as UTC.

The trick is knowing which events are which. If you're scheduling something for a machine, probably just use UTC. If you're machine-recording the time of an event, even if it's based on a human pushing a button, use UTC and convert back to local time for display if necessary. If the user told you what time they wanted to store, store it as local time with a symbolic timezone.

Where it gets really difficult is interoperating with systems that don't do things properly, or working within systems that don't let you do things properly.

Post reply on HN