Timestamps done right • Hacker News Board

Re: Timestamps done right

#41

Post by mollmerx » Thu, Jan 21, 2016, 6:28 PM UTC

smartmic wrote: post #26

I wonder if a closed source language for data processing tasks can thrive on the long run. The concept for timestamps is nice though. But I do not want to spend time of learning proprietary languages.

I think that's a very interesting question, to which the answer probably depends on how you define 'thrive'.

It seems unlikely to me that any language could become mainstream without being open source. The expectation that a compiler or interpreter should have its source available is only growing.

At the same time, kx's kdb+ is an example of a product that sits in a niche and has generated significant revenue despite being closed source. kdb+ has achieved this by primarily targeting financial services, which is a sector that's less sensitive to closed source than most, and is able to spend money on whichever product solves their problem.

If kerf manages to gain an edge over its competitors for a particular set of problems then sure, it can thrive in the long run the way kdb+ has.

Re: Timestamps done right

#42

Post by xixi77 » Thu, Jan 21, 2016, 6:29 PM UTC

stanmancan wrote: post #27

skywhopper wrote: post #13

Earlier quoted context omitted.

By my count it's over 30 characters less "clean". The Java syntax obscures some meaning and requires a lot more boilerplate in favor of less magical (and thus complex) syntax. I agree that this sort of first-class datetime-type representation may not be appropriate for every language, but myself, I find it refreshing and brilliant, and I'd love to see more languages support this sort of syntax instead of overloading…

How is plusMonths(1).plusDays(1) obscure in any way? You could show that to my grandparents and they would be able to guess what it does. "+1m1d" on the other hand they wouldn't have a clue. Length, in either direction, does not correlate to "clean". Clarity and intent does. Clean Coder ( http://www.amazon.ca/Clean-Code-Handbook-Software-Craftsmans... ) does a fantastic job talking about this, it's worth picking up i…

Length itself does not; but there is an obvious benefit to a syntax that is both short and clear -- when reviewing the code, it takes much less time to figure out what's happening.

Take matrix algebra for example: what takes less time to process (e.g. when debugging code) --

`a = b'*c+d` or

`a = matrixAdd(matrixMult(matrixTranspose(b),c),d)`

or perhaps

`a = b.inverse().times(c).add(d)`? What if there are hundreds of operations like that?

Now, there _might_ be people for whom syntax (3) is the clearest -- for example, these could be people who know some programming, but are not familiar with matrix data or operations. However, their convenience, or that of your grandparents, doesn't really matter: if it's a one-off job for them, figuring it out would take a very small fraction of their time and is not worth optimizing for, and if it's not, extra time to learn the syntax would more than pay for itself when they have to regularly work with it. We don't use words to describe such expressions in print for exactly the same reason! :)

Re: Timestamps done right

#43

Post by xixi77 » Thu, Jan 21, 2016, 6:40 PM UTC

32bitkid wrote: post #39

nickpsecurity wrote: post #37

Earlier quoted context omitted.

Why does that happen? I'd think they were equivalent. I don't do time-series or anything to be clear. I would just expect the a + b = b + a principle to apply for the m/d value.

because 1m = [28d,29d,30d,31d] depending on which month you are in. By adding 1d, pre or post, it can alter the definition of 1m.

Sure (assuming that's how a "month" is defined in this context -- I haven't actually read the whole article, so I'm not sure); but why do you find the Java syntax less confusing? They look pretty equivalent, except for one is shorter than the other.

Re: Timestamps done right

#44

Post by xixi77 » Thu, Jan 21, 2016, 6:50 PM UTC

ehartsuyker wrote: post #2

> I mean, just try it in Java. For ages JodaTime actually nailed it, and the Java 8 date API was based off this. > Not an add on type as in R or Python or Java. Again, let's talk about the modern version of the language and not act like prior screw ups are the end all for a language. Also > 2012.01.01 + 1m1d How is that more clean than: > new DateTime(2012, 1, 1).plusMonths(1).plusDays(1)

They are both about equally clear, but the former syntax will always take less time to process (unless perhaps you are a Java programmer who is familiar with the latter syntax but unfamiliar with the former, in which case it's not a valid comparison anyway). If it's an analysis-oriented package with most of the work done at a REPL, less time to type, too.

Re: Timestamps done right

#45

Post by henrikschroder » Thu, Jan 21, 2016, 7:03 PM UTC

xixi77 wrote: post #43

32bitkid wrote: post #39

Earlier quoted context omitted.

because 1m = [28d,29d,30d,31d] depending on which month you are in. By adding 1d, pre or post, it can alter the definition of 1m.

Sure (assuming that's how a "month" is defined in this context -- I haven't actually read the whole article, so I'm not sure); but why do you find the Java syntax less confusing? They look pretty equivalent, except for one is shorter than the other.

By splitting the addition up into two discrete calls, the ordering is made much more explicit, there's no room for ambiguity.

In C#, the largest part of a TimeSpan is a day, you can't have a TimeSpan of "one month and one day", because adding that to a DateTime would add a different amount of absolute time depending on the DateTime. You can have a TimeSpan of "32 days", and adding that to any DateTime yields a consistent result.

Re: Timestamps done right

#46

Post by henrikschroder » Thu, Jan 21, 2016, 7:03 PM UTC

TimWolla wrote: post #38

henrikschroder wrote: post #35

Earlier quoted context omitted.

2011-02-28 + 1m1d = 2011-03-29 2011-02-28 + 1d1m = 2011-04-01 (edit: stupid leap-year!)

February 2012 has got 29 days.

This is why everyone hates working with time and date functions. :-)

Re: Timestamps done right

#47

Post by jandrese » Thu, Jan 21, 2016, 7:08 PM UTC

ehartsuyker wrote: post #2

> I mean, just try it in Java. For ages JodaTime actually nailed it, and the Java 8 date API was based off this. > Not an add on type as in R or Python or Java. Again, let's talk about the modern version of the language and not act like prior screw ups are the end all for a language. Also > 2012.01.01 + 1m1d How is that more clean than: > new DateTime(2012, 1, 1).plusMonths(1).plusDays(1)

IMHO, working on timestamps in code like this is just asking for edge case to bite you in the ass.

The only sane way I've found to work with time is to convert any timestamp into a seconds-since-the-epoch value when doing any internal work and then covert back to the timestamp format for display. As an added bonus your code won't get super messy when you start getting timestamps from different sources that are formatted differently. Everything gets normalized to the internal representation before you do work on it.

Re: Timestamps done right

#48

Post by xixi77 » Thu, Jan 21, 2016, 7:09 PM UTC

mollmerx wrote: post #41

smartmic wrote: post #26

I wonder if a closed source language for data processing tasks can thrive on the long run. The concept for timestamps is nice though. But I do not want to spend time of learning proprietary languages.

I think that's a very interesting question, to which the answer probably depends on how you define 'thrive'. It seems unlikely to me that any language could become mainstream without being open source. The expectation that a compiler or interpreter should have its source available is only growing. At the same time, kx's kdb+ is an example of a product that sits in a niche and has generated significant revenue despite…

It could change in the future, but there are a plenty of languages that got pretty mainstream while closed source -- in terms of working with numbers and data, I'd definitely call matlab and sas mainstream, stata and mathematica are nearly mainstream, R of course is open source which is a huge reason for its popularity, but it's a clone of closed-source s+ which was fairly popular already. In the general-purpose world, c# and swift got most of their traction while still closed source, etc. Surely there are more examples in other domains... But yes, there is definitely a very welcome trend to open source stuff.

Re: Timestamps done right

#49

Post by philsnow » Thu, Jan 21, 2016, 7:10 PM UTC

When you build that syntax into the language / tokenizer, you're making a bunch of choices for the user.

As many have said elsewhere in the thread, what is `m` supposed to mean, minute or month? If the smallest value you can represent with this special syntax is 1s, you're excluding people who work with sub-second timestamps. Unless, you want them to do like 1m0.0000415s or something like that, but now you have the same problem as you started with (long series of digits being hard to read for humans).

More problems: different cultures have different concepts of a "month" (because they have different calendars). Also, the units that you want to use depend heavily on the application. `1y` is different from `52w` is different from `365d`, but all of those concepts are very similar to users. I expect people to create bugs where these tokens are mixed. When you write `now + 1y` do you mean "this time on this date, one year from now" or do you mean "this instant plus 606024*365 seconds" (which is slightly greater/less, I forget which way)? How do you communicate that to the user? If they can't control which sense ("human" time vs "physical" time) the larger units represent, then those people have to calculate the time offsets by hand.

Just pick nanos or micros since some Epoch (the Unix one is one reasonable choice), use them everywhere internally, and let the user pretty-print them. If you want to support high-energy timeseries where the time scale is smaller, either make your nanos fractional or use femtos/attos/zeptos/plancks.

Re: Timestamps done right

#50

Post by jandrese » Thu, Jan 21, 2016, 7:12 PM UTC

NyxWulf wrote: post #7

DiabloD3 wrote: post #3

Why are we not using unix timestamps (seconds since the epoch) for timestamps on everything that doesn't require sub-second precision?

They are difficult to read for humans. Without using a computer can you tell me when this timestamp was taken? 1453383978 Ultimately that was the point of the article, but the tone of the article about how they got it right and everyone else is wrong bugged me. My approach has been to store everything in epoch form, do all of my calculations and manipulations from there, then build tools that make converting back to…

Who cares how pretty the internal representation is? Only the processor sees that, you output pretty printed versions of the timestamp as you mentioned. You can even add the necessary bits to your debugger to automatically convert timestamps if necessary.

Dealing with sub-second precision isn't that hard. You just need either a second value that holds the microseconds/nanoseconds or 64 bit time that counts nanoseconds since the epoch instead of seconds.