Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Code that breaks 2 or 3 times every month

ngzhian.github.io

Post reply on HN
Print view View original

11–20 of 20 posts

Re: Code that breaks 2 or 3 times every month

#11

Post by dalke »

hkolk wrote: post #6
cnvogel wrote: post #3

Earlier quoted context omitted.

Sure enough. But is it a sane thing to do for a "convert readable date to date object" function to silently amend the current day-of-month to a otherwise underdpecified date? I really don't see the use case for that anywhere.

I would expect to use the first day of a month as a default, just like when time is not specified it should use 00:00 as default. Taking the current time/day as a default sounds off. Probably hard to fix backwards compatible though...

According to the documentation there's a 'default' parameter, which contains the datetime used to get otherwise unspecified fields. If not given, the default is the current date. Thus to get what you want:

    >>> from datetime import datetime
    >>> from dateutil import parser
    >>> now = datetime.now()
    >>> first = datetime(now.year, now.month, 1, 12, 0, 0)
    >>> dateutil.parser.parse('February 2006', default=first)
    datetime.datetime(2006, 2, 1, 12, 0)

Re: Code that breaks 2 or 3 times every month

#12

Post by [deleted] »

hkolk wrote: post #6
cnvogel wrote: post #3

Earlier quoted context omitted.

Sure enough. But is it a sane thing to do for a "convert readable date to date object" function to silently amend the current day-of-month to a otherwise underdpecified date? I really don't see the use case for that anywhere.

I would expect to use the first day of a month as a default, just like when time is not specified it should use 00:00 as default. Taking the current time/day as a default sounds off. Probably hard to fix backwards compatible though...

[deleted]

Re: Code that breaks 2 or 3 times every month

#13

Post by hobs »

xtrumanx wrote: post #9

I made an app that generated reports. The queries ran quickly when I tested it directly on the database but when I ran my app it would hang on a particular query. Sometimes. I started making some small seemingly insignificant changes to my code and suddenly my app generated all the reports quickly. All was well until the next month when I had to generate the same reports and my app failed on the same query. I solved…

I do a lot of minor performance tuning in mssql, and while I am not any senior professional, I often see parameter sniffing or a cached plan that makes no sense causing an issue when the server thinks its too expensive to create a new plan, and as a result the cardinality estimates are all wrong, the operators are often wrong, and you can see a query that often runs in insignificant time take forever.

As far as I know, the small changes you made just invalidated the plan (as the code wasnt the same) and make the server recompile your query's plan. You can accomplish the same thing by using sp_recompile

Depending on the code, I have fixed this with a few different answers: 1. You can write code that is clearer and simpler for the optimizer, sargeable queries and all that. 2. You can actually (I think in 2012 or later) issue a plan guide so that the query always runs in the way you decide. 3. You can do more hacky things like deciding to recompile the plan each time you run the query, or do a similar thing by assigning all your parameters to local variables, the query optimizer then cant parameter sniff.

You want to learn a little bit about parameter sniffing, because an execution plan that is poor can often be caused by re-using old values to plan the query,

Re: Code that breaks 2 or 3 times every month

#15

Post by ngzhian »

hkolk wrote: post #6
cnvogel wrote: post #3

Earlier quoted context omitted.

Sure enough. But is it a sane thing to do for a "convert readable date to date object" function to silently amend the current day-of-month to a otherwise underdpecified date? I really don't see the use case for that anywhere.

I would expect to use the first day of a month as a default, just like when time is not specified it should use 00:00 as default. Taking the current time/day as a default sounds off. Probably hard to fix backwards compatible though...

Exactly, not sure how much code has been written to take advantage of this fact. Taking day 1 is a much safer option definitely.

Re: Code that breaks 2 or 3 times every month

#16

Post by ngzhian »

xtrumanx wrote: post #9

I made an app that generated reports. The queries ran quickly when I tested it directly on the database but when I ran my app it would hang on a particular query. Sometimes. I started making some small seemingly insignificant changes to my code and suddenly my app generated all the reports quickly. All was well until the next month when I had to generate the same reports and my app failed on the same query. I solved…

Oh wow, that is a pretty bothersome trouble. I don't know nearly enough about databases and generating reports to understand this, but that link is saved. Thanks for sharing!

Re: Code that breaks 2 or 3 times every month

#17

Post by ngzhian »

rmetzler wrote: post #5

Once I recognized a certain test pass on all days except on Mondays . The test created models for a few days and tested for a sum of the current week. The dates where all relative (today, yesterday, etc.), so when the test run on a Monday some records where for the last week and that's why the sum wasn't the same. But I guess your problem is harder to debug.

Tests that relied on date (especially involving getting the current date) are usually hard to get right. I had an assignment in school to build a To-do list, and had lots of problems trying to test it. I think I solved it by having methods take 2 dates, initial and relative date, rather than just getting the current date.

Re: Code that breaks 2 or 3 times every month

#19

Post by rmetzler »

ngzhian wrote: post #17
rmetzler wrote: post #5

Once I recognized a certain test pass on all days except on Mondays . The test created models for a few days and tested for a sum of the current week. The dates where all relative (today, yesterday, etc.), so when the test run on a Monday some records where for the last week and that's why the sum wasn't the same. But I guess your problem is harder to debug.

Tests that relied on date (especially involving getting the current date) are usually hard to get right. I had an assignment in school to build a To-do list, and had lots of problems trying to test it. I think I solved it by having methods take 2 dates, initial and relative date, rather than just getting the current date.

Yes, a method is usually much easier to test, if the method gets the current date passed as an argument. The same goes for methods calculating on random values. Passing these to the method enables testing.
Post reply on HN