Bug #915: Please help • Hacker News Board

Re: Bug #915: Please help

#11

Post by DominoTree » Sun, Jan 12, 2020, 8:59 PM UTC

contingencies wrote: post #4

Had an I/O issue in SQLite yesterday: the cause was a full disk, due to years of SQLite backups!

The solution is clearly to restore a full system backup from a time that it had fewer SQLite backups :P

Re: Bug #915: Please help

#12

Post by pdq » Sun, Jan 12, 2020, 9:01 PM UTC

As mentioned on the Github issue [1], I'd suspect their SQLite optimizations:

    self.execute("pragma journal_mode=off").close()
    # This pragma makes writing faster.
    self.execute("pragma synchronous=off").close()

I've always used WAL journaling mode, and NORMAL synchronous mode.

[1] https://github.com/nedbat/coveragepy/issues/915

Re: Bug #915: Please help

#13

Post by nedbat » Sun, Jan 12, 2020, 9:04 PM UTC

pdq wrote: post #12

As mentioned on the Github issue [1], I'd suspect their SQLite optimizations: self.execute("pragma journal_mode=off").close() # This pragma makes writing faster. self.execute("pragma synchronous=off").close() I've always used WAL journaling mode, and NORMAL synchronous mode. [1] https://github.com/nedbat/coveragepy/issues/915

I have tried removing the two pragmas, and the problem still happens.

Re: Bug #915: Please help

#14

Post by nedbat » Sun, Jan 12, 2020, 9:05 PM UTC

TAForObvReasons wrote: post #3

OT: After the lamentation of the "Github Monoculture", why was development of coveragepy moved from bitbucket to github? https://bitbucket.org/ned/coveragepy/src/default/ original project https://nedbatchelder.com/blog/201405/github_monoculture.htm... original post https://news.ycombinator.com/item?id=7690897 HN comments

If the monoculture were easy to resist, it wouldn't need lamentation. :)

Re: Bug #915: Please help

#15

Post by blattimwind » Sun, Jan 12, 2020, 9:05 PM UTC

mmastrac wrote: post #2

There was a similar bug in unraid that they ended up fixing in 6.8.0 where it turned out that SQLite wasn't handling some sort of error condition in read-ahead I/O? I wonder if this is related. https://forums.unraid.net/bug-reports/prereleases/sqlite-dat... From that report: ====== 8 > In the Linux block layer each READ or WRITE can have various modifier bits set. In the case of a read-ahead you get READ|REQ_RAHEAD w…

SQLite doesn't use direct I/O.

Re: Bug #915: Please help

#16

Post by jakeogh » Sun, Jan 12, 2020, 9:19 PM UTC

TAForObvReasons wrote: post #3

OT: After the lamentation of the "Github Monoculture", why was development of coveragepy moved from bitbucket to github? https://bitbucket.org/ned/coveragepy/src/default/ original project https://nedbatchelder.com/blog/201405/github_monoculture.htm... original post https://news.ycombinator.com/item?id=7690897 HN comments

My blocker with bitbucket is it requires JS to use the web interface. Otherwise, blank page.

Re: Bug #915: Please help

#17

Post by slovenlyrobot » Sun, Jan 12, 2020, 9:23 PM UTC

nedbat wrote: post #13

pdq wrote: post #12

As mentioned on the Github issue [1], I'd suspect their SQLite optimizations: self.execute("pragma journal_mode=off").close() # This pragma makes writing faster. self.execute("pragma synchronous=off").close() I've always used WAL journaling mode, and NORMAL synchronous mode. [1] https://github.com/nedbat/coveragepy/issues/915

I have tried removing the two pragmas, and the problem still happens.

For some reason Docker is struggling to download the image right now (lots of interested parties? :), I just wanted to trap writes to the database file and look at what's actually getting written there.

Is there any possibility e.g. of an FD reuse issue in the code somewhere? It might even be possible to spot it with strace (although you need to start the container with the right caps to try that). An example where this can happen is programs closing 'stderr' fd 2, only for e.g. SQLite to open its database reusing fd 2, then some library code calls fprintf(stderr, ..) etc.

Would love a copy of the DB 'sometime before' and after to look at. There are tools around for inspecting SQLite databases on a page-by-page basis

Re: Bug #915: Please help

#18

Post by jp_sc » Sun, Jan 12, 2020, 9:28 PM UTC

x1798DE wrote: post #5

This is an editorialized title and as far as I know it's not even accurate. The author maintains, but did not create coverage.py. (Edit: Original title has been changed, so this comment is no longer necessary.)

Looking at the commit history, he is the author of almost all the code written in the last ten years. Even if there was someone before him (who was it? I couldn't find any reference to them), how could we not consider him the author of the project by now?

Re: Bug #915: Please help

#19

Post by lostmsu » Sun, Jan 12, 2020, 9:34 PM UTC

mmastrac wrote: post #9

lelf wrote: post #8

Earlier quoted context omitted.

-o >(grep O_DIRECT) This is cool. Never seen it (but it makes sense.)

Its cousin diff is also very handy.

Not a command line guru, but this looks interesting. Does it compare outputs of two commands? How does it do it?

Re: Bug #915: Please help

#20

Post by mjw1007 » Sun, Jan 12, 2020, 9:36 PM UTC

jp_sc wrote: post #18

x1798DE wrote: post #5

This is an editorialized title and as far as I know it's not even accurate. The author maintains, but did not create coverage.py. (Edit: Original title has been changed, so this comment is no longer necessary.)

Looking at the commit history, he is the author of almost all the code written in the last ten years. Even if there was someone before him (who was it? I couldn't find any reference to them), how could we not consider him the author of the project by now?

It was Gareth Rees.

https://garethrees.org/2001/12/04/python-coverage/