Live data from Hacker News

Bug #915: Please help

nedbatchelder.com

51–60 of 97 posts

Re: Bug #915: Please help

#51

The fix was to undo a seemingly safe performance tweak... From https://github.com/nedbat/coveragepy/commit/e4b8389b27ce4e76... "Put the filename calc back in _connect. Fixes #916 It was moved to __init__ to avoid recalculating, but the directory could have changed, so we need to wait to do the work. Instead, only do the relpath on systems that need it (Windows Py 2)."

This post is about issue #915 which seems unrelated/still open

Re: Bug #915: Please help

#52

This doesn't quite get all the way there, but the failure is being caused by the closure of a _io.FileIO due to GC, which happens to have the same fd as the sqlite database. The closure happens in the middle of a SQLite operation, which causes a subsequent flock() call to fail. strace log from the failure: open("/home/travis/apprise-api/.coverage", O_RDWR|O_CREAT|O_CLOEXEC, 0644) = 31 fstat(31, {st_mode=S_IFREG|0644,…

I suppose adding f.close() right after the flush might help.

Re: Bug #915: Please help

#53
post #19

Earlier quoted context omitted.

Not a command line guru, but this looks interesting. Does it compare outputs of two commands? How does it do it?

It is called process substitution and specific to Bash (not POSIX shell compliant). The output of the commands in parenthesis will become runtime (file) inputs for the diff command. More: https://www.gnu.org/software/bash/manual/html_node/Process-S...

It's not specific to bash -- it's in ksh as well. And zsh.

I'm also pretty sure it originated in ksh, not bash.

Re: Bug #915: Please help

#55

This doesn't quite get all the way there, but the failure is being caused by the closure of a _io.FileIO due to GC, which happens to have the same fd as the sqlite database. The closure happens in the middle of a SQLite operation, which causes a subsequent flock() call to fail. strace log from the failure: open("/home/travis/apprise-api/.coverage", O_RDWR|O_CREAT|O_CLOEXEC, 0644) = 31 fstat(31, {st_mode=S_IFREG|0644,…

Would https://www.python.org/dev/peps/pep-0446/ have anything to do with this?

We recently upgraded to celery 4 which had a side effect of file descriptors leaking between master and child processes, and database fds were being used for redis. Still no idea how it's happening, but the linked PEP seems to suggest there may be issues that I don't quite understand.

Re: Bug #915: Please help

#56

This doesn't quite get all the way there, but the failure is being caused by the closure of a _io.FileIO due to GC, which happens to have the same fd as the sqlite database. The closure happens in the middle of a SQLite operation, which causes a subsequent flock() call to fail. strace log from the failure: open("/home/travis/apprise-api/.coverage", O_RDWR|O_CREAT|O_CLOEXEC, 0644) = 31 fstat(31, {st_mode=S_IFREG|0644,…

Very curious about this wonderful magic. How were you able to get the call stack from the strace line?

Re: Bug #915: Please help

#57
post #52

This doesn't quite get all the way there, but the failure is being caused by the closure of a _io.FileIO due to GC, which happens to have the same fd as the sqlite database. The closure happens in the middle of a SQLite operation, which causes a subsequent flock() call to fail. strace log from the failure: open("/home/travis/apprise-api/.coverage", O_RDWR|O_CREAT|O_CLOEXEC, 0644) = 31 fstat(31, {st_mode=S_IFREG|0644,…

I suppose adding f.close() right after the flush might help.

The file is in a with block - it should be automatically closed at the end of the block.

Re: Bug #915: Please help

#58

Earlier quoted context omitted.

On linux at least, these two constructions are effectively identical from the view of the "piped" process. # cat /dev/pts/0 l-wx------ 1 root root 64 Jan 12 15:02 1 -> pipe:[20519634] lrwx------ 1 root root 64 Jan 12 15:02 2 -> /dev/pts/0 lr-x------ 1 root root 64 Jan 12 15:02 3 -> /proc/23611/fd/ # ls -l /proc/self/fd/ | cat - total 0 lrwx------ 1 root root 64 Jan 12 15:02 0 -> /dev/pts/0 l-wx------ 1 root root 64 J…

Yes, because “cat” can use either STDIN or a file as input: https://github.com/coreutils/coreutils/blob/master/src/cat.c... It’s an intentional design decision.

I think the point is that the directory listing is the same, not that cat supports both syntaxes.

Re: Bug #915: Please help

#59

Can we please have an informative title? Bug #915: Please help tells me nothing.

The original title was something like "Ned Batchelder, the author of coverage.py, was asking for help on Bug #915", but it was against the guidelines, so the title was restored.

Re: Bug #915: Please help

#60
post #52

Earlier quoted context omitted.

I suppose adding f.close() right after the flush might help.

The file is in a with block - it should be automatically closed at the end of the block.

Yes, but if it fixes the issue then there's something off with NamedTemporaryFile and its __exit__. You mentioned the file descriptor was being closed by a GC run, right?
Post reply on HN