Earlier quoted context omitted.
-o >(grep O_DIRECT) This is cool. Never seen it (but it makes sense.)
Its cousin diff is also very handy.
It also has a great unofficial name: the penguin operator.
See? Penguin!
Bug #915: Please help
41–50 of 97 posts
Re: Bug #915: Please help
#42Earlier quoted context omitted.
Not a command line guru, but this looks interesting. Does it compare outputs of two commands? How does it do it?
It might be easiest to view it like so: $ echo The shell spawns two commands connected to pipes, then replaces those with a file that represents the other (read) side of that pipe. The command above with >(grep something) does the same thing, just with the other end of a pipe.
Does anybody know if this can be done in Windows (not WSL)? E.g. is it possible to open a named pipe (?) through its name, and work with it as a sequential file?
Re: Bug #915: Please help
#43Earlier quoted context omitted.
I have tried removing the two pragmas, and the problem still happens.
Neither of those make much sense as the cause anyway. Both sacrifice database consistency if the hosting process crashes or os crashes or reboots due to power failure, but with something like a coverage database, neither is particularly worrying. Worst case the users nukes the db, and reruns the tests. Even if removing those pragmas fixed things, it feels likely that would just be hiding the issue instead of fixing i…
Thanks, this was a good idea. I tried adding "pragma temp_store = 2", and it did not fix the problem, though I'm not sure I wasn't already using memory temp storage. The interaction of compile-time macros and pragmas is hard to decipher from the Python bindings.
Re: Bug #915: Please help
#44Earlier quoted context omitted.
It is a pipe, just for a different definition of pipe. | is the pipe operator in sh for doing a shell "pipeline". However, the parent comment is referring to a linux pipe as in pipe(7) [0]. One easy way to see this is with the following: $ ls -l 'pipe:[2937585]' As you can see, that command created a fd (16) which referred to a pipe (pipe:[2937585]). Those file descriptors were created using the pipe(2)[1] call by th…
On POSIX systems, the line between file and pipe doesn’t exist. If you create a named pipe w/ mknod or mkfifo, you would interact with the named pipe as you would a normal file. Or block/character device. Etc. In case of unnamed pipe the fds just exist as long as the process is running, then it disappears. Point is it’s irrelevant what you refer to as pipe, at the end of the day some file-like object is either writte…
The line absolutely exists, it's just behind an abstraction barrier for read(), write(), and close() calls. The creator of a pipe calls pipe() or pipe2(), and holders of the fd can get access to pipe-specific features with fcntl() and ioctl().
Re: Bug #915: Please help
#45Can someone suggest what the admins can turn this title into so it's not mystery meat (e.g. no idea what we're clicking through to, no idea whether the domain in question is ad-laden, etc).
Re: Bug #915: Please help
#46Can someone suggest what the admins can turn this title into so it's not mystery meat (e.g. no idea what we're clicking through to, no idea whether the domain in question is ad-laden, etc).
Help: Unresolved SQLite I/O Error in coverage.py
Re: Bug #915: Please help
#47Can we please have an informative title? Bug #915: Please help tells me nothing.
Whenever I come across a mailing list bug or a reddit post or similar with a "PLEASE HELP ME!" title I just auto-downvote/remove/delete it.
I refer to them as "help-me zombies". They have no brains and they want to eat yours. They don't know what they need help with, but they need help.
Re: Bug #915: Please help
#48This doesn't quite get all the way there, but the failure is being caused by the closure of a _io.FileIO due to GC, which happens to have the same fd as the sqlite database. The closure happens in the middle of a SQLite operation, which causes a subsequent flock() call to fail.
strace log from the failure:
open("/home/travis/apprise-api/.coverage", O_RDWR|O_CREAT|O_CLOEXEC, 0644) = 31
fstat(31, {st_mode=S_IFREG|0644, st_size=192512, ...}) = 0
fstat(31, {st_mode=S_IFREG|0644, st_size=192512, ...}) = 0
lseek(31, 0, SEEK_SET) = 0
read(31, "SQLite format 3\0\4\0\1\1\0@ \0\0\0\n\0\0\0\274"..., 100) = 100
write(3, "00001299 0001 6169 execute('prag"..., 54) = 54
write(3, "Executing 'pragma journal_mode=o"..., 36) = 36
write(3, "self: #0 internal_close (self=) at ./Modules/_io/fileio.c:126
#1 _io_FileIO_close_impl (self=) at ./Modules/_io/fileio.c:171
#2 _io_FileIO_close (self=, _unused_ignored=) at ./Modules/_io/clinic/fileio.c.h:23
#3 0x00007ffff7975be7 in _PyCFunction_FastCallDict (func_obj=func_obj@entry=0x7fffeefaee58, args=args@entry=0x7fffffff7590, nargs=0, kwargs=kwargs@entry=0x0) at Objects/methodobject.c:192
#4 0x00007ffff791b89e in _PyObject_FastCallDict (func=func@entry=0x7fffeefaee58, args=args@entry=0x7fffffff7590, nargs=, kwargs=kwargs@entry=0x0) at Objects/abstract.c:2313
#5 0x00007ffff791c8a4 in PyObject_CallMethodObjArgs (callable=0x7fffeefaee58, name=) at Objects/abstract.c:2759
#6 0x00007ffff7a976b4 in buffered_close (self=0x7fffee02ae08, args=) at ./Modules/_io/bufferedio.c:538
#7 0x00007ffff7975be7 in _PyCFunction_FastCallDict (func_obj=func_obj@entry=0x7fffedf7eab0, args=args@entry=0x7fffffff77c0, nargs=0, kwargs=kwargs@entry=0x0) at Objects/methodobject.c:192
#8 0x00007ffff791b89e in _PyObject_FastCallDict (func=func@entry=0x7fffedf7eab0, args=args@entry=0x7fffffff77c0, nargs=, kwargs=kwargs@entry=0x0) at Objects/abstract.c:2313
#9 0x00007ffff791c8a4 in PyObject_CallMethodObjArgs (callable=0x7fffedf7eab0, callable@entry=0x7fffee02ae08, name=) at Objects/abstract.c:2759
#10 0x00007ffff7a90def in iobase_finalize (self=0x7fffee02ae08) at ./Modules/_io/iobase.c:266
#11 0x00007ffff7a59e4f in finalize_garbage (collectable=0x7fffffff7950) at Modules/gcmodule.c:806
#12 collect (generation=generation@entry=1, n_collected=n_collected@entry=0x7fffffff79f0, n_uncollectable=n_uncollectable@entry=0x7fffffff79f8, nofail=nofail@entry=0) at Modules/gcmodule.c:1005
#13 0x00007ffff7a5a63b in collect_with_callback (generation=1) at Modules/gcmodule.c:1128
#14 0x00007ffff7a5ae8b in collect_generations () at Modules/gcmodule.c:1151
#15 _PyObject_GC_Alloc (basicsize=, use_calloc=0) at Modules/gcmodule.c:1729
#16 _PyObject_GC_Malloc (basicsize=) at Modules/gcmodule.c:1739
#17 0x00007ffff7a5afdd in _PyObject_GC_New (tp=tp@entry=0x7ffff7d53ea0 ) at Modules/gcmodule.c:1751
#18 0x00007ffff794766f in gen_new_with_qualname (qualname=0x7ffff4c1b450, name=0x7ffff4c218f0, f=0x7ffff292daf8, type=0x7ffff7d53ea0 ) at Objects/genobject.c:802
#19 PyGen_NewWithQualName (f=0x7ffff292daf8, name=0x7ffff4c218f0, qualname=0x7ffff4c1b450) at Objects/genobject.c:830
#20 0x00007ffff7a07f31 in _PyEval_EvalCodeWithName (_co=0x7ffff4c23270, globals=globals@entry=0x7ffff4cdd9d8, locals=locals@entry=0x0, args=, argcount=1, kwnames=kwnames@entry=0x0, kwargs=0x7ffff4c32930,
kwcount=0, kwstep=1, defs=0x0, defcount=0, kwdefs=0x0, closure=0x7fffede2cfd0, name=0x7ffff4c218f0, qualname=0x7ffff4c1b450) at Python/ceval.c:4150
#21 0x00007ffff7a081d7 in fast_function (kwnames=0x0, nargs=, stack=, func=0x7fffedc03f28) at Python/ceval.c:4978
#22 call_function (pp_stack=pp_stack@entry=0x7fffffff7c60, oparg=oparg@entry=1, kwnames=kwnames@entry=0x0) at Python/ceval.c:4858
#23 0x00007ffff7a0a633 in _PyEval_EvalFrameDefault (f=, throwflag=) at Python/ceval.c:3335
#0 0x00007ffff769b560 in __close_nocancel () from /lib/x86_64-linux-gnu/libpthread.so.0
#1 0x00007ffff3a23d8c in robust_close (lineno=29378, h=, pFile=0x113cfd8) at sqlite3.c:28631
#2 closeUnixFile (id=id@entry=0x113cfd8) at sqlite3.c:29378
#3 0x00007ffff3a24f43 in unixClose (id=0x113cfd8) at sqlite3.c:29427
#4 0x00007ffff3a4858b in sqlite3OsClose (pId=0x113cfd8) at sqlite3.c:17944
#5 sqlite3PagerClose (pPager=0x113ce68) at sqlite3.c:47961
#6 0x00007ffff3a5930b in sqlite3BtreeClose (p=0xfb8088) at sqlite3.c:58266
#7 0x00007ffff3a59501 in sqlite3LeaveMutexAndCloseZombie (db=0x1260598) at sqlite3.c:134102
#8 0x00007ffff3a59ae2 in sqlite3Close (db=0x1260598, forceZombie=0) at sqlite3.c:134045
#9 0x00007ffff3cc8fe2 in pysqlite_connection_close (self=0x7fffedf5fd50, args=) at /tmp/python-build.20181021062245.3423/Python-3.6.7/Modules/_sqlite/connection.c:337
#10 0x00007ffff7975be7 in _PyCFunction_FastCallDict (func_obj=0x7fffedf7eab0, args=0x7fffede2e890, nargs=0, kwargs=kwargs@entry=0x0) at Objects/methodobject.c:192
#11 0x00007ffff7975eb7 in _PyCFunction_FastCallKeywords (func=func@entry=0x7fffedf7eab0, stack=stack@entry=0x7fffede2e890, nargs=, kwnames=kwnames@entry=0x0) at Objects/methodobject.c:294
#12 0x00007ffff7a082c1 in call_function (pp_stack=pp_stack@entry=0x7fffffff9640, oparg=oparg@entry=0, kwnames=kwnames@entry=0x0) at Python/ceval.c:4837
with NamedTemporaryFile() as f:
# Write our content to disk
f.write(config.encode())
f.flush()
Re: Bug #915: Please help
#49The fix is described at .." rel="nofollow">https://github.com/nedbat/coveragepy/commit/e4b8389b27ce4e76.... it was simple, but elusive because of unsafe assumptions made in the name of performance.
"Put the filename calc back in _connect. Fixes #916
It was moved to __init__ to avoid recalculating, but the directory could have changed, so we need to wait to do the work. Instead, only do the relpath on systems that need it (Windows Py 2)."
Re: Bug #915: Please help
#50The fix was to undo a seemingly safe performance tweak...
From https://github.com/nedbat/coveragepy/commit/e4b8389b27ce4e76...
"Put the filename calc back in _connect. Fixes #916
It was moved to __init__ to avoid recalculating, but the directory could have changed, so we need to wait to do the work. Instead, only do the relpath on systems that need it (Windows Py 2)."