Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Filesystem devs should aim to make “badly written” app code “just work” (2009)

lwn.net

Post reply on HN
Print view View original

111–117 of 117 posts

Re: Filesystem devs should aim to make “badly written” app code “just work” (2009)

#111

Post by silvestrov »

tobias3 wrote: post #106
silvestrov wrote: post #86

Earlier quoted context omitted.

> Unit files. ... you create a new file, which replaces the old one on close. Classic MacOS-9 had the PBExchangeFiles call which did this perfectly. Before call: dirEntryA -> fileContentsA dirEntryB -> fileContentsB after call dirEntryA -> fileContentsB dirEntryB -> fileContentsA This meant that the user kept all meta info for files, e.g. tags, window position, custom icons, etc. So when saving a new document you wro…

Like renameat2 on Linux with RENAME_EXCHANGE ?

No renameat2 moves the meta info (e.g. xattrs)as it’s an atomic “mv a temp; mv b a; mv tmp b”

So tags from a will end up on b. If b is a temp file, then the tags will be lost after the temp file is deleted.

Re: Filesystem devs should aim to make “badly written” app code “just work” (2009)

#112

Post by sargun »

Animats wrote: post #82
sargun wrote: post #80

Earlier quoted context omitted.

Nagle, I believe I last read your comment on this ( https://news.ycombinator.com/item?id=13964053 ), progress has been made . We have the temporary files you're asking for. https://lwn.net/Articles/619146/ With O_TMPFILE, you can also write new data, and then automatically replace a file on disk.

With O_TMPFILE, you can also write new data, and then automatically replace a file on disk. How? Using "linkat"? That's not automatic.

renameat2 is meant to be atomic. From the man page:

  If newpath already exists, it will be atomically replaced, so that
  there is no point at which another process attempting to access
  newpath will find it missing.  However, there will probably be a
  window in which both oldpath and newpath refer to the file being
  renamed.

Re: Filesystem devs should aim to make “badly written” app code “just work” (2009)

#113

Post by jlokier »

blt wrote: post #79
jrumbut wrote: post #51

Earlier quoted context omitted.

I think this is about the rare cases where you have multiple competing implementations that become better or worse depending on small changes elsewhere or when a simple initial implementation can serve as documentation for a carefully optimized subsequent implementation. I tend to think there is always a better way, say creating a module that contains both implementations as alternatives, but if this is the worst war…

I would go for a comment like "a simpler naive implementation is in commit A0348C. It ran approx. 2.5x slower in 2019."

I would do similar but include the commit timestamp and/or words from the subject to help locate the code if the commit id can't be retrieved.

Rebasing and other means of moving patches (email etc, adding signed-off-by) can cause the original commit id to become invalid, and eventually unretrievable.

Re: Filesystem devs should aim to make “badly written” app code “just work” (2009)

#114

Post by Animats »

sargun wrote: post #112
Animats wrote: post #82

Earlier quoted context omitted.

With O_TMPFILE, you can also write new data, and then automatically replace a file on disk. How? Using "linkat"? That's not automatic.

renameat2 is meant to be atomic. From the man page: If newpath already exists, it will be atomically replaced, so that there is no point at which another process attempting to access newpath will find it missing. However, there will probably be a window in which both oldpath and newpath refer to the file being renamed.

Yes, that's one of the many workarounds for not having unit files. There are different workarounds in the Windows world, and the Linux workround does not work on some file system types.

Re: Filesystem devs should aim to make “badly written” app code “just work” (2009)

#115

Post by Animats »

asveikau wrote: post #105
Animats wrote: post #59

I'd argue that UNIX-type file systems should offer several types of files: * Unit files. When you create a file and write it, it's not visible for other opens until you close it. If you open a file with O_CREAT|O_WRONLY|O_TRUNC, you create a new file, which replaces the old one on close. In the event of a program or system crash, or exiting via "abort" without closing first, the old file remains. So there's always on…

> Log files. I wonder if this would ideally also provide a rotation API. Rotating log files in the presence of multiple writers is messy, and maybe shouldn't be reimplemented at the application layer every time. > Temporary files Windows is not always a model of filesystem elegance, but it has a "delete on close" flag in their equivalent to open, which makes it go away on the last close (handle can still be duplicate…

I wonder if this would ideally also provide a rotation API. Rotating log files in the presence of multiple writers is messy, and maybe shouldn't be reimplemented at the application layer every time.

That feature was in UNIVAC EXEC8, now OS-2200. It was called F-cycles.[1] Over half a century old, and still working.

[1] https://public.support.unisys.com/2200/docs/CP18.0/PDF/78307...

Re: Filesystem devs should aim to make “badly written” app code “just work” (2009)

#116

Post by JetSpiegel »

skissane wrote: post #107
TeMPOraL wrote: post #94

Earlier quoted context omitted.

I would prefer it to not be the default; it's sometimes useful to keep temporary files in the event of a process crash, especially on a server that would then immediately restart said process. I had such a case very recently, and was thankful for the existing behavior of tmp files.

Rather than delete the file straight away, put it in a “trash can” or “recycle bin”. A background process deletes files from “trash can” at a later date. It could normally give them a grace period (e.g. 7 days, configurable) but the files could be deleted early if storage space is running low. The same feature could be used to provide undelete for non-temporary files too.

mktemp is enough for this.

Re: Filesystem devs should aim to make “badly written” app code “just work” (2009)

#117

Post by the8472 »

skissane wrote: post #73
Animats wrote: post #59

I'd argue that UNIX-type file systems should offer several types of files: * Unit files. When you create a file and write it, it's not visible for other opens until you close it. If you open a file with O_CREAT|O_WRONLY|O_TRUNC, you create a new file, which replaces the old one on close. In the event of a program or system crash, or exiting via "abort" without closing first, the old file remains. So there's always on…

> * Temporary files. You can do all the file operations, and the file disappears on a reboot. It would be nice to be able to have a process tree own a temporary file, such that when the last process in the tree exits (not necessarily the process which created the file), the file is automatically deleted, rather than having to wait for the next reboot.

If you use O_TMPFILE and pass file descriptors to child processes instead of paths this is possible. It's allocated on disk but never given a directory entry, so it gets deleted once the last process holding an fd terminates.
Post reply on HN