Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Don't defer Close() on writable files (2017)

joeshaw.org

Post reply on HN
Print view View original

101–110 of 311 posts

Re: Don't defer Close() on writable files (2017)

#101

Post by red_admiral »

randomdata wrote: post #54
nickcw wrote: post #23

Here is my favorite solution to this problem // CheckClose is a utility function used to check the return from // Close in a defer statement. func CheckClose(c io.Closer, err *error) { cerr := c.Close() if *err == nil { *err = cerr } } Use like this - you must name the error return func whatever() (err error) { f, err := os.Open(blah) // ... defer CheckClose(f, &err) // ... } This closes the file and if there wasn't…

>>> f, err := os.Open(blah) // ... defer CheckClose(f, &err) What knowledge do you hope to gain of f.Close fails here?

If I understand the OP correctly, if Close() fails then you can't trust that the data was written, even if the previous Write() succeeded.

Re: Don't defer Close() on writable files (2017)

#103

Post by red_admiral »

masklinn wrote: post #73
magicalhippo wrote: post #60

Earlier quoted context omitted.

I've never heard about this in my years of programming. I just tried to read through the Win32 documentation, as I've done several times over the years, and it mentions a lot of edge cases but not this that I could see. Is this some Linux/Unix specific thing? Am I blind?

I am talking about posix semantics yes, I have no idea how things work on windows.

There's the old saying that on Windows*, files have names; in POSIX, names have files. I think that's what makes the difference here.

* technically it's the filesystem as much as the OS that is relevant here.

Re: Don't defer Close() on writable files (2017)

#104

Post by immibis »

ekimekim wrote: post #97
cyco130 wrote: post #26

Earlier quoted context omitted.

This problem isn't solved with exceptions either. The problem is that finalizers (C++ destructors, Java's `finally` blocks, Go's `defer` etc.) shouldn't fail but `close()` can fail. Therefore, for 100% correctness, `close()` calls should be handled explicitly and not left to finalizers. Finalizers shouldn't fail because they might be executed while another exception is already in flight. Three languages have three di…

Python handles this case by raising the new error but including a reference to the original error. By default, the formatted error shows both: >>> mylist = [] >>> try: ... first = mylist[0] ... finally: ... inverse_length = 1.0 / len(mylist) # imagine this was something more complex ... Traceback (most recent call last): File " ", line 2, in IndexError: list index out of range During handling of the above exception,…

Java also does this.

Re: Don't defer Close() on writable files (2017)

#105

Post by RenThraysk »

everforward wrote: post #82
dist-epoch wrote: post #13

Earlier quoted context omitted.

So if I do fopen/fwrite/fsync/fclose, that is not enough? That is crazy, I think 90% of apps don't fsync the parent directory. Also, how many levels of parents do you need to fsync?

That should be only for creating files, and maybe updating their metadata (not sure about that one). The confusion stems from people thinking that files and directories are more different than they are. Both are inodes, and both are basically containers for data. File inodes are containers for actual data, while directory inodes are containers for other inodes. All inodes need to be fsynced when you write to them. Fo…

In the case of atomic file writing, the rename() doesn't cause a fsync of the parent directory?

Re: Don't defer Close() on writable files (2017)

#107

Post by bryancoxwell »

randomdata wrote: post #93
Belphemur wrote: post #29

Earlier quoted context omitted.

I still question why defer doesn't support doing exactly that. After all it's like the go language provide us with a cleanup function that in 99% of the time shouldn't be used unless we manually wrap what it's calling to properly handle error. In the end, what's the point of defer ?

> I still question why defer doesn't support doing exactly that. When would it ever be useful? You'd soon start to hate life if you actually tried using the above function in anything beyond a toy application. > 99% of the time shouldn't be used 1. 99% of the time it is fine to use without further consideration. Even if there are errors, they don't matter. The example from the parent comment is a perfect case in poin…

The whole point of this post is that an error returned from file.Close DOES matter

Re: Don't defer Close() on writable files (2017)

#108

Post by lionkor »

jiggawatts wrote: post #21
dist-epoch wrote: post #13

Earlier quoted context omitted.

So if I do fopen/fwrite/fsync/fclose, that is not enough? That is crazy, I think 90% of apps don't fsync the parent directory. Also, how many levels of parents do you need to fsync?

Crazy is the right term. File system APIs in general have too many sharp edges and need a ground-up rethink. Consider S3-like protocols: these recognise that 99% of the time applications just want “create file with given contents” or “read back what they’ve previously written.” The edge cases should be off the beaten path, not in your way tripping you up to when you want the simple scenario.

It sounds more like you are asking for an abstraction

Re: Don't defer Close() on writable files (2017)

#110

Post by DarkNova6 »

thiht wrote: post #68
adrianmsmith wrote: post #4

Surely this would all go away if Go had an exception handling mechanism like most mainstream languages do? You'd just concentrate on the "happy path", you'd close the file, there'd be nothing to forget or write blog posts about because the exception would be propagated, without needing to write any lines of code.

Exceptions are a terrible error handling mechanism. You have no idea what throws and what doesn’t, it’s impossible to write defensive code that makes sense with exceptions. Errors as value is the only sane way to deal with errors. Granted, Go does it pretty badly but it’s still infinitely better than exceptions.

> You have no idea what throws and what doesn’t

You do in Java. It's called Checked Exceptions. It's a binding API contract and communicates this well not only when writing the code, but also when reviewing it.

Post reply on HN