Live data from Hacker News

Don't defer Close() on writable files (2017)

joeshaw.org

61–70 of 311 posts

Re: Don't defer Close() on writable files (2017)

#61
post #55
post #26

Earlier quoted context omitted.

This problem isn't solved with exceptions either. The problem is that finalizers (C++ destructors, Java's `finally` blocks, Go's `defer` etc.) shouldn't fail but `close()` can fail. Therefore, for 100% correctness, `close()` calls should be handled explicitly and not left to finalizers. Finalizers shouldn't fail because they might be executed while another exception is already in flight. Three languages have three di…

Java with try-with-resources does the correct thing: It attaches the new exception as a secondary exception to the currently in-flight exception. Since function calls form a tree, exceptions must form a tree as well. Doing this automatically is also one of the killer arguments for exceptions over error codes, IMO.

How automatic it is depends on the language, but error "objects" (rather than codes) can do this too. It is pretty great.

Re: Don't defer Close() on writable files (2017)

#62
post #23

Here is my favorite solution to this problem // CheckClose is a utility function used to check the return from // Close in a defer statement. func CheckClose(c io.Closer, err *error) { cerr := c.Close() if *err == nil { *err = cerr } } Use like this - you must name the error return func whatever() (err error) { f, err := os.Open(blah) // ... defer CheckClose(f, &err) // ... } This closes the file and if there wasn't…

I use a similar pattern, cribbed from one of the Go databases that aspired never to ignore errors. The difference is that errs.Capture preserves existing errors and formats the error with a message--important if generalized to handle any error function.

    package errs

    // Capture runs errFunc and assigns the error, if any, to *errPtr. Preserves the
    // original error by wrapping with errors.Join if the errFunc err is non-nil.
    func Capture(errPtr *error, errFunc func() error, msg string) {
        err := errFunc()
        if err == nil {
            return
        }
        *errPtr = errors.Join(*errPtr, fmt.Errorf("%s: %w", msg, err))
    }


I conventionally use mErr to distinguish from err.

    func doThing() (_ string, mErr error) {
        f, err := os.Open("foo")
        if err != nil {
            return "", fmt.Errorf("open file: %w", err)
        }
        // Use the file...
        defer errs.Capture(&mErr, f.Close, "close file")

        return "", nil
    }

Re: Don't defer Close() on writable files (2017)

#63
post #55

Earlier quoted context omitted.

Java with try-with-resources does the correct thing: It attaches the new exception as a secondary exception to the currently in-flight exception. Since function calls form a tree, exceptions must form a tree as well. Doing this automatically is also one of the killer arguments for exceptions over error codes, IMO.

How automatic it is depends on the language, but error "objects" (rather than codes) can do this too. It is pretty great.

Yes, with enough syntactic sugar it can become equivalent. Exceptions can be viewed as sum types with special syntactic sugar in conjunction with the regular return types, and can in principle be implemented as such.

When I say "exceptions", I mean the source-level semantics, not how it's implemented behind the scenes.

Re: Don't defer Close() on writable files (2017)

#64
post #22
post #10

Earlier quoted context omitted.

Explicit error handling is a choice and implicit error handling through exceptions is not necessarily a feature. Both have advantages and disadvantages, I’d say the more “modern” approach actually the opposite to what you state here, and is in my opinion the way to Go (pun int intended), though it’s also how Haskell does it. You’ll find the same philosophy in Rust, Zig, Swift and others which all build on the previou…

> Explicit error handling is a choice Yeah, it is - a bad choice IMO. I know the "if err != nil" pattern is spoken up as some sort of cultural idiosyncrasy of Go, similar to the whitespace formatting in python. But so far, I haven't seen any actual data (or even arguments) why it is superior to exceptions, or which inherent problems of exceptions it solves. (The classical example of "it makes control flow more obviou…

Sibling comments have good points but what made it click for me was the book "Exceptional C++" where all the possible sources of exceptions are pointed out -- it's usually twice as many as you would first guess by looking at the code, sometimes made more complicated by type-converting constructors and operator overloading and RAII.

Then consider that if any code you're calling is not exception-safe it makes the task of writing your code to be exception safe that much harder.

Then add on top of that the question of responsibility -- do you handle the exception close to where it gets thrown or farther up the call stack? Handling it too close to the issue may be missing some context and result in a lot of the same exception handling across many areas of the codebase. But handling it too far up misses context, too, and often leaves the program in an uncertain state where it's not clear whether the show must go on or if it's time to shut it down. It's not uncommon to see a mix of exceptions and returned error values to try and find a happy middle ground there.

Java tried to paper over some of these problems with fewer gotchas (more explicit separation between resource allocation and initialization) and compiler checks that exceptions are in the method signature and are always handled _somewhere_ up the call stack but this often results in handling of very vague exception types at a high level. Some developers, uncertain what the right way to handle an exception is, and not wanting to crash the program, will just silently drop the exception instead! Admittedly, you can do this in go-style and C-style error handling, too, but at least then it's not as far removed from the source of an error because it's annoying to have to keep passing an error response through so many function signatures.

I used to use exceptions a lot in C++ and Java. This changed when I started working at a place where a lot of C++ is used but without exceptions. It was ostensibly about runtime costs but when seniors were pressed on the issue it was clear there were a lot of these other reasons stated above, and ultimately about readability of the code (error handling being close to error source) and a philosophy of failing quickly when assertions fail (because keeping calm and carrying on leads to programs running that should have died before they could make more of a mess).

I know it's unpopular but, in light of some real problems with exceptions, I actually prefer the way Go makes you do it. It often encourages doing all the setup in one (or a few) function scopes, and it becomes pretty evident from a function's signature whether you should expect something to go wrong or not. Would I prefer it for a game-dev scripting language? no! But for a system language it is better IMHO

Re: Don't defer Close() on writable files (2017)

#65

Earlier quoted context omitted.

Go does indeed has an exception handling system like most other languages. Errors and exceptions are very different things, though. Of course, nothing stops you from building your own file handing package that overloads exception handlers to deal with errors. If it gains traction then it would prove the stdlib should consider a v2 API. But that you already haven’t done so is telling…

There's a self-selection problem though. People who prefer error handling through exceptions are just not going to use go. Period. Will not use it. Been there done that. No no no. So it is telling. But I think what it actually tells is that people would have done it just use another language instead.

Most of the other options don't include native code compilation though. I suppose Swift is there but doesn't make much sense outside of the Apple ecosystem.

Re: Don't defer Close() on writable files (2017)

#66
Here is a related question that has been on my mind for a while, but I have yet to find a good answer for:

If I write to file on a reasonably recent Linux and a sane file system like ext or zfs, at which point do I have the guarantee that when I read the same file back, that it is consistent and complete?

Do I really need to fsync or is Linux smart enough to give me back the buffer cache? Does it make a difference if reader and and writer are in the same thread or same process?

Re: Don't defer Close() on writable files (2017)

#67
post #39
post #23

Here is my favorite solution to this problem // CheckClose is a utility function used to check the return from // Close in a defer statement. func CheckClose(c io.Closer, err *error) { cerr := c.Close() if *err == nil { *err = cerr } } Use like this - you must name the error return func whatever() (err error) { f, err := os.Open(blah) // ... defer CheckClose(f, &err) // ... } This closes the file and if there wasn't…

Which of course isn't really a "solution", as it results in ambiguous error semantics; this is especially the case with Go's defer, as it pushes the code to the end of the entire function, not merely some intermediate relevant scope... the result is that a file used near the beginning of a function might fail to close but that error will not be realized until after something else in the function fails, after the poin…

> FWIW, Linus has suggested that the kernel should largely accept that application developers don't ever check the return value of close... but has also stated that developers should sync the file first if they care and also check close (at least, to be maximally correct).

Close just behaves very differently depending on the actual filesystem, too. Usually it’s very fast because it doesn’t do much of anything, but e.g. on NFS close will actually wait for writeback to the server to complete (due to close-to-open semantics).

Re: Don't defer Close() on writable files (2017)

#68

Surely this would all go away if Go had an exception handling mechanism like most mainstream languages do? You'd just concentrate on the "happy path", you'd close the file, there'd be nothing to forget or write blog posts about because the exception would be propagated, without needing to write any lines of code.

Exceptions are a terrible error handling mechanism. You have no idea what throws and what doesn’t, it’s impossible to write defensive code that makes sense with exceptions. Errors as value is the only sane way to deal with errors. Granted, Go does it pretty badly but it’s still infinitely better than exceptions.

Re: Don't defer Close() on writable files (2017)

#69

Earlier quoted context omitted.

Is it that the defer() that recovers is called before the defer() that closes (and potentially fails)?

No. No rearranging of the code would fix the problem.

Hmm. Checked the docs, looks like os.Create returns *File and err, but the sample doesn’t check that error.

Re: Don't defer Close() on writable files (2017)

#70

Here is a related question that has been on my mind for a while, but I have yet to find a good answer for: If I write to file on a reasonably recent Linux and a sane file system like ext or zfs, at which point do I have the guarantee that when I read the same file back, that it is consistent and complete? Do I really need to fsync or is Linux smart enough to give me back the buffer cache? Does it make a difference if…

Normally, immediately when write(2) returns.

It’s more complicated if the computer shut down in between, depending on how clean the shutdown was.

Post reply on HN