Live data from Hacker News

Don't defer Close() on writable files (2017)

joeshaw.org

71–80 of 311 posts

Re: Don't defer Close() on writable files (2017)

#71

Earlier quoted context omitted.

No. No rearranging of the code would fix the problem.

Hmm. Checked the docs, looks like os.Create returns *File and err, but the sample doesn’t check that error.

os.Create, per the topic of discussion, is said to throw an error using the exception handling mechanism rather than return error. The exception handler checks the error.

If you focus on errors, you’re going to never get it. The whole idea here is that the problem is bigger than errors.

Re: Don't defer Close() on writable files (2017)

#72
post #22
post #10

Earlier quoted context omitted.

Explicit error handling is a choice and implicit error handling through exceptions is not necessarily a feature. Both have advantages and disadvantages, I’d say the more “modern” approach actually the opposite to what you state here, and is in my opinion the way to Go (pun int intended), though it’s also how Haskell does it. You’ll find the same philosophy in Rust, Zig, Swift and others which all build on the previou…

> Explicit error handling is a choice Yeah, it is - a bad choice IMO. I know the "if err != nil" pattern is spoken up as some sort of cultural idiosyncrasy of Go, similar to the whitespace formatting in python. But so far, I haven't seen any actual data (or even arguments) why it is superior to exceptions, or which inherent problems of exceptions it solves. (The classical example of "it makes control flow more obviou…

I prefer it because of location. The error handling happens where the error occur. It makes large systems much more maintainable in an enterprise setting.

It’s not like exception handling and throwing around things to have them caught later is inherently bad. It’s just a different philosophy, one that I don’t personally like anymore. It’s down the same alley as things like OOP, SOLID or DRY. Things which have good concepts that way too often leads to code bases which are incredibly annoying to work with. Maybe not for small systems with short life times, but for systems where you’re going to be the 100th person working on something that’s been running for 30 years it’s just nice to not have to play detective. I’d like to put a little disclaimer in here, because that isn’t inherently a consequence of exception handling or any of the other concepts but it’s just what happens when people work on code on a Thursday afternoon after a tough week. The simpler less abstract things are made, the easier it’ll be to unravel, and simple error handling is dealing with the errors exactly where they occur.

As others point out, it’s not without its disadvantages. It’s just that in my experience, those are better disadvantages than the disadvantages of implicit error handling.

Re: Don't defer Close() on writable files (2017)

#73

Earlier quoted context omitted.

> So if I do fopen/fwrite/fsync/fclose, that is not enough? That is my understanding. > Also, how many levels of parents do you need to fsync? Only one, at least if you didn't create the parent directory (if you did then you might have to fsync its parent, recursively). The fsync on the parent directory ensures the dir entry for your new file is flushed to disk.

I've never heard about this in my years of programming. I just tried to read through the Win32 documentation, as I've done several times over the years, and it mentions a lot of edge cases but not this that I could see. Is this some Linux/Unix specific thing? Am I blind?

I am talking about posix semantics yes, I have no idea how things work on windows.

Re: Don't defer Close() on writable files (2017)

#74
post #26

Surely this would all go away if Go had an exception handling mechanism like most mainstream languages do? You'd just concentrate on the "happy path", you'd close the file, there'd be nothing to forget or write blog posts about because the exception would be propagated, without needing to write any lines of code.

This problem isn't solved with exceptions either. The problem is that finalizers (C++ destructors, Java's `finally` blocks, Go's `defer` etc.) shouldn't fail but `close()` can fail. Therefore, for 100% correctness, `close()` calls should be handled explicitly and not left to finalizers. Finalizers shouldn't fail because they might be executed while another exception is already in flight. Three languages have three di…

It might fail, but usually doesn't. In C#, to handle an e.g. file open failure, you can just write

    try {
        using var file = File.OpenWrite("test.txt");
        file.Write("Hello, World!"u8);
    }
    catch (IOException e) {
        Console.WriteLine(e.Message);
    }
The exception handler is an enclosing scope for both file open and dispose (flush and close) operations. You can also hoist file variable to an outer scope to, for example, decide what to do if dispose throws for some reason. In practical terms, this is as much of an edge case as it gets, but can be expressed in a fairly straightforward way.

Re: Don't defer Close() on writable files (2017)

#75

Earlier quoted context omitted.

I've never heard about this in my years of programming. I just tried to read through the Win32 documentation, as I've done several times over the years, and it mentions a lot of edge cases but not this that I could see. Is this some Linux/Unix specific thing? Am I blind?

I am talking about posix semantics yes, I have no idea how things work on windows.

Phew. I've primarily used Windows. Not that any of the posix programs I've been exposed to have done the dir sync though.

For cross-platform stuff I've mainly used Boost, which I assumed handled such details.

Re: Don't defer Close() on writable files (2017)

#76
post #23

Here is my favorite solution to this problem // CheckClose is a utility function used to check the return from // Close in a defer statement. func CheckClose(c io.Closer, err *error) { cerr := c.Close() if *err == nil { *err = cerr } } Use like this - you must name the error return func whatever() (err error) { f, err := os.Open(blah) // ... defer CheckClose(f, &err) // ... } This closes the file and if there wasn't…

I caution against this approach, as you are not really dealing with the error when it occurs. If the work you do after the defer has other side effects, you may have just gotten your application into an inconsistent state and it's very hard to see in code why this might be.

`defer` is really not well-suited for error handling, its benefit is mainly in resource cleanup where failure is impossible or doesn't matter. (This makes it fine for `Close` on read-only file I/O operations, and not so great for writes.)

Re: Don't defer Close() on writable files (2017)

#77
post #26

Earlier quoted context omitted.

This problem isn't solved with exceptions either. The problem is that finalizers (C++ destructors, Java's `finally` blocks, Go's `defer` etc.) shouldn't fail but `close()` can fail. Therefore, for 100% correctness, `close()` calls should be handled explicitly and not left to finalizers. Finalizers shouldn't fail because they might be executed while another exception is already in flight. Three languages have three di…

It might fail, but usually doesn't. In C#, to handle an e.g. file open failure, you can just write try { using var file = File.OpenWrite("test.txt"); file.Write("Hello, World!"u8); } catch (IOException e) { Console.WriteLine(e.Message); } The exception handler is an enclosing scope for both file open and dispose (flush and close) operations. You can also hoist file variable to an outer scope to, for example, decide w…

I don't think this does the right thing. You are catching exception if Close fails and nothing else. The problem thought is what to do when after file is opened something fails first, and then Close also fails.

Re: Don't defer Close() on writable files (2017)

#78
post #77

Earlier quoted context omitted.

It might fail, but usually doesn't. In C#, to handle an e.g. file open failure, you can just write try { using var file = File.OpenWrite("test.txt"); file.Write("Hello, World!"u8); } catch (IOException e) { Console.WriteLine(e.Message); } The exception handler is an enclosing scope for both file open and dispose (flush and close) operations. You can also hoist file variable to an outer scope to, for example, decide w…

I don't think this does the right thing. You are catching exception if Close fails and nothing else. The problem thought is what to do when after file is opened something fails first, and then Close also fails.

I'm catching any exception that occurs within the scope of try block.

This includes an exception when trying to open or write to the file. If I fail to write to a file, it will try to dispose the stream, which flushes it and closes the file handle. If disposing the file handle itself fails, which should never happen, the exception will occur in the finally block, which this exception handler catches too. If you need to disambiguate and handle each case differently, which is rarely needed, you can order try-catch-finally blocks differently with explicit dispose and different nesting. This, again, is not a practical scenario and most user code will just `using file = File.OpenWrite` it and let the exception bubble up.

Re: Don't defer Close() on writable files (2017)

#79
post #6
post #2

Rust has the same problem. Files are closed in `Drop` when the value goes out of scope, but all errors are silently ignored. To solve this there's `sync_all`[0]. Generally, relying on defer in Go or Drop in Rust for anything that can fail seems like an anti-pattern to me. 0: https://doc.rust-lang.org/std/fs/struct.File.html#method.syn...

An ownership consuming close(self) would make sense, but has not been added, there must be some good reason for that?

IIRC the Rust book talks about files automatically being closed when dropped, and how that's better than having a close method. That's probably why it's not a separate method, even though it suppresses errors.

Re: Don't defer Close() on writable files (2017)

#80

Arguably, one should call `flush()` on the file first. Resource deallocation must always succeed; otherwise a lot of invariants break. This is why Zig's close method[0] ignores errors (with the exception of `EBADF`). [0]: https://github.com/ziglang/zig/blob/fb0028a0d7b43a2a5dd05f07...

Note that the "unreachable" there is equivalent to assert(error != EBADF), so really it's not even an exception, it's just helpful to crash there when debugging if you get that error. Important to understand that EBADF is not a catchable error because the kernel may have already reused that file descriptor for something else, in which case you wouldn't get EBADF, you would close an unrelated file descriptor.
Post reply on HN