Live data from Hacker News

Don't defer Close() on writable files (2017)

joeshaw.org

31–40 of 311 posts

Re: Don't defer Close() on writable files (2017)

#31

Surely this would all go away if Go had an exception handling mechanism like most mainstream languages do? You'd just concentrate on the "happy path", you'd close the file, there'd be nothing to forget or write blog posts about because the exception would be propagated, without needing to write any lines of code.

[dead]

Re: Don't defer Close() on writable files (2017)

#32

Surely this would all go away if Go had an exception handling mechanism like most mainstream languages do? You'd just concentrate on the "happy path", you'd close the file, there'd be nothing to forget or write blog posts about because the exception would be propagated, without needing to write any lines of code.

> Surely this would all go away if Go had an exception handling mechanism like most mainstream languages do?

Or monads, but that might be a step too far for the Go world considering the push-back against generics. When you have declarative error handling (a good thing imho) then monads really are the bees knees.

Re: Don't defer Close() on writable files (2017)

#33
post #22

Earlier quoted context omitted.

> Explicit error handling is a choice Yeah, it is - a bad choice IMO. I know the "if err != nil" pattern is spoken up as some sort of cultural idiosyncrasy of Go, similar to the whitespace formatting in python. But so far, I haven't seen any actual data (or even arguments) why it is superior to exceptions, or which inherent problems of exceptions it solves. (The classical example of "it makes control flow more obviou…

Well, let's assume Go did commonly make use of exception handlers for error cases: defer func() { if r := recover(); r != nil { fmt.Println("Failed to write file", r) } }() f := os.Create("file") defer f.Close() io.WriteString(f, "Hello, World!") Cool. You've solved one problem layer, perhaps. But if you look closely you'll notice that code still has a bug! So clearly exception handlers aren't enough. There might be…

Is it that the defer() that recovers is called before the defer() that closes (and potentially fails)?

Re: Don't defer Close() on writable files (2017)

#34

Earlier quoted context omitted.

Well, let's assume Go did commonly make use of exception handlers for error cases: defer func() { if r := recover(); r != nil { fmt.Println("Failed to write file", r) } }() f := os.Create("file") defer f.Close() io.WriteString(f, "Hello, World!") Cool. You've solved one problem layer, perhaps. But if you look closely you'll notice that code still has a bug! So clearly exception handlers aren't enough. There might be…

Is it that the defer() that recovers is called before the defer() that closes (and potentially fails)?

No. No rearranging of the code would fix the problem.

Re: Don't defer Close() on writable files (2017)

#35
post #8
post #3

Don't call close() twice if you're not using pidfds, this is racy. The fd could be reused in between the two calls. You'll risk closing random things a frustratingly small fraction of the time, creating very hard bugs for yourself.

Surely File.close will clear the fd on success...

Yeah but is it safe for concurrent use?

Re: Don't defer Close() on writable files (2017)

#36
post #22
post #10

Earlier quoted context omitted.

Explicit error handling is a choice and implicit error handling through exceptions is not necessarily a feature. Both have advantages and disadvantages, I’d say the more “modern” approach actually the opposite to what you state here, and is in my opinion the way to Go (pun int intended), though it’s also how Haskell does it. You’ll find the same philosophy in Rust, Zig, Swift and others which all build on the previou…

> Explicit error handling is a choice Yeah, it is - a bad choice IMO. I know the "if err != nil" pattern is spoken up as some sort of cultural idiosyncrasy of Go, similar to the whitespace formatting in python. But so far, I haven't seen any actual data (or even arguments) why it is superior to exceptions, or which inherent problems of exceptions it solves. (The classical example of "it makes control flow more obviou…

You interpret Go using a different pattern for error handling as a criticism of or challenge to exceptions. Consider that languages can take different approaches, without having to get into “best” or “worse.” Every approach to error handling has advantages and shortcomings.

The Go designers have explained and talked about this decision many times. Some people don’t like it and maybe choose a different language. No one pushes FUD regarding exceptions. I get the impression that if you choose a Toyota you would think that your neighbor buying a Honda poses a criticism you have to address.

Discussed before many times on HN, such as:

https://news.ycombinator.com/item?id=4159641

Original article:

https://commandcenter.blogspot.com/2012/06/less-is-exponenti...

Re: Don't defer Close() on writable files (2017)

#37
AFAIK Python/C# use a similar approach to Go - instead of `defer`, they have `using`/`with` statements. Go's `defer` seems more flexible though - it can execute custom code each time it's used, whereas `using`/`with` always call `__exit__`/`Dispose`.

How does the Python/C# approach compare to Go's in this situation? How are errors in `__exit__`/`Dispose` handled?

Re: Don't defer Close() on writable files (2017)

#38
post #7

There are more wrinkles with this: - if you are creating a file, to ensure full synchronisation you also need to fsync the parent directory, otherwise the file can be fsynced but the update to the directory lost - if sync fails, you can not assume anything about the file, whether on-disk or in memory , critically one understanding which got dubbed "fsyncgate" and lead to many RDBMS having to be updated is that you ca…

So if I do fopen/fwrite/fsync/fclose, that is not enough? That is crazy, I think 90% of apps don't fsync the parent directory. Also, how many levels of parents do you need to fsync?

One of the advantages of using a database for files is that it's relatively more likely that these platform-dependent considerations were indeed considered and you don't have to moonlight as an DBMS engineer while writing your application.

Re: Don't defer Close() on writable files (2017)

#39
post #23

Here is my favorite solution to this problem // CheckClose is a utility function used to check the return from // Close in a defer statement. func CheckClose(c io.Closer, err *error) { cerr := c.Close() if *err == nil { *err = cerr } } Use like this - you must name the error return func whatever() (err error) { f, err := os.Open(blah) // ... defer CheckClose(f, &err) // ... } This closes the file and if there wasn't…

Which of course isn't really a "solution", as it results in ambiguous error semantics; this is especially the case with Go's defer, as it pushes the code to the end of the entire function, not merely some intermediate relevant scope... the result is that a file used near the beginning of a function might fail to close but that error will not be realized until after something else in the function fails, after the point of no return on the close.

Really, this is kind of a fundamental limitation of automatic allocation semantics, which is effectively a monad that is being stacked with the error propagation monad in a confusing manner that means you "should" only defer operations which don't have errors.

Even in languages with exceptions, such as C++ and Java, they had to wrangle with this problem and failed to solve it: in C++ 11 or 17 or whatever, deconstructors are now by default nothrow in an attempt to prevent this kind of mistake.

...but then, what does one do with close?! In some sense, the entire concept of close must not fail, and yet it exists in a world where we don't really believe in anything that can't fail, as we like moving around failure semantics.

FWIW, Linus has suggested that the kernel should largely accept that application developers don't ever check the return value of close... but has also stated that developers should sync the file first if they care and also check close (at least, to be maximally correct).

But like, what does one do then? How do you ever recover from this? I actually think you can't, if you are in a cleanup operation... not without breaking your error regime. I think--and this is also where the article eventually goes in the updates (though without noting you can add your own boolean)--you should therefore both explicitly close (and/or maybe flush/sync) the file after writing to it and have a "if I didn't close this, close it" in your cleanup handler; critically, the (explicit) former checks for errors, while the (implicit) latter doesn't.

Re: Don't defer Close() on writable files (2017)

#40
post #26

Surely this would all go away if Go had an exception handling mechanism like most mainstream languages do? You'd just concentrate on the "happy path", you'd close the file, there'd be nothing to forget or write blog posts about because the exception would be propagated, without needing to write any lines of code.

This problem isn't solved with exceptions either. The problem is that finalizers (C++ destructors, Java's `finally` blocks, Go's `defer` etc.) shouldn't fail but `close()` can fail. Therefore, for 100% correctness, `close()` calls should be handled explicitly and not left to finalizers. Finalizers shouldn't fail because they might be executed while another exception is already in flight. Three languages have three di…

> in my opinion they all do the wrong thing

What would be the right thing? Combining the original exception and the error from `close` into some kind of `MultipleError`?

Post reply on HN