Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Don't defer Close() on writable files (2017)

joeshaw.org

Post reply on HN
Print view View original

21–30 of 311 posts

Re: Don't defer Close() on writable files (2017)

#21

Post by jiggawatts »

dist-epoch wrote: post #13
masklinn wrote: post #7

There are more wrinkles with this: - if you are creating a file, to ensure full synchronisation you also need to fsync the parent directory, otherwise the file can be fsynced but the update to the directory lost - if sync fails, you can not assume anything about the file, whether on-disk or in memory , critically one understanding which got dubbed "fsyncgate" and lead to many RDBMS having to be updated is that you ca…

So if I do fopen/fwrite/fsync/fclose, that is not enough? That is crazy, I think 90% of apps don't fsync the parent directory. Also, how many levels of parents do you need to fsync?

Crazy is the right term. File system APIs in general have too many sharp edges and need a ground-up rethink.

Consider S3-like protocols: these recognise that 99% of the time applications just want “create file with given contents” or “read back what they’ve previously written.”

The edge cases should be off the beaten path, not in your way tripping you up to when you want the simple scenario.

Re: Don't defer Close() on writable files (2017)

#22

Post by xg15 »

devjab wrote: post #10
adrianmsmith wrote: post #4

Surely this would all go away if Go had an exception handling mechanism like most mainstream languages do? You'd just concentrate on the "happy path", you'd close the file, there'd be nothing to forget or write blog posts about because the exception would be propagated, without needing to write any lines of code.

Explicit error handling is a choice and implicit error handling through exceptions is not necessarily a feature. Both have advantages and disadvantages, I’d say the more “modern” approach actually the opposite to what you state here, and is in my opinion the way to Go (pun int intended), though it’s also how Haskell does it. You’ll find the same philosophy in Rust, Zig, Swift and others which all build on the previou…

> Explicit error handling is a choice

Yeah, it is - a bad choice IMO.

I know the "if err != nil" pattern is spoken up as some sort of cultural idiosyncrasy of Go, similar to the whitespace formatting in python. But so far, I haven't seen any actual data (or even arguments) why it is superior to exceptions, or which inherent problems of exceptions it solves. (The classical example of "it makes control flow more obvious and makes it easier to ensure that mandatory finalizers are not skipped" was just disproven by this very article)

So if there is more substantial criticism against exceptions than just FUD, I'd like to know it.

Re: Don't defer Close() on writable files (2017)

#23

Post by nickcw »

Here is my favorite solution to this problem

  // CheckClose is a utility function used to check the return from
  // Close in a defer statement.
  func CheckClose(c io.Closer, err *error) {
          cerr := c.Close()
          if *err == nil {
                  *err = cerr
          }
  }
Use like this - you must name the error return

  func whatever() (err error) {
      f, err := os.Open(blah)
      // ...
      defer CheckClose(f, &err)
      // ...
  }
This closes the file and if there wasn't an existing error, writes the error from f.Close() in there.

Re: Don't defer Close() on writable files (2017)

#24

Post by xg15 »

randomdata wrote: post #11
adrianmsmith wrote: post #4

Surely this would all go away if Go had an exception handling mechanism like most mainstream languages do? You'd just concentrate on the "happy path", you'd close the file, there'd be nothing to forget or write blog posts about because the exception would be propagated, without needing to write any lines of code.

Go does indeed has an exception handling system like most other languages. Errors and exceptions are very different things, though. Of course, nothing stops you from building your own file handing package that overloads exception handlers to deal with errors. If it gains traction then it would prove the stdlib should consider a v2 API. But that you already haven’t done so is telling…

"You can always make your own" is missing the point. Of course it's useless if a single library implements error handling differently than the rest of the language. The question is why the language does have this kind of manual error handling as a standard in the first kind.

> If it gains traction then it would prove the stdlib should consider a v2 API.

Some library that behaves completely differently from the rest of the language and breaks all interop with the rest of the ecosystem will have a hard time gaining traction, no matter if the way the library does it is objectively better or not.

Re: Don't defer Close() on writable files (2017)

#25

Post by pjmlp »

devjab wrote: post #10
adrianmsmith wrote: post #4

Surely this would all go away if Go had an exception handling mechanism like most mainstream languages do? You'd just concentrate on the "happy path", you'd close the file, there'd be nothing to forget or write blog posts about because the exception would be propagated, without needing to write any lines of code.

Explicit error handling is a choice and implicit error handling through exceptions is not necessarily a feature. Both have advantages and disadvantages, I’d say the more “modern” approach actually the opposite to what you state here, and is in my opinion the way to Go (pun int intended), though it’s also how Haskell does it. You’ll find the same philosophy in Rust, Zig, Swift and others which all build on the previou…

The modern approach of Rust and Swift is basically checked exceptions done with a different painting.

Re: Don't defer Close() on writable files (2017)

#26

Post by cyco130 »

adrianmsmith wrote: post #4

Surely this would all go away if Go had an exception handling mechanism like most mainstream languages do? You'd just concentrate on the "happy path", you'd close the file, there'd be nothing to forget or write blog posts about because the exception would be propagated, without needing to write any lines of code.

This problem isn't solved with exceptions either. The problem is that finalizers (C++ destructors, Java's `finally` blocks, Go's `defer` etc.) shouldn't fail but `close()` can fail. Therefore, for 100% correctness, `close()` calls should be handled explicitly and not left to finalizers.

Finalizers shouldn't fail because they might be executed while another exception is already in flight. Three languages have three different behaviors when that happens but in my opinion they all do the wrong thing:

In C++, if you throw from a destructor while another exception is in flight, the program will be terminated. In Java, throwing from a `finally` block will "forget" the original exception. In Go (according to this article, I'm not familiar with it), error from `defer` will be ignored. None of these are ideal.

Re: Don't defer Close() on writable files (2017)

#27

Post by randomdata »

xg15 wrote: post #22
devjab wrote: post #10

Earlier quoted context omitted.

Explicit error handling is a choice and implicit error handling through exceptions is not necessarily a feature. Both have advantages and disadvantages, I’d say the more “modern” approach actually the opposite to what you state here, and is in my opinion the way to Go (pun int intended), though it’s also how Haskell does it. You’ll find the same philosophy in Rust, Zig, Swift and others which all build on the previou…

> Explicit error handling is a choice Yeah, it is - a bad choice IMO. I know the "if err != nil" pattern is spoken up as some sort of cultural idiosyncrasy of Go, similar to the whitespace formatting in python. But so far, I haven't seen any actual data (or even arguments) why it is superior to exceptions, or which inherent problems of exceptions it solves. (The classical example of "it makes control flow more obviou…

Well, let's assume Go did commonly make use of exception handlers for error cases:

    defer func() {
         if r := recover(); r != nil {
             fmt.Println("Failed to write file", r)
         }
     }()
     f := os.Create("file")
     defer f.Close()
     io.WriteString(f, "Hello, World!")
Cool. You've solved one problem layer, perhaps. But if you look closely you'll notice that code still has a bug!

So clearly exception handlers aren't enough. There might be some good ideas in using exception handling, but if you strictly limit its use to errors you end up half-assing it. Why not go all the way and find a solution that works in all cases?

Re: Don't defer Close() on writable files (2017)

#28

Post by ghosty141 »

xg15 wrote: post #22
devjab wrote: post #10

Earlier quoted context omitted.

Explicit error handling is a choice and implicit error handling through exceptions is not necessarily a feature. Both have advantages and disadvantages, I’d say the more “modern” approach actually the opposite to what you state here, and is in my opinion the way to Go (pun int intended), though it’s also how Haskell does it. You’ll find the same philosophy in Rust, Zig, Swift and others which all build on the previou…

> Explicit error handling is a choice Yeah, it is - a bad choice IMO. I know the "if err != nil" pattern is spoken up as some sort of cultural idiosyncrasy of Go, similar to the whitespace formatting in python. But so far, I haven't seen any actual data (or even arguments) why it is superior to exceptions, or which inherent problems of exceptions it solves. (The classical example of "it makes control flow more obviou…

> I haven't seen any actual data (or even arguments) why it is superior to exceptions, or which inherent problems of exceptions it solves

It's faster. Doing all error handling via exception is not viable if you want speed.

Exceptions work well for errors in the sense that these rarely happen. But using them for general "this is the bad outcome" of an operation that can happen in the hot-path is problematic. For example pythons "next()" function which raises an exception if the end of the iterator is reached, if C++ did this it just wouldn't be used.

So in my opinion exceptions are nice if you want ease of use, and explicit error handling (look at rust with their question mark operation which makes it pretty easy to do) is the way to go for performance.

One advantage of explicit error handling like rust does it is: it forces you to handle the error path like other business logic. Again, if you write a short script, this is annoying and doesn't get you much but if the application is very important then doing so is a good thing since it forces you to think about where which errors can happen and how you handle them. With exceptions its very easy to completely forget that a can even happen and thus they get ignored and suddenly the program crashes with a unreadable error message.

Re: Don't defer Close() on writable files (2017)

#29

Post by Belphemur »

nickcw wrote: post #23

Here is my favorite solution to this problem // CheckClose is a utility function used to check the return from // Close in a defer statement. func CheckClose(c io.Closer, err *error) { cerr := c.Close() if *err == nil { *err = cerr } } Use like this - you must name the error return func whatever() (err error) { f, err := os.Open(blah) // ... defer CheckClose(f, &err) // ... } This closes the file and if there wasn't…

I still question why defer doesn't support doing exactly that.

After all it's like the go language provide us with a cleanup function that in 99% of the time shouldn't be used unless we manually wrap what it's calling to properly handle error.

In the end, what's the point of defer ?

Re: Don't defer Close() on writable files (2017)

#30

Post by randomdata »

xg15 wrote: post #24
randomdata wrote: post #11

Earlier quoted context omitted.

Go does indeed has an exception handling system like most other languages. Errors and exceptions are very different things, though. Of course, nothing stops you from building your own file handing package that overloads exception handlers to deal with errors. If it gains traction then it would prove the stdlib should consider a v2 API. But that you already haven’t done so is telling…

"You can always make your own" is missing the point. Of course it's useless if a single library implements error handling differently than the rest of the language. The question is why the language does have this kind of manual error handling as a standard in the first kind. > If it gains traction then it would prove the stdlib should consider a v2 API. Some library that behaves completely differently from the rest o…

> The question is why the language does have this kind of manual error handling as a standard in the first kind.

Probably for the same reason Rust does, and why it suffers much the same problem:

1. It is what was in vogue in the 2010s.

2. More importantly, the problem isn't limited to errors. What have you gained treating errors as some hyper special case when they aren't any different than any other value?

I think we agree that we can do better, but seeing errors as special doesn't get you there. We need something that understands the all-encompassing problem.

So, failing that understanding, if you're going to do something that sucks, you may as well choose the least-sucky option, surely? Exception handling brings a horrible developer experience. To the point that in languages where errors over exception handling semantics are the norm, you will find that most developers simply give up on error handling entirely because it is so painful.

> Some library that behaves completely differently from the rest of the language and breaks all interop with the rest of the ecosystem will have a hard time gaining traction

I'm not sure history agrees. Ruby was also of the return values over exception handling mind before Rails came along. Rails pushed exception handlers for errors and developers went for it. Provide an API people actually want to use, and they'll use it. What was common before is inconsequential.

I expect what you are really saying is that exception handling wouldn't actually improve this example case even in the best case, and in the worst case developers would end up giving up on error handling leaving such a package to be a net negative to a codebase.

Post reply on HN