Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Don't defer Close() on writable files (2017)

joeshaw.org

Post reply on HN
Print view View original

51–60 of 311 posts

Re: Don't defer Close() on writable files (2017)

#51

Post by shawnz »

cyco130 wrote: post #26
adrianmsmith wrote: post #4

Surely this would all go away if Go had an exception handling mechanism like most mainstream languages do? You'd just concentrate on the "happy path", you'd close the file, there'd be nothing to forget or write blog posts about because the exception would be propagated, without needing to write any lines of code.

This problem isn't solved with exceptions either. The problem is that finalizers (C++ destructors, Java's `finally` blocks, Go's `defer` etc.) shouldn't fail but `close()` can fail. Therefore, for 100% correctness, `close()` calls should be handled explicitly and not left to finalizers. Finalizers shouldn't fail because they might be executed while another exception is already in flight. Three languages have three di…

The problem is handled with exceptions plus Java's try-with-resources or C#'s using statements or Python's context managers though, right?

Furthermore in Java since version 7 you can actually see both exceptions with the suppressed exceptions pattern.

Re: Don't defer Close() on writable files (2017)

#52

Post by cyco130 »

Almondsetat wrote: post #41
cyco130 wrote: post #26

Earlier quoted context omitted.

This problem isn't solved with exceptions either. The problem is that finalizers (C++ destructors, Java's `finally` blocks, Go's `defer` etc.) shouldn't fail but `close()` can fail. Therefore, for 100% correctness, `close()` calls should be handled explicitly and not left to finalizers. Finalizers shouldn't fail because they might be executed while another exception is already in flight. Three languages have three di…

What about Java's try-with-resources?

My Java knowledge is at least 10 years out of date so I'm not familiar with try-with-resources. But from what I can gather from Google, it looks like they now provide a way to access the suppressed exceptions, which is probably a step in the right direction.

Re: Don't defer Close() on writable files (2017)

#53

Post by EasyMark »

formerly_proven wrote: post #38
dist-epoch wrote: post #13

Earlier quoted context omitted.

So if I do fopen/fwrite/fsync/fclose, that is not enough? That is crazy, I think 90% of apps don't fsync the parent directory. Also, how many levels of parents do you need to fsync?

One of the advantages of using a database for files is that it's relatively more likely that these platform-dependent considerations were indeed considered and you don't have to moonlight as an DBMS engineer while writing your application.

One reason Iike use sqlite so much in my personal projects, sometimes just using it to store various config data bits within row entries of the most generic table ever, works well with small projects, probably not so much with large scaled ones.

Re: Don't defer Close() on writable files (2017)

#54

Post by randomdata »

nickcw wrote: post #23

Here is my favorite solution to this problem // CheckClose is a utility function used to check the return from // Close in a defer statement. func CheckClose(c io.Closer, err *error) { cerr := c.Close() if *err == nil { *err = cerr } } Use like this - you must name the error return func whatever() (err error) { f, err := os.Open(blah) // ... defer CheckClose(f, &err) // ... } This closes the file and if there wasn't…

>>>

      f, err := os.Open(blah)
      // ...
      defer CheckClose(f, &err)
What knowledge do you hope to gain of f.Close fails here?

Re: Don't defer Close() on writable files (2017)

#55

Post by layer8 »

cyco130 wrote: post #26
adrianmsmith wrote: post #4

Surely this would all go away if Go had an exception handling mechanism like most mainstream languages do? You'd just concentrate on the "happy path", you'd close the file, there'd be nothing to forget or write blog posts about because the exception would be propagated, without needing to write any lines of code.

This problem isn't solved with exceptions either. The problem is that finalizers (C++ destructors, Java's `finally` blocks, Go's `defer` etc.) shouldn't fail but `close()` can fail. Therefore, for 100% correctness, `close()` calls should be handled explicitly and not left to finalizers. Finalizers shouldn't fail because they might be executed while another exception is already in flight. Three languages have three di…

Java with try-with-resources does the correct thing: It attaches the new exception as a secondary exception to the currently in-flight exception.

Since function calls form a tree, exceptions must form a tree as well.

Doing this automatically is also one of the killer arguments for exceptions over error codes, IMO.

Re: Don't defer Close() on writable files (2017)

#56

Post by cyco130 »

shawnz wrote: post #51
cyco130 wrote: post #26

Earlier quoted context omitted.

This problem isn't solved with exceptions either. The problem is that finalizers (C++ destructors, Java's `finally` blocks, Go's `defer` etc.) shouldn't fail but `close()` can fail. Therefore, for 100% correctness, `close()` calls should be handled explicitly and not left to finalizers. Finalizers shouldn't fail because they might be executed while another exception is already in flight. Three languages have three di…

The problem is handled with exceptions plus Java's try-with-resources or C#'s using statements or Python's context managers though, right? Furthermore in Java since version 7 you can actually see both exceptions with the suppressed exceptions pattern.

See my reply to the sibling comment about try-with-resources. I'm not familiar with any of these mechanisms but giving access to suppressed exceptions is probably a step in the right direction.

Re: Don't defer Close() on writable files (2017)

#57

Post by randomdata »

rerdavies wrote: post #48
randomdata wrote: post #20

Earlier quoted context omitted.

Which languages see exception handlers work across go routines? Anyway, most don't. There is no difference from exception handlers in most other languages. The syntax is a little different. Is that where you've become confused?

Languages in which try/catch/throw work across coroutine boundaries: Java, Javascript, and it's a standard feature in C++ coroutine libraries (and completely supported by the core C++ coroutine engine). So in my limited personal experience, among the languages that I am familiar with ... all of them except go. Are there actually ANY languages other than go that have coroutines, and try/catch/throw mechanisms, where y…

> Languages in which try/catch/throw work across coroutine boundaries

You wha...? The question was about goroutines, not coroutines.

Besides, you'll notice that exception handlers cross coroutine boundaries in Go just fine. Your random tangent isn't even correct. Where did you dream up this idea to the contrary? I know coroutines are still new to Go, only officially landing in the latest release (experimentally in 1.22), but you'd think that would also mean their behaviour is fresh in your memory.

I'll take your avoidance of the original question to mean that no other language does it either.

Re: Don't defer Close() on writable files (2017)

#58

Post by layer8 »

dist-epoch wrote: post #13
masklinn wrote: post #7

There are more wrinkles with this: - if you are creating a file, to ensure full synchronisation you also need to fsync the parent directory, otherwise the file can be fsynced but the update to the directory lost - if sync fails, you can not assume anything about the file, whether on-disk or in memory , critically one understanding which got dubbed "fsyncgate" and lead to many RDBMS having to be updated is that you ca…

So if I do fopen/fwrite/fsync/fclose, that is not enough? That is crazy, I think 90% of apps don't fsync the parent directory. Also, how many levels of parents do you need to fsync?

Yes. File system implementations never really thought this through, and hence here we are.

Re: Don't defer Close() on writable files (2017)

#59

Post by randomdata »

rerdavies wrote: post #44
randomdata wrote: post #11

Earlier quoted context omitted.

Go does indeed has an exception handling system like most other languages. Errors and exceptions are very different things, though. Of course, nothing stops you from building your own file handing package that overloads exception handlers to deal with errors. If it gains traction then it would prove the stdlib should consider a v2 API. But that you already haven’t done so is telling…

There's a self-selection problem though. People who prefer error handling through exceptions are just not going to use go. Period. Will not use it. Been there done that. No no no. So it is telling. But I think what it actually tells is that people would have done it just use another language instead.

As they should. Scripting tasks are, indeed, best performed in scripting languages.

Re: Don't defer Close() on writable files (2017)

#60

Post by magicalhippo »

masklinn wrote: post #17
dist-epoch wrote: post #13

Earlier quoted context omitted.

So if I do fopen/fwrite/fsync/fclose, that is not enough? That is crazy, I think 90% of apps don't fsync the parent directory. Also, how many levels of parents do you need to fsync?

> So if I do fopen/fwrite/fsync/fclose, that is not enough? That is my understanding. > Also, how many levels of parents do you need to fsync? Only one, at least if you didn't create the parent directory (if you did then you might have to fsync its parent, recursively). The fsync on the parent directory ensures the dir entry for your new file is flushed to disk.

I've never heard about this in my years of programming. I just tried to read through the Win32 documentation, as I've done several times over the years, and it mentions a lot of edge cases but not this that I could see.

Is this some Linux/Unix specific thing? Am I blind?

Post reply on HN