Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Unix’s file durability problem

utcc.utoronto.ca

Post reply on HN
Print view View original

131–140 of 161 posts

Re: Unix’s file durability problem

#131

Post by cm2187 »

Stupid question: why don't all computers (irrespective of the OS) have a built in power loss mechanism? It seems to be such a common and obvious problem.

1. The PSU would have a big enough capacitor to keep the computer running for a few seconds at its stated output power

2. The PSU would notify the OS of a power loss

3. The OS would immediately flush all caches and adopt a "brace position".

4. Events are spread system wide so that apps can also flush and brace.

It should work even if it is the PSU that fails (as long as the capacitor is there).

Surely the problem cannot be the cost. Why don't modern desktops have that feature?

Re: Unix’s file durability problem

#132

Post by illotum »

cm2187 wrote: post #131

Stupid question: why don't all computers (irrespective of the OS) have a built in power loss mechanism? It seems to be such a common and obvious problem. 1. The PSU would have a big enough capacitor to keep the computer running for a few seconds at its stated output power 2. The PSU would notify the OS of a power loss 3. The OS would immediately flush all caches and adopt a "brace position". 4. Events are spread syst…

I would guess because no promise is much better than a broken promise. No one wants to take blame.

Re: Unix’s file durability problem

#133

Post by rasz_pl »

cm2187 wrote: post #131

Stupid question: why don't all computers (irrespective of the OS) have a built in power loss mechanism? It seems to be such a common and obvious problem. 1. The PSU would have a big enough capacitor to keep the computer running for a few seconds at its stated output power 2. The PSU would notify the OS of a power loss 3. The OS would immediately flush all caches and adopt a "brace position". 4. Events are spread syst…

1 is not possible on anything with more than one >100MHz chip

Re: Unix’s file durability problem

#134

Post by cm2187 »

illotum wrote: post #132
cm2187 wrote: post #131

Stupid question: why don't all computers (irrespective of the OS) have a built in power loss mechanism? It seems to be such a common and obvious problem. 1. The PSU would have a big enough capacitor to keep the computer running for a few seconds at its stated output power 2. The PSU would notify the OS of a power loss 3. The OS would immediately flush all caches and adopt a "brace position". 4. Events are spread syst…

I would guess because no promise is much better than a broken promise. No one wants to take blame.

We have to accept that it wouldn't save the day in all scenarios. If there is a 60MB excel file to save on a slow network drive, the file will be lost. But this mechanism would still reduce file/disk/database corruptions by an order of magnitude.

Re: Unix’s file durability problem

#135

Post by cm2187 »

rasz_pl wrote: post #133
cm2187 wrote: post #131

Stupid question: why don't all computers (irrespective of the OS) have a built in power loss mechanism? It seems to be such a common and obvious problem. 1. The PSU would have a big enough capacitor to keep the computer running for a few seconds at its stated output power 2. The PSU would notify the OS of a power loss 3. The OS would immediately flush all caches and adopt a "brace position". 4. Events are spread syst…

1 is not possible on anything with more than one >100MHz chip

You mean there is no capacitor powerful enough?

Re: Unix’s file durability problem

#136

Post by drewm1980 »

cm2187 wrote: post #131

Stupid question: why don't all computers (irrespective of the OS) have a built in power loss mechanism? It seems to be such a common and obvious problem. 1. The PSU would have a big enough capacitor to keep the computer running for a few seconds at its stated output power 2. The PSU would notify the OS of a power loss 3. The OS would immediately flush all caches and adopt a "brace position". 4. Events are spread syst…

You have just described a UPS. Or a laptop. A battery just makes more sense than a capacitor in this application.

Re: Unix’s file durability problem

#137

Post by cm2187 »

drewm1980 wrote: post #136
cm2187 wrote: post #131

Stupid question: why don't all computers (irrespective of the OS) have a built in power loss mechanism? It seems to be such a common and obvious problem. 1. The PSU would have a big enough capacitor to keep the computer running for a few seconds at its stated output power 2. The PSU would notify the OS of a power loss 3. The OS would immediately flush all caches and adopt a "brace position". 4. Events are spread syst…

You have just described a UPS. Or a laptop. A battery just makes more sense than a capacitor in this application.

The idea is rather to have it built in. A retail consumer shouldn't have to worry about these things. And the idea would not be to sustain a long power interruption like a UPS, just to give time to flush caches and avoid corruption. Which should be much cheaper than a UPS.

Re: Unix’s file durability problem

#138

Post by memracom »

twa927 wrote: post #57
rwmj wrote: post #52

Earlier quoted context omitted.

I'm pretty sure any OS or API developed continuously over 25-40 years (depending how you count) would suffer from the same problems. Technology changes. Early assumptions prove to be wrong. Any OS developed from scratch now would suffer the same problem 25 years from now. It's worthwhile to use libraries which embody the knowledge, trials and errors of years, such as SQLite.

SQLite is an SQL database engine. Using it for storing a single value is like attacking a fly with a nuclear weapon. Unless you are talking about a function in the SQLite library that implements the durability calls properly and can be called from other programs.

If you do not want to use SQLITE to solve this problem then download the SQLITE source code and copy the way that they achieve these guarantees.

Stand on the shoulders of the giants who have come before us.

Re: Unix’s file durability problem

#139

Post by memracom »

schmichael wrote: post #82

Disks can buffer, disks can have firmware bugs, disks can fail both catastrophically and subtly. Even the fanciest battery-backed RAID controllers have firmware bugs and dead batteries. Those are just ways your storage hardware can fail you even if the kernel and libraries are bug free and you follow the right mystic incantations to sync data. While there's no excuse for bad documentation or poor APIs, you can never…

EBS, Riak, etc. Lots of easy solutions to this problem of putting the data in several diverse places with one write call.

Use what works and don't complain that broken stuff is broken.

Re: Unix’s file durability problem

#140

Post by brigade »

drewm1980 wrote: post #136
cm2187 wrote: post #131

Stupid question: why don't all computers (irrespective of the OS) have a built in power loss mechanism? It seems to be such a common and obvious problem. 1. The PSU would have a big enough capacitor to keep the computer running for a few seconds at its stated output power 2. The PSU would notify the OS of a power loss 3. The OS would immediately flush all caches and adopt a "brace position". 4. Events are spread syst…

You have just described a UPS. Or a laptop. A battery just makes more sense than a capacitor in this application.

Why? Wouldn't a large enough capacitor last longer than batteries? UPS batteries I've seen are only rated for 3 years, and half of my recent laptop batteries have physically swelled after 3 years of use.
Post reply on HN