HFS+ is crazy • Hacker News Board

Re: HFS+ is crazy

#121

Post by mdip » Fri, Dec 04, 2015, 7:58 PM UTC

So, based on reading the comments (and I tend to agree), the issue isn't that HFS+ is bad because it supports Resource Forks, many file systems support similar concepts.

The question becomes why is OSX storing font data in the Resource Fork rather than what everything else understands is the actual content of the file itself? On other file systems that I'm familiar with, alternate streams are for storing metadata. There's important information there, for sure, but I'm not aware of other cases where the metadata stream stores the data in absence of the data being present in the part of the file that everything understands is the file's data. In this case, it appears OSX stores the font in this alternate stream. This seems like an error in design when coupled with the fact that other, normal applications, can't understand the file in a way that makes doing normal file operations on it possible (attaching it to an e-mail or sending it via Skype).

I'm sure there's a reason the content is stored there that my lack of OSX experience would explain. The part I have a problem with is that reading that file provides an inconsistent experience within the operating system. Finder can see the file. 'cp' not only sees the file, but when copying it, renders a copy with the contents in the data, not resource fork[1]. It smells like an API problem; the wrong method is being used to read the file by these other programs (like grabbing a pointer to a symbolic link instead of grabbing what it points to), but I don't write software for OSX, so does anyone know the specifics of why this design was chosen for fonts and other resources vs. storing the data in the actual part of the file that other programs would expect to to reside?

[1] I'm basing this statement only on the author's description of what happened. I do not own a Mac, myself.

Re: HFS+ is crazy

#122

Post by snowwrestler » Fri, Dec 04, 2015, 8:08 PM UTC

KirinDave wrote: post #41

gnu8 wrote: post #3

This is why it was really disappointing that Apple had to abandon ZFS. ZFS is one of the best file systems available and because Oracle owns it it is virtually unusable in most contexts. I wonder if Apple can afford to buy out Oracle at this point, take ZFS and wind up the useless parts.

I mean, there is http://open-zfs.org/wiki/Main_Page and FreeBSD has been proving the implementations reliability for some time now. Apple could adopt it. They don't. That's probably because the desktop OS is on life support, and there are indications Apple wants to abandon it in favor of a migration to a wholly iOS model.

And what file system does iOS use? I believe it is HFS+. So, if there was a compelling reason to switch from HFS+ to ZFS, it would apply equally to an iOS-only future at Apple. (Although I see no indications of such a future.)

I think it is more likely that:

a) Apple is working on their own new filesystem, optimized for their use. A company that invests in their own chip designs can certainly invest in their own filesystem.

OR

b) Apple feels that HFS+ is not actually holding them back, and plans to stick with it for the foreseeable future.

Re: HFS+ is crazy

#123

Post by sabujp » Fri, Dec 04, 2015, 8:10 PM UTC

HFS+ is "modern".

Re: HFS+ is crazy

#124

Post by chetanahuja » Fri, Dec 04, 2015, 8:18 PM UTC

rogerbinns wrote: post #20

Alternate parts to the main file are present on several platforms, and have the same problems. They are called resource forks on Mac. On Windows/NTFS they are called alternate data streams - https://en.wikipedia.org/wiki/Fork_(file_system) - to use specify a colon and name after the filename (eg example.txt:myads). On Unix, Linux, OS/2 etc you can find extended attributes - https://en.wikipedia.org/wiki/Extended_file…

Conflating extattrs with HFS style resource forks doesn't really make sense.

Extended attributes are just that... attributes. They are not the main "data" of the file. Each and every unix file system has some attributes to begin with (permissions, timestamps etc.) and they vary from filesystem to filesystem (e.g., setuid bit, immutable bit etc.). User supplied attributes just extends this concept in a natural way. All the standard tools expect to be able to call open() on the file and start reading a stream of bytes assuming there's sufficient access. That's what a "file" is.

Moreover, it's not just unix as an isolated systems. Those expectations are baked into the structure of the entire Internet. When you receiving a "file" over any medium like email, web etc., you're expecting to receive the aforementioned stream of bytes. The attributes (or extended attributes) are not expected to accompany the file data as a general case.

Resource forks on the other hand just completely work against reasonable user expectations. The example given in the OP's post is one such instance. A font file that shows up as having zero bytes to every tool that works with files including tools that expect to transfer "files" over the internet. It's just broken by design.

Re: HFS+ is crazy

#125

Post by makomk » Fri, Dec 04, 2015, 8:18 PM UTC

gnu8 wrote: post #6

vezzy-fnord wrote: post #5

Earlier quoted context omitted.

The utility of such a feature relative to its security, integrity and astonishment costs can be debated, however. ADS in NTFS has long been a subject of pain, for instance, though I'm unaware of its present status.

I think it is chiefly used by malware developers at this time.

To be fair, it's also used by DRM schemes and those aren't technically malware as such.

Re: HFS+ is crazy

#126

Post by fzn » Fri, Dec 04, 2015, 8:19 PM UTC

blakeyrat wrote: post #84

coldtea wrote: post #27

Earlier quoted context omitted.

> That said, I agree, it's a horrible design - but it's existed for 20+ years already. Actually there's absolutely nothing horrible about resource forks/extended attributes in theory. We use way worse ideas like "sidecar" files and metadata stored centrally for the same use cases, which are worse ways to handle the issue. The real problem is the lack of agreement/interoperability in handling them across FSs (and perh…

The problem is that *nix tools only support technology that existed in 1972. Anything newer than that is screwed. Mac Classic's death knell was the increasing popularity of the Internet. Run by servers that couldn't possibly store Mac Classic files correctly, because guess what? Resource forks/alternative data streams/whatever didn't exist back in 1972. And yes I am still bitter about this.

On one hand, it was easy to overlook this issue, and you'd end up with catastrophic data loss (been there, done that).

On the other hand:

1. it's the sort of mistake you tend to learn quickly not to redo ;(

2. a variety of encoding schemes were pretty common, and usually integrated in Mac browsers, mail clients, ftp clients. See for example this page from the Fetch website (http://fetchsoftworks.com/fetch/help/Contents/Concepts/Uploa...), or the hexbin(1) man page.

My personal take on what killed MacOS is this.

The MacOS was, for the start, a clever pile of kludges, for 68K series CPUs. Hot patching of routines was how you fixed bugs, introduced support for new hardware, etc. (See https://en.wikipedia.org/wiki/Macintosh_Toolbox#Advent_and_i...) (Btw, a System 7.1 source code archive leaked years ago.)

I can imagine debugging and extending it became more and more painful. And core data structure choices (16-bit-friendly) might have been becoming wasteful as architectures came and went...

So Apple had people grinding at Copland (apparently didn't quite make it), and then NeXT was bought, .

TL;DR: IMHO, that's not the main point (not either a notable one, once you're educated about it). MacOS grew to a point where it became a fragile, quite complex, house of cards.

Re: HFS+ is crazy

#127

Post by wsc981 » Fri, Dec 04, 2015, 8:21 PM UTC

zeveb wrote: post #47

Resource forks were pretty cool, actually: they were a simple database attached to every file, which could be used to store metadata or other information. It was pretty common for applications to store fonts, icons &c. all in their resource forks. Back when I was a kid I used to use ResEdit to change application & file icons, fiddle with GUI controls, change menu structures and so on. Happy days!

I used to edit levels with ResEdit for the Mac Bomberman clone BOOM. Good times :)

Re: HFS+ is crazy

#128

Post by Flow » Fri, Dec 04, 2015, 8:22 PM UTC

justizin wrote: post #103

I'm sorry, OP, but NTFS has resource forks as well, and they have value. The fonts are probably in resource forks because you are prohibited by copyright from copying or transferring them, but in other situations, they represent data that is only valuable to Mac OS, so that, say, if you rsync a bunch of files to a UNIX/Linux machine, you don't get things that are OSX-specific. That said, it is a fairly non-transparen…

FYI - Steam requires a case-insensitive filesystem. :-(

https://support.steampowered.com/kb_article.php?ref=8601-RYP...

Re: HFS+ is crazy

#129

Post by WorldMaker » Fri, Dec 04, 2015, 8:24 PM UTC

treve wrote: post #90

coldtea wrote: post #27

Earlier quoted context omitted.

> That said, I agree, it's a horrible design - but it's existed for 20+ years already. Actually there's absolutely nothing horrible about resource forks/extended attributes in theory. We use way worse ideas like "sidecar" files and metadata stored centrally for the same use cases, which are worse ways to handle the issue. The real problem is the lack of agreement/interoperability in handling them across FSs (and perh…

Who cares if it's a fine idea in theory, when in reality it's completely unpractical. What you call 'worse' is a model that works. Files showing the wrong size are a very small part of the problem. If people actively start storing critical data in resource forks (as it was done in the case of the font), a lot of other things will break. Just think of the humble HTML upload form or Git. Almost everywhere a file is con…

Well, in practice if more files had resource forks more tools would be smarter in how they operated on them, including making sure not to lose important forks.

> HTML upload form

The idea of a file doesn't change with resource forks: you upload the file as a whole and the thing uploads. Just as humble and simple. The onus is on the browser on the one side and server on the other not to drop the metadata.

The only other change might be in rare cases you might want to upload only a single fork in the file instead of the whole file. The old Mac Classic file picker had a way to this for advanced users, and there would be nothing stopping you from adding such an advanced option to any other file picker.

> or Git

Actually, something like git could probably make good use of something like resource forks if given the chance. For instance, the git pack format is essentially a relative to the resource fork format (a collection of a bunch of smaller objects wrapped into a single file). In a world of practical resource forks everywhere, you could presumably attach something like git packs of files to themselves, which could give you the benefit of a file being its own source control history without having to truck that information along as its own files/directories. (In which case you get the benefit of using that humble HTML upload form to much more easily upload a file and its entire source control history, rather than needing a dedicated git server... admittedly there would be performance trade-offs there though.)

Re: HFS+ is crazy

#130

Post by makomk » Fri, Dec 04, 2015, 8:28 PM UTC

blakeyrat wrote: post #84

coldtea wrote: post #27

Earlier quoted context omitted.

> That said, I agree, it's a horrible design - but it's existed for 20+ years already. Actually there's absolutely nothing horrible about resource forks/extended attributes in theory. We use way worse ideas like "sidecar" files and metadata stored centrally for the same use cases, which are worse ways to handle the issue. The real problem is the lack of agreement/interoperability in handling them across FSs (and perh…

The problem is that *nix tools only support technology that existed in 1972. Anything newer than that is screwed. Mac Classic's death knell was the increasing popularity of the Internet. Run by servers that couldn't possibly store Mac Classic files correctly, because guess what? Resource forks/alternative data streams/whatever didn't exist back in 1972. And yes I am still bitter about this.

No, the problem is that there were all kinds of interesting non-Unixy implementations of files out there and they were all incompatible with each other. For example, VMS supported file versioning and record-based files, neither of which Macs supported, but in turn had no way of supporting resource forks. Similarly, NTFS supports alternate data streams which can store a resource fork, but Mac Classic had no way of storing other alternate data streams. The Internet forced everyone to use Unix's idea of files because it's the only subset of functionality that actually worked between more than one platform.