Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Origins of DS_store (2006)

arno.org

Post reply on HN
Print view View original

241–250 of 280 posts

Re: The Origins of DS_store (2006)

#241

Post by skissane »

p_l wrote: post #239
skissane wrote: post #236

Earlier quoted context omitted.

> Essentially, attribute names directly specify the attribute type - so $SECURITY_DESCRIPTOR declared the entry in FILE attribute list to be a security descriptor. DATA attributes have another name field to handle multiple instances If you at the Linux kernel source code, `fs/ntfs3/ntfs.h` contains the following: struct ATTRIB { enum ATTR_TYPE type; // 0x00: The type of this attribute. __le32 size; // 0x04: The size…

Arguably from the point of On Disk Structure (to reuse terminology from NTFS' ancestor in VMS), all attributes can have names as well. Now, implementation in ntfs.sys is another thing and I have no idea if it's just an unused code path or if something would explode, and from what I heard Microsoft ended up in situation where people are scared to touch it not because of code quality but because of being scared of brea…

> Now, implementation in ntfs.sys is another thing and I have no idea if it's just an unused code path or if something would explode,

ntfs.sys has validation checks in it which prevent you from directly creating anything other than named or unnamed $DATA attributes on a regular file, and named $DATA attributes on a directory, and (indirectly) creating other stuff (directories, file names, standard attributes, EAs) through the appropriate APIs. If you try to do anything funky, you'll get an "Access Denied" error code returned by ntfs.sys

Re: The Origins of DS_store (2006)

#242

Post by saagarjha »

derefr wrote: post #222
klodolph wrote: post #116

Earlier quoted context omitted.

> One metadata and one the file contents. I’d say this is not the right way to describe a resource fork. Instead, think of it as two sets of file contents—one called "data" and one called "rsrc". On-disk, they are both just bytestreams. The catch is that you usually store a specific structure in the resource fork—smaller chunks of data indexed by 4-byte type codes and 2-byte integer IDs. Applications on the 68K norma…

From https://en.wikipedia.org/wiki/Resource_fork : > While the data fork allows random access to any offset within it, access to the resource fork works like extracting structured records from a database. So, whatever the on-disk structure, the motivation here is that from an OS API perspective , software (including the OS itself) can interact with files as one "seekable stream of bytes" (the data fork), and one "ran…

I am not old enough to know how resource forks were implemented on Mac OS but this is definitely not the case today. Resource forks are implemented (or maybe "emulated" is a better word to use? Not sure how much effort is put into them) as random-access. You can use POSIX APIs to interact with them (using _PATH_RSRCFORKSPEC) and these are typically faster than other interfaces.

Re: The Origins of DS_store (2006)

#243

Post by saagarjha »

blerb795 wrote: post #224
worstspotgain wrote: post #41

Earlier quoted context omitted.

A lot of Classic Mac apps just used the resource fork to store all their data. It was basically used as a Berkeley DB, except the keys were limited to a 32-bit OSType plus a 16-bit integer, and performance was horrible. But it got the job done when the files were small, had low on-disk overhead, and was ridiculously easy to deploy. Once you pushed an app beyond the level of usage the developer had performed in their…

> When Apple finally got rid of it Oh, they're not gone -- still very much part of APFS. You can read the contents of the resource fork for a file at path `$FILE` by reading `$FILE/..namedfork/rsrc` The resource fork is still how custom icons for files and directories are implemented! (Look for a hidden file called `Icon\r` inside any directory with a custom icon, and you can dump its resource fork to a `.icns` file…

They also back transparent filesystem compression

Re: The Origins of DS_store (2006)

#244

Post by p_l »

skissane wrote: post #241
p_l wrote: post #239

Earlier quoted context omitted.

Arguably from the point of On Disk Structure (to reuse terminology from NTFS' ancestor in VMS), all attributes can have names as well. Now, implementation in ntfs.sys is another thing and I have no idea if it's just an unused code path or if something would explode, and from what I heard Microsoft ended up in situation where people are scared to touch it not because of code quality but because of being scared of brea…

> Now, implementation in ntfs.sys is another thing and I have no idea if it's just an unused code path or if something would explode, ntfs.sys has validation checks in it which prevent you from directly creating anything other than named or unnamed $DATA attributes on a regular file, and named $DATA attributes on a directory, and (indirectly) creating other stuff (directories, file names, standard attributes, EAs) th…

I was thinking more of "ntfs.sys encounters filesystem structure with names set for normally unnamed attributes".

The API preventing arbitrary messing up is a separate (and good and valid) concern.

Re: The Origins of DS_store (2006)

#245

Post by saagarjha »

sneed_chucker wrote: post #137
gregmac wrote: post #119

As a non-Mac user, I always find it somewhat annoying when I download some .tgz published on Github or something and find .DS_Store littered inside. I guess macos probably just uses GNU tar? It's kind of surprising it wasn't modified or configured by default to ignore .DS_Store.

Most of Mac's Unix utils come straight from FreeBSD without any special sauce from Apple.

Apple patches most of their utilities. This is frequently annoying when trying to track down what they did :)

Re: The Origins of DS_store (2006)

#246

Post by p_l »

skissane wrote: post #240
p_l wrote: post #238

Earlier quoted context omitted.

EaData & EaFile remind me of the murky memories of OS/2 APIs. HPFS had a different approach of internally handling EAs, but OS/2 did create extra file on FAT16 filesystems to store EAs, which could point to origin of $EA. (HPFS itself has special EA-handling implemented in its FNODE, equivalent of inode/FILE entry) I do not recall the EA actually being used anywhere by new code though, quite shocked by the mention of…

> I do not recall the EA actually being used anywhere by new code though, quite shocked by the mention of WSL. This explains it: https://learn.microsoft.com/en-au/archive/blogs/wsl/wsl-file... uid, gid, mode, and POSIX format timestamps are stored in an EA. It also mentions file capabilities being stored in an ADS. On Linux, capabilities and ACLs are stored in xattrs, so that seems to imply that xattrs are stored in…

the dollar sign convention predates NT, it's one of the things inherited from Files-11, where the metadata-files were not hidden from end user, just marked with strict enough permission checks. (A lot of VMS APIs used dollar signs for namespacing, too, and I believe some aspects of the naming scheme come from specific PDP assemblers when referring to some names?)

Looking at NTFS from on-disk structure side, it always seemed quite obvious to me that a lot of accolades given to BeFS applied to NTFS - it's the lack of actually using the abilities - and IIRC a lot of the indexing system is actually used by Windows Search, which in tech spaces I always found mentioned as "useless thing I disabled", yet I found out later offices where people are very much dependant on the component (helps that MS Office installed document handlers to index its documents in it)

Re: The Origins of DS_store (2006)

#248

Post by k4rli »

demondemidi wrote: post #231

Funny all the people complaining about “this makes Mac ugly compared to Linux” meanwhile every Linux tool I install craps dotfiles not only in ~/ but in working folders as well. It’s a fact of life that tens of thousands of smart programmers realized is necessary.

Every Linux tool? Simply not true. Most tools keep it in a ~ subdir like ~/.config

MacOS creates a junk file/folder just by visiting any folder. It's not comparable.

Re: The Origins of DS_store (2006)

#249

Post by skissane »

p_l wrote: post #246
skissane wrote: post #240

Earlier quoted context omitted.

> I do not recall the EA actually being used anywhere by new code though, quite shocked by the mention of WSL. This explains it: https://learn.microsoft.com/en-au/archive/blogs/wsl/wsl-file... uid, gid, mode, and POSIX format timestamps are stored in an EA. It also mentions file capabilities being stored in an ADS. On Linux, capabilities and ACLs are stored in xattrs, so that seems to imply that xattrs are stored in…

the dollar sign convention predates NT, it's one of the things inherited from Files-11, where the metadata-files were not hidden from end user, just marked with strict enough permission checks. (A lot of VMS APIs used dollar signs for namespacing, too, and I believe some aspects of the naming scheme come from specific PDP assemblers when referring to some names?) Looking at NTFS from on-disk structure side, it always…

> Looking at NTFS from on-disk structure side, it always seemed quite obvious to me that a lot of accolades given to BeFS applied to NTFS - it's the lack of actually using the abilities

Microsoft had some very grand plans in this area... Cairo, OFS, WinFS... but they just kept on getting delayed, cancelled, pulled from the beta for too many issues. I think contemporary Microsoft has lost interest in this (it was something Bill Gates was big on) and moved on to other ideas.

Re: The Origins of DS_store (2006)

#250

Post by skissane »

p_l wrote: post #244
skissane wrote: post #241

Earlier quoted context omitted.

> Now, implementation in ntfs.sys is another thing and I have no idea if it's just an unused code path or if something would explode, ntfs.sys has validation checks in it which prevent you from directly creating anything other than named or unnamed $DATA attributes on a regular file, and named $DATA attributes on a directory, and (indirectly) creating other stuff (directories, file names, standard attributes, EAs) th…

I was thinking more of "ntfs.sys encounters filesystem structure with names set for normally unnamed attributes". The API preventing arbitrary messing up is a separate (and good and valid) concern.

> I was thinking more of "ntfs.sys encounters filesystem structure with names set for normally unnamed attributes".

From reading the source code of the Linux kernel NTFS driver (the ntfs3 one in the latest Linux kernel, not the older one it replaced), its (pretty reasonable) strategy is just to ignore things it doesn't expect. But I don't know what ntfs.sys does in such a scenario, I've never tried.

Post reply on HN