Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The trouble with symbolic links

lwn.net

Post reply on HN
Print view View original

51–60 of 126 posts

Re: The trouble with symbolic links

#51

Post by [deleted] »

lxgr wrote: post #43
js2 wrote: post #38

I'm sorry symlinks are a thorn in Jeremy's side, but they are useful from a user's perspective. Hard links don't fill the same need. You can't normally hard link directories. If a file has multiple links, finding them all normally requires scanning the entire file system, so deleting a file now becomes harder. A file with multiple links doesn't have an obvious canonical path. As an example of all these issues, I mana…

> You can't normally hard link directories. That's only to avoid loops, as far as I understand. Symlinks do allow loops, but require application programmers to handle them. So maybe we just need better APIs/API contracts around loops, rather than two types of links? > If a file has multiple links, finding them all normally requires scanning the entire file system Couldn't this pretty easily be solved at the file syst…

[deleted]

Re: The trouble with symbolic links

#52

Post by klodolph »

masklinn wrote: post #26
klodolph wrote: post #3

Earlier quoted context omitted.

Could you elaborate? Seems like there’s a bunch of things that can’t be batched together on an ordinary file, without involving symlinks.

What they (and TFA) are saying is that there is no transactional view of the FS. If you could work in “repeatable read” (only and always see the state of the FS before you started the transaction) symlink races wouldn’t be possible.

Right, but there is no transactional view with or without symlinks.

Re: The trouble with symbolic links

#53

Post by shadowgovt »

jhallenworld wrote: post #24
shadowgovt wrote: post #16

Don't hard links suffer from the issue that because they're actually links to a specific file, not path pointers, that you can replace the target file thinking you have updated something in the system and instead have stale hard links lying around, referencing the older version when you intended to replace the older version for all users? I think that in a hypothetical world where symlinks worked like hard links, we'…

Hardlinks exist, so you already have this problem. Tar has to keep track of inode numbers to recreate hard links for example.

What I mean is that it's pretty SOP to have a package that installs a new command to work by installing to /opt/my-package and then symlinking /usr/bin/my-cmd to /opt/my-package/my-cmd.

In the absence of symlinks, that link would be hard and dpkg et. al would have to do package management by deleting the /usr/bin/my-cmd link and re-creating it instead of letting it ride, trusting that it will point to the correct thing when the update completes because the target bin will have changed.

Re: The trouble with symbolic links

#54

Post by XorNot »

lxgr wrote: post #43
js2 wrote: post #38

I'm sorry symlinks are a thorn in Jeremy's side, but they are useful from a user's perspective. Hard links don't fill the same need. You can't normally hard link directories. If a file has multiple links, finding them all normally requires scanning the entire file system, so deleting a file now becomes harder. A file with multiple links doesn't have an obvious canonical path. As an example of all these issues, I mana…

> You can't normally hard link directories. That's only to avoid loops, as far as I understand. Symlinks do allow loops, but require application programmers to handle them. So maybe we just need better APIs/API contracts around loops, rather than two types of links? > If a file has multiple links, finding them all normally requires scanning the entire file system Couldn't this pretty easily be solved at the file syst…

Hard links don't have a canonical name though - they're all equally the same file, and this is really a problem: opening and editing a file in one location, edits it in all of them without you knowing what those locations might be.

Symlinks at least explicitly declare the dependency and how it should mutate.

A classic being /etc/resolve.conf symlinks - if I'm untarring and restore a symlink for it, I'm currently saying the file should have content from somewhere else on the system - not that the file is specific content.

Re: The trouble with symbolic links

#55

Post by js2 »

lxgr wrote: post #43
js2 wrote: post #38

I'm sorry symlinks are a thorn in Jeremy's side, but they are useful from a user's perspective. Hard links don't fill the same need. You can't normally hard link directories. If a file has multiple links, finding them all normally requires scanning the entire file system, so deleting a file now becomes harder. A file with multiple links doesn't have an obvious canonical path. As an example of all these issues, I mana…

> You can't normally hard link directories. That's only to avoid loops, as far as I understand. Symlinks do allow loops, but require application programmers to handle them. So maybe we just need better APIs/API contracts around loops, rather than two types of links? > If a file has multiple links, finding them all normally requires scanning the entire file system Couldn't this pretty easily be solved at the file syst…

I tried to construct my argument to make it clear that I'm aware there are ways to solve the issues with hard links, but they have their own sets of trade-offs.

For hard links, it's not only that they can cause loops. There are the other issues I outlined (linking across file systems, no single canonical representation of the file in the file system, finding all the links to the file, etc).

There's no "just store a back pointer." That will obviously introduce its own set of complexities and trade-offs. Where do you store the pointers? What's the API for viewing them? What's the CLI for viewing them? Is it a new switch to `ls`? A new CLI entirely? How do you keep the pointers up to date? What sort of locking is needed when updating the pointers? What about `fsck`? How do you get this implemented across the multitude of Unix and Unix-like OS's and file systems?

(As an aside, I've been really trying to stop using the word "just" lately as I've learned that things are rarely so simple to justify the word.)

Again, I'm not saying there isn't a better solution, but I don't think it's patching up hard links. I think it's something outside the box of both hard links and symbolic links.

Re: The trouble with symbolic links

#56

Post by masklinn »

klodolph wrote: post #52
masklinn wrote: post #26

Earlier quoted context omitted.

What they (and TFA) are saying is that there is no transactional view of the FS. If you could work in “repeatable read” (only and always see the state of the FS before you started the transaction) symlink races wouldn’t be possible.

Right, but there is no transactional view with or without symlinks.

Without symlinks it doesn't matter, because by definition a symlink race requires a symlink to be involved.

Re: The trouble with symbolic links

#57

Post by masklinn »

XorNot wrote: post #54
lxgr wrote: post #43

Earlier quoted context omitted.

> You can't normally hard link directories. That's only to avoid loops, as far as I understand. Symlinks do allow loops, but require application programmers to handle them. So maybe we just need better APIs/API contracts around loops, rather than two types of links? > If a file has multiple links, finding them all normally requires scanning the entire file system Couldn't this pretty easily be solved at the file syst…

Hard links don't have a canonical name though - they're all equally the same file, and this is really a problem: opening and editing a file in one location, edits it in all of them without you knowing what those locations might be. Symlinks at least explicitly declare the dependency and how it should mutate. A classic being /etc/resolve.conf symlinks - if I'm untarring and restore a symlink for it, I'm currently sayi…

> Hard links don't have a canonical name though - they're all equally the same file, and this is really a problem: opening and editing a file in one location, edits it in all of them without you knowing what those locations might be.

That is something the filesystem could store tho, in the same way it stores the number of links to a file it could be a bit more capable and store the links themselves (possibly in a xattr).

> Symlinks at least explicitly declare the dependency and how it should mutate.

They only declare one dependency one way, it's not like a symlink gives you all the other symlinks to the terminal location it will affect.

Re: The trouble with symbolic links

#58

Post by jerf »

jmillikin wrote: post #4

IMO the article's conclusion is backwards. There's nothing wrong with symlinks when files are opened with openat(), since in principal the program (or controlling program) should always be in control of the filesystem layout. It's open() that causes problems, and complex interactions with symlinks in attacker-controlled directories are just one of them. The POSIX file API was designed before the concept of capability…

"There's nothing wrong with symlinks when files are opened with openat(), since in principal the program (or controlling program) should always be in control of the filesystem layout. It's open() that causes problems, and complex interactions with symlinks in attacker-controlled directories are just one of them."

One of my minor annoyances with new languages is the continued persistence of open-based file APIs, with openat APIs shoved off to the side if they are even implemented at all. If you start from scratch with an openat-based API, it's not even that hard; you basically get a file object, just one with some different attributes and methods (or appropriate local ideas), most of which you don't care about, and it's not that hard to work with if you start with that from day one. It can be quite hard to backport something deeply based on string-based path manipulation into the *at-based APIs, though.

I haven't deeply studied it but you ought to be able to simulate an openat-based API on a conventional filesystem that doesn't support it. It may not immunize you to security issues, but at least the code ought to be as portable as any other code that starts getting detailed about its interactions with filesystems, which is already "kinda, not very, some elbow grease required"... it's not like the bar is sky high because all that stuff already works perfectly across all platforms and filesystems anyhow.

Re: The trouble with symbolic links

#59

Post by stjohnswarts »

Does anyone else read something like this

Jeremy Allison gave a talk titled "The UNIX Filesystem API is profoundly broken: What to do about it?".

and immediately shut down on the person saying it because he all know it's extremely hyperbolic given what is happening in reality?

It's great to not like something and point out its flaws as far as you use them and "here's the great idea to fix those issues" but to try to offend you audience and the Unix community as the first words you see in a talk is a great way to have people shut off and think "oh great another neckbeard with an overinflated sense of self"

Re: The trouble with symbolic links

#60

Post by toast0 »

littlestymaar wrote: post #9
mongol wrote: post #7

"X is fundamentally broken" is a tired trope. To me, something is broken if it is no longer working as intended. It used to work, but now it does not - it is broken. If something works as intended, but its utility is limited, and it can be improved, it is not broken.

That “X − designed in the 70s when we had no idea of anything regarding computers − is fundamentally broken” isn't so surprising after all. In fact, computers are probably the only place in the entire technology landscape where we keep using almost unmodified stuff from the 70s and decided we cannot change it because there's too much things relying on it. I don't like breaking everything all the time more than anyone…

Internal plumbing is largely unchanged. Sure, there's more flexible pipe, and a lot more plastic pipe, and a lot more quarter turn valves, but thread pitch and pipe diameters are largely unchanged and unchangable.
Post reply on HN