S3 is files, but not a filesystem
calpaterson.com
S3 is files, but not a filesystem
1–10 of 456 posts
Re: S3 is files, but not a filesystem
#2I dunno, are features like partial file overwrites necessary to make something a filesystem? This reminds me of how there are lots of internal systems at Google whose maintainers keep asserting are not filesystems, but everyone considers them so, to the point where "_____ is not a filesystem" has become an inside joke.
Re: S3 is files, but not a filesystem
#3My big pet peeve is AWS adding buttons in the UI to make "folders".
It is also a fiction! There are no folders in S3.
> When you create a folder in Amazon S3, S3 creates a 0-byte object with a key that's set to the folder name that you provided. For example, if you create a folder named photos in your bucket, the Amazon S3 console creates a 0-byte object with the key photos/. The console creates this object to support the idea of folders.
https://docs.aws.amazon.com/AmazonS3/latest/userguide/using-...
Re: S3 is files, but not a filesystem
#4S3 is a tagged versioned object storage with file like semantics implemented in the AWS SDK (via AWS S3 API's). The S3 object key is the tag.
Files and folders are used to make S3 buckets more approachable to those who either don't know or don't want to know what it actually is, and one day they get a surprise.
Re: S3 is files, but not a filesystem
#5My big pet peeve is AWS adding buttons in the UI to make "folders". It is also a fiction! There are no folders in S3. > When you create a folder in Amazon S3, S3 creates a 0-byte object with a key that's set to the folder name that you provided. For example, if you create a folder named photos in your bucket, the Amazon S3 console creates a 0-byte object with the key photos/. The console creates this object to suppor…
Is that really so different from how folders work on other systems? A directory inode is just an inode.
Re: S3 is files, but not a filesystem
#6My big pet peeve is AWS adding buttons in the UI to make "folders". It is also a fiction! There are no folders in S3. > When you create a folder in Amazon S3, S3 creates a 0-byte object with a key that's set to the folder name that you provided. For example, if you create a folder named photos in your bucket, the Amazon S3 console creates a 0-byte object with the key photos/. The console creates this object to suppor…
What exactly do you think a folder is? It’s just an abstraction for organising data.
Re: S3 is files, but not a filesystem
#7My big pet peeve is AWS adding buttons in the UI to make "folders". It is also a fiction! There are no folders in S3. > When you create a folder in Amazon S3, S3 creates a 0-byte object with a key that's set to the folder name that you provided. For example, if you create a folder named photos in your bucket, the Amazon S3 console creates a 0-byte object with the key photos/. The console creates this object to suppor…
Is that really so different from how folders work on other systems? A directory inode is just an inode.
The payload still contains a list of other inodes though
Re: S3 is files, but not a filesystem
#8My big pet peeve is AWS adding buttons in the UI to make "folders". It is also a fiction! There are no folders in S3. > When you create a folder in Amazon S3, S3 creates a 0-byte object with a key that's set to the folder name that you provided. For example, if you create a folder named photos in your bucket, the Amazon S3 console creates a 0-byte object with the key photos/. The console creates this object to suppor…
Weird that it says folders now. I remember it being very strictly called a prefix when I was at AWS.
Re: S3 is files, but not a filesystem
#9My big pet peeve is AWS adding buttons in the UI to make "folders". It is also a fiction! There are no folders in S3. > When you create a folder in Amazon S3, S3 creates a 0-byte object with a key that's set to the folder name that you provided. For example, if you create a folder named photos in your bucket, the Amazon S3 console creates a 0-byte object with the key photos/. The console creates this object to suppor…
Is that really so different from how folders work on other systems? A directory inode is just an inode.
Yes. It is, in practice, incredibly different.
Imagine you have a file named /some/dir/file.jpg.
In a filesystem, there’s an inode for /some. It contains an entry for /some/dir, which is also an inode, and then in the very deepest level, there is an inode for /some/dir/file.jpg. You can rename /some to /something_else if you want. Think of it kind of like a table:
+-------+--------+----------+-------+
| inode | parent | name | data |
+-------+--------+----------+-------+
| 1 | (null) | some | (dir) |
| 2 | 1 | dir | (dir) |
| 3 | 2 | file.jpg | jpeg |
+-------+--------+----------+-------+
+-------------------+------+
| key | data |
+-------------------+------+
| some/dir/file.jpg | jpeg |
+-------------------+------+
You can, if you want, use a completely different character to separate “components” in S3, rather than using /, because / is not special. If you want something like “some:dir:file.jpg” or “some.dir.file.jpg” you can do that. Again, because / is not special.
Re: S3 is files, but not a filesystem
#10My big pet peeve is AWS adding buttons in the UI to make "folders". It is also a fiction! There are no folders in S3. > When you create a folder in Amazon S3, S3 creates a 0-byte object with a key that's set to the folder name that you provided. For example, if you create a folder named photos in your bucket, the Amazon S3 console creates a 0-byte object with the key photos/. The console creates this object to suppor…
Is that really so different from how folders work on other systems? A directory inode is just an inode.
You can create a simulated directory, and write a bunch of files in it, but you can't atomically rename it--behind the scenes each file needs to be copied from old name to new.