Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Mountpoint – file client for S3 written in Rust, from AWS

github.com

Post reply on HN
Print view View original

61–70 of 102 posts

Re: Mountpoint – file client for S3 written in Rust, from AWS

#61

Post by awill »

cyberax wrote: post #40
mlhpdx wrote: post #36

Earlier quoted context omitted.

This is misleading. S3 is also incredibly fast. The former when you’re sequentially writing (or reading) objects and the latter when concurrently writing (or reading) vast numbers of them.

That depends on what you consider "fast". EFS (the "serverless" NFS) has sub-millisecond operation latency. S3 is more in the 10-20ms range for most operations, with occasional spikes. BTW, if you need a pure Go client for NFSv4 (including AWS EFS), feel free to check my: https://github.com/Cyberax/go-nfs-client

fast is an overloaded word. Could mean throughput, or latency. S3 throughput is incredible.

note, I worked on Amazon at S3 2015-2017.

Re: Mountpoint – file client for S3 written in Rust, from AWS

#63

Post by worewood »

Arnavion wrote: post #59
ary wrote: post #15

This is really interesting and something I've been thinking about for a while now. The SEMANTICS[1] doc details what is and isn't supported from a POSIX filesystem API perspective, and this stands out: Write operations (write, writev, pwrite, pwritev) are not currently supported. In the future, Mountpoint for Amazon S3 will support sequential writes, but with some limitations: Writes will only be supported to new fil…

I use a WebDAV server for storing backups (Fastmail Files). The server allows 10GB usage, but max file size is 250MB, and in any case WebDAV does not support partial writes. So writing a file requires reuploading it, which is the same situation as S3. What I did is: 1. Create 10000 files, each of 1MB size, so that the total usage is 10GB. 2. Mount each file as a loopback block device using `losetup`. 3. Create a RAID…

This is a very nice solution.

Re: Mountpoint – file client for S3 written in Rust, from AWS

#64

Post by kwk1 »

e12e wrote: post #58
thangngoc89 wrote: post #12

Earlier quoted context omitted.

In theory, you can just use this library since GC Storage supports S3 protocol. But in practice, I’m not sure

Or https://clone.org ?

Presumably you forgot the 'r': https://rclone.org/

Re: Mountpoint – file client for S3 written in Rust, from AWS

#65

Post by hardwaresofton »

Arnavion wrote: post #59
ary wrote: post #15

This is really interesting and something I've been thinking about for a while now. The SEMANTICS[1] doc details what is and isn't supported from a POSIX filesystem API perspective, and this stands out: Write operations (write, writev, pwrite, pwritev) are not currently supported. In the future, Mountpoint for Amazon S3 will support sequential writes, but with some limitations: Writes will only be supported to new fil…

I use a WebDAV server for storing backups (Fastmail Files). The server allows 10GB usage, but max file size is 250MB, and in any case WebDAV does not support partial writes. So writing a file requires reuploading it, which is the same situation as S3. What I did is: 1. Create 10000 files, each of 1MB size, so that the total usage is 10GB. 2. Mount each file as a loopback block device using `losetup`. 3. Create a RAID…

What a coincidence, I just recently did something similar.

Did you run into any problems with discard/zeroing/trim support?

This was a problem with sshfs — I can’t change the version/settings on the other side, and files seemed to simply grow and become more fragmented.

I suspected WebDAV and Samba might have had been the solution but never looked into it since sshfs is so solid.

Re: Mountpoint – file client for S3 written in Rust, from AWS

#66

Post by profquail »

netfortius wrote: post #23
the8472 wrote: post #22

Earlier quoted context omitted.

They have NFS (called EFS), but it's about 10x more expensive.

Good luck mounting EFS in Windows.

WinFsp (FUSE for Windows) has an NFS driver: https://github.com/winfsp/nfs-win

Re: Mountpoint – file client for S3 written in Rust, from AWS

#67

Post by operator-name »

hardwaresofton wrote: post #65
Arnavion wrote: post #59

Earlier quoted context omitted.

I use a WebDAV server for storing backups (Fastmail Files). The server allows 10GB usage, but max file size is 250MB, and in any case WebDAV does not support partial writes. So writing a file requires reuploading it, which is the same situation as S3. What I did is: 1. Create 10000 files, each of 1MB size, so that the total usage is 10GB. 2. Mount each file as a loopback block device using `losetup`. 3. Create a RAID…

What a coincidence, I just recently did something similar. Did you run into any problems with discard/zeroing/trim support? This was a problem with sshfs — I can’t change the version/settings on the other side, and files seemed to simply grow and become more fragmented. I suspected WebDAV and Samba might have had been the solution but never looked into it since sshfs is so solid.

If they're using LUKS then I think trimming/discard won't be possible.

Re: Mountpoint – file client for S3 written in Rust, from AWS

#68

Post by hardwaresofton »

operator-name wrote: post #67
hardwaresofton wrote: post #65

Earlier quoted context omitted.

What a coincidence, I just recently did something similar. Did you run into any problems with discard/zeroing/trim support? This was a problem with sshfs — I can’t change the version/settings on the other side, and files seemed to simply grow and become more fragmented. I suspected WebDAV and Samba might have had been the solution but never looked into it since sshfs is so solid.

If they're using LUKS then I think trimming/discard won't be possible.

My immediate instinct was that LUKS could issue trim/discard.

It looks like there's some anecdotal evidence out there that LUKS can discard

https://superuser.com/questions/124310/does-luks-encryption-... https://unix.stackexchange.com/questions/341442/luks-discard...

My question is more for the mdraid at the bottom of the stack than anything. I'm also a little curious about performance of something webdav vs. samba vs. sshfs (sshfs usually wins out and webdav does not strike me as particularly efficient)

Re: Mountpoint – file client for S3 written in Rust, from AWS

#69

Post by Arnavion »

hardwaresofton wrote: post #65
Arnavion wrote: post #59

Earlier quoted context omitted.

I use a WebDAV server for storing backups (Fastmail Files). The server allows 10GB usage, but max file size is 250MB, and in any case WebDAV does not support partial writes. So writing a file requires reuploading it, which is the same situation as S3. What I did is: 1. Create 10000 files, each of 1MB size, so that the total usage is 10GB. 2. Mount each file as a loopback block device using `losetup`. 3. Create a RAID…

What a coincidence, I just recently did something similar. Did you run into any problems with discard/zeroing/trim support? This was a problem with sshfs — I can’t change the version/settings on the other side, and files seemed to simply grow and become more fragmented. I suspected WebDAV and Samba might have had been the solution but never looked into it since sshfs is so solid.

I did create the block files as sparse originally (using `truncate`), but at some point in the process they became realized on disk. Don't know if it was the losetup or the mdadm or the cryptsetup. I didn't really worry about it, since the block files need to be synced to the WebDAV server in full anyway.

Re: Mountpoint – file client for S3 written in Rust, from AWS

#70

Post by hardwaresofton »

Arnavion wrote: post #69
hardwaresofton wrote: post #65

Earlier quoted context omitted.

What a coincidence, I just recently did something similar. Did you run into any problems with discard/zeroing/trim support? This was a problem with sshfs — I can’t change the version/settings on the other side, and files seemed to simply grow and become more fragmented. I suspected WebDAV and Samba might have had been the solution but never looked into it since sshfs is so solid.

I did create the block files as sparse originally (using `truncate`), but at some point in the process they became realized on disk. Don't know if it was the losetup or the mdadm or the cryptsetup. I didn't really worry about it, since the block files need to be synced to the WebDAV server in full anyway.

Ahh OK, I think I see -- since the block files are synced in full, you are always swapping blocks and doing ~1MB of writing no matter what.

> I use a WebDAV server for storing backups (Fastmail Files). The server allows 10GB usage, but max file size is 250MB, *and in any case WebDAV does not support partial writes*. So writing a file requires reuploading it, which is the same situation as S3.

This is the part I absolutely missed. I was wondering how you were ensuring 1MB writes -- whether it was at the XFS level or mdraid level...

I think another thing that is missing which I'm inferring (hopefully correctly) is that you've mounted your webdav server to disk. So your stack is:

- LUKS

- mdraid

- losetup

- webdav fs mount

Is that correct?

Post reply on HN