Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ultra fast Thunderbolt NAS with Apple M1 and Linux

chrisbergeron.com

Post reply on HN
Print view View original

31–40 of 102 posts

Re: Ultra fast Thunderbolt NAS with Apple M1 and Linux

#31

Post by geerlingguy »

vitejose wrote: post #23
devonkim wrote: post #20

Earlier quoted context omitted.

10gbe can be done over RJ45 with a direct cable connection and is something I've done before in a pinch between my personal NAS and a client machine (just be careful about your storage protocol because NFS has its pros and cons just like iSCSI and FCoE, and with macOS as a client you're likely in for a Bad Time like I did).

Out of curiosity, what issues did you run into with NFS on macOS? I'm looking into building a NAS for a network of macOS, Linux, and BSD machines and figured NFS would be my first choice.

I've had weird issues with some applications and NFS on macOS (notably, Final Cut Pro X and Photoshop). And I've also had weird issues with SMB and some apps (notably, Photoshop again!).

But at least SMB has been pretty stable for years. NFS support has had some hiccups. A few 10.14 macOS releases broke NFSv3 and NFSv4 for me, but it was fixed in later releases.

Re: Ultra fast Thunderbolt NAS with Apple M1 and Linux

#32

Post by whalesalad »

cbushko wrote: post #26
whalesalad wrote: post #18

Just yesterday I was thinking how badly I want an obscenely fast local datastore for Docker. I am managing a handful of clients right now who are all using Docker containers for their microservice environments and my local machine runs out of disk daily. What I really want is lightning quick network storage, but I don't think it would be feasible to roll out 10g networking in my current rental home.

Would running something like the docker proxy below on an old laptop help? TLDR: A caching proxy for Docker; allows centralized management of (multiple) registries and their authentication; caches images from any registry. Caches the potentially huge blob/layer requests (for bandwidth/time savings), and optionally caches manifest requests ("pulls") to avoid rate-limiting. https://github.com/rpardini/docker-registry-p…

At some point you need to build images locally or store images locally to run a local docker-compose cluster.

I need a fat block store for that.

Re: Ultra fast Thunderbolt NAS with Apple M1 and Linux

#33

Post by yawniek »

whalesalad wrote: post #18

Just yesterday I was thinking how badly I want an obscenely fast local datastore for Docker. I am managing a handful of clients right now who are all using Docker containers for their microservice environments and my local machine runs out of disk daily. What I really want is lightning quick network storage, but I don't think it would be feasible to roll out 10g networking in my current rental home.

why don't you simply use one or many nvme thunderbolt/usb disks and move the caches there?

Re: Ultra fast Thunderbolt NAS with Apple M1 and Linux

#34

Post by 411111111111111 »

vitejose wrote: post #23
devonkim wrote: post #20

Earlier quoted context omitted.

10gbe can be done over RJ45 with a direct cable connection and is something I've done before in a pinch between my personal NAS and a client machine (just be careful about your storage protocol because NFS has its pros and cons just like iSCSI and FCoE, and with macOS as a client you're likely in for a Bad Time like I did).

Out of curiosity, what issues did you run into with NFS on macOS? I'm looking into building a NAS for a network of macOS, Linux, and BSD machines and figured NFS would be my first choice.

I'm not the author nor am I an apple user anymore, though i was at one point.

NFS doesn't have a lot of metadata, so you loose a lot of functionality. iPhoto for example could cause major issues if you persisted its library on a nfs, potentially making the library irreparable.

Re: Ultra fast Thunderbolt NAS with Apple M1 and Linux

#35

Post by ehutch79 »

yawniek wrote: post #33
whalesalad wrote: post #18

Just yesterday I was thinking how badly I want an obscenely fast local datastore for Docker. I am managing a handful of clients right now who are all using Docker containers for their microservice environments and my local machine runs out of disk daily. What I really want is lightning quick network storage, but I don't think it would be feasible to roll out 10g networking in my current rental home.

why don't you simply use one or many nvme thunderbolt/usb disks and move the caches there?

Not the op, but if the nas was pre-existing, and already serving other network clients, no reason to add a second one.

Re: Ultra fast Thunderbolt NAS with Apple M1 and Linux

#36

Post by cbushko »

whalesalad wrote: post #32
cbushko wrote: post #26

Earlier quoted context omitted.

Would running something like the docker proxy below on an old laptop help? TLDR: A caching proxy for Docker; allows centralized management of (multiple) registries and their authentication; caches images from any registry. Caches the potentially huge blob/layer requests (for bandwidth/time savings), and optionally caches manifest requests ("pulls") to avoid rate-limiting. https://github.com/rpardini/docker-registry-p…

At some point you need to build images locally or store images locally to run a local docker-compose cluster. I need a fat block store for that.

That is painful for sure.

Re: Ultra fast Thunderbolt NAS with Apple M1 and Linux

#37

Post by devonkim »

vitejose wrote: post #23
devonkim wrote: post #20

Earlier quoted context omitted.

10gbe can be done over RJ45 with a direct cable connection and is something I've done before in a pinch between my personal NAS and a client machine (just be careful about your storage protocol because NFS has its pros and cons just like iSCSI and FCoE, and with macOS as a client you're likely in for a Bad Time like I did).

Out of curiosity, what issues did you run into with NFS on macOS? I'm looking into building a NAS for a network of macOS, Linux, and BSD machines and figured NFS would be my first choice.

NFSv3 and NFSv4 clients were fine when local machines over TCP and always-on like with a previous Mac Mini I had but resuming from a laptop was pulling teeth and may or may not have been the reason for some of the crashes I got. iSCSI in theory should be better but I had a lot of problems getting the iSCSI initiator I found to work for myself despite being a pretty simple use case I thought, and I gave up on the project around then as I found macOS simply required more budget for myself to buy an appropriate iSCSI initiator. This was over 5 years ago and things may have changed between then and now.

Re: Ultra fast Thunderbolt NAS with Apple M1 and Linux

#38

Post by cyberge99 »

yawniek wrote: post #33
whalesalad wrote: post #18

Just yesterday I was thinking how badly I want an obscenely fast local datastore for Docker. I am managing a handful of clients right now who are all using Docker containers for their microservice environments and my local machine runs out of disk daily. What I really want is lightning quick network storage, but I don't think it would be feasible to roll out 10g networking in my current rental home.

why don't you simply use one or many nvme thunderbolt/usb disks and move the caches there?

Author here: I actually used a thunderbolt nvme and adapter for a bit. Then, for kicks I got another one and combined them in a raid-0 on macos. Realizing I didn’t want to run nvme’s, in raid-0 over two cables in any sort of production (homelab - I use production loosely here), is what sent me down the Thunderbolt rabbit hole that precipitated this blog post.

Re: Ultra fast Thunderbolt NAS with Apple M1 and Linux

#39

Post by SahAssar »

jeroenhd wrote: post #21
michelb wrote: post #19

Earlier quoted context omitted.

Only safari on big sur has webp support.

That's very strange. Is this a versioning problem (i.e. Catalina not being supported anymore?) or is there some kind of hardware restriction? From what I can find, Catalina should still be supported for more than a year from now, right? Edit: well, that's bizarre. Caniuse.com [0] lists the same restriction, WebP is only available from Big Sur onwards. And then people wonder why nobody likes developing for Safari. [0]…

For media formats the support is often dependent on the OS. It's been that way forever (ask anyone deploying video in early html5 and getting it to work cross-os and cross-platform). I guess it's more expected with videos, but images are (usually) handled the same.

Re: Ultra fast Thunderbolt NAS with Apple M1 and Linux

#40

Post by cyberge99 »

xiconfjs wrote: post #4

this blog uses webp as image format without a fallback option. Had to switch to Firefox because Safari wouldn't display the pictures.

Thanks for pointing this out. I’ll add a fallback using the “picture” element (TIL).

I also use some javascript to delay rendering until the browser window stays on an image. That could very well be introducing some wonkiness.

Post reply on HN