Ultra fast Thunderbolt NAS with Apple M1 and Linux • Hacker News Board

Re: Ultra fast Thunderbolt NAS with Apple M1 and Linux

#91

Post by reacharavindh » Thu, Aug 05, 2021, 8:04 AM UTC

SilverRed wrote: post #74

reacharavindh wrote: post #10

Earlier quoted context omitted.

Local storage and Networked storage always have their places. Local solid state storage almost always gets you better performance because you negate the overhead of network. Unless, you’re talking fancy arrays connected to saturate 100G NICs. Local storage falls apart the moment you need that storage accessible from another machine…

Networked storage also falls apart the moment you need anything better than basic file storage. So many filesystem features do not work networked, most network file mounts do not support any kind of modern security and the ones that do are often super slow. Everything has its place, but I wasted so much time trying to get a nas to work as storage for my home server.

Not to sound pedantic, but Local storage and networked storage are not alternatives. They satisfy different needs.

If you need reasonable access to your storage from more than one computer, you need networked storage(of any kind).

Alternatively, if you are running say a database server, and you know the primary storage could be local, it is ideal to choose better performing local storage and of course plan for backups to networked storage.

As much as I agree with your pain points of networked storage, it’s not like you can “replace” it with local storage everywhere.

I have dealt with a couple PB of networked storage for a research cluster. (FreeBSD+ZFS) over NFS on a dedicated network to a cluster of about 100 servers running recent versions of Linux. It worked like a charm while we admittedly kept things simple in terms of file system features.

All clients mount the appropriate NFS shares they need via autofs(only when they are needed).

ZFS being ZFS was just a marvel of software engineering, was solid as a rock. While giving us transparent compression, superb read caching in memory, and cheap snapshots.

Re: Ultra fast Thunderbolt NAS with Apple M1 and Linux

#92

Post by johnwalkr » Thu, Aug 05, 2021, 12:29 PM UTC

This year I set up 10Gbps networking in my home lab since I am lucky enough to have 10Gbps internet and it didn’t make sense to pay for that but not have it for all my machines.

Or at least I tried to set it up. 10G pcie cards are now cheap enough, but routers are not. Thunderbolt adapters are also expensive, bulky and use a ton of power[1]. I’m keeping my eye open for a surplus router and cheaper adapters but in the meantime I am only using a 2 port card in my main machine to serve at 10Gbps to one other machine. I wonder if I could also serve my m1 MacBook using thunderbolt 3.

[1] In my work life I also tried to use Ethernet in some embedded projects. It turns out Ethernet is just power hungry in general. Even 100Mbps Ethernet uses about 0.5W per port. There are standards for lower power but I’ve yet to see anyone use them.

Re: Ultra fast Thunderbolt NAS with Apple M1 and Linux

#93

Post by whalesalad » Thu, Aug 05, 2021, 12:31 PM UTC

pr0nin wrote: post #89

whalesalad wrote: post #18

Just yesterday I was thinking how badly I want an obscenely fast local datastore for Docker. I am managing a handful of clients right now who are all using Docker containers for their microservice environments and my local machine runs out of disk daily. What I really want is lightning quick network storage, but I don't think it would be feasible to roll out 10g networking in my current rental home.

What about separate Linux box laying around just to run docker? 8GB M1 Mac just don't have enough RAM for docker VM. Looks like local docker server should just work exactly same way as local docker while you at same location.

Then you get into the problem of low latency directory sync between your box and that host. I’m still trying to crack that nut.

Re: Ultra fast Thunderbolt NAS with Apple M1 and Linux

#94

Post by lizknope » Thu, Aug 05, 2021, 12:42 PM UTC

wil421 wrote: post #70

conradev wrote: post #60

I was actually looking for something like this. I have one computer that I want to back up to both local hard drives and to the cloud. A NAS is great, because you can back up to it and have the backup stored on local hard drives, but it also has a CPU and it can be responsible for uploading the data to the cloud. This is better to do from a dedicated device, especially if you are uploading large amounts of data with…

Running FreeNAS on a 1GbE network and I wouldn’t call it slow at all. The initial back is much much faster than my 1TB personal media library I sometimes backup to OneDrive. I think I have 3TB of video from Usenet that is downloaded onto a windows box, uses a 10 year old intel as an unpack scratch pad, and then is stored on the NAS box. Sure the initial library transfer is slow but if you can use a wire it’s not that…

I got 2 used SFP+ cards and a DAC cable for $60 total on eBay. 10G between my file server and primary desktop is great. 1G for everything is fine.

Re: Ultra fast Thunderbolt NAS with Apple M1 and Linux

#95

Post by smaccona » Thu, Aug 05, 2021, 1:34 PM UTC

pronoiac wrote: post #25

> In this post I discuss how you can upgrade a NAS Server by adding Thunderbolt 3 for lightning fast connectivity at 20 or 40Gbps. This is an interesting angle of attack, though I'll note that 10Gbps Ethernet was added as an option on the M1 Mac Minis.

10Gbps was already an option on the Intel Mac Minis. My guess is that Apple envisioned a use case for it to be a file server or similar.

Re: Ultra fast Thunderbolt NAS with Apple M1 and Linux

#96

Post by BackBlast » Thu, Aug 05, 2021, 1:57 PM UTC

vitejose wrote: post #23

devonkim wrote: post #20

Earlier quoted context omitted.

10gbe can be done over RJ45 with a direct cable connection and is something I've done before in a pinch between my personal NAS and a client machine (just be careful about your storage protocol because NFS has its pros and cons just like iSCSI and FCoE, and with macOS as a client you're likely in for a Bad Time like I did).

Out of curiosity, what issues did you run into with NFS on macOS? I'm looking into building a NAS for a network of macOS, Linux, and BSD machines and figured NFS would be my first choice.

I've had stability problems with nfs on MacOS. I ended up using smb instead. Smb ended up being CPU bound on my NAS due to encryption I haven't taken the time to figure out how to disable. It's close to line speed for me though. It's also reliable.

Re: Ultra fast Thunderbolt NAS with Apple M1 and Linux

#97

Post by bpye » Thu, Aug 05, 2021, 5:28 PM UTC

johnwalkr wrote: post #92

This year I set up 10Gbps networking in my home lab since I am lucky enough to have 10Gbps internet and it didn’t make sense to pay for that but not have it for all my machines. Or at least I tried to set it up. 10G pcie cards are now cheap enough, but routers are not. Thunderbolt adapters are also expensive, bulky and use a ton of power[1]. I’m keeping my eye open for a surplus router and cheaper adapters but in the…

Have you considered using an x86 machine with dual 10G NICs as your router, and then all you’d need is a 10G switch which may be cheaper than a router. I’ve considered the approach but haven’t undertaken it yet.

Re: Ultra fast Thunderbolt NAS with Apple M1 and Linux

#98

Post by wil421 » Thu, Aug 05, 2021, 5:36 PM UTC

lizknope wrote: post #94

wil421 wrote: post #70

Earlier quoted context omitted.

Running FreeNAS on a 1GbE network and I wouldn’t call it slow at all. The initial back is much much faster than my 1TB personal media library I sometimes backup to OneDrive. I think I have 3TB of video from Usenet that is downloaded onto a windows box, uses a 10 year old intel as an unpack scratch pad, and then is stored on the NAS box. Sure the initial library transfer is slow but if you can use a wire it’s not that…

I got 2 used SFP+ cards and a DAC cable for $60 total on eBay. 10G between my file server and primary desktop is great. 1G for everything is fine.

What brand and model of cards? Right now I’m limited to hardware that is compatible with FreeNAS.

Re: Ultra fast Thunderbolt NAS with Apple M1 and Linux

#99

Post by rsync » Thu, Aug 05, 2021, 8:06 PM UTC

Teknoman117 wrote: post #52

This is cool, but my main takeaway is that you can get around needing the thunderbolt enablement header by shorting two pins together and use it on any motherboard? What the hell Intel...

"What the hell Intel... "

I don't understand any of it ...

If I wanted to add a LPT parallel port to my motherboard, I would find an appropriate add-in card with the physical port and plug it into the PCI bus and locate an appropriate driver.

Thunderbolt is just another kind of port, right ? If you buy the add-in card to get the physical port and you have a driver to drive the card ... what in the world is "enablement" and why would it be required ?

Re: Ultra fast Thunderbolt NAS with Apple M1 and Linux

#100

Post by lizknope » Thu, Aug 05, 2021, 8:48 PM UTC

wil421 wrote: post #98

lizknope wrote: post #94

Earlier quoted context omitted.

I got 2 used SFP+ cards and a DAC cable for $60 total on eBay. 10G between my file server and primary desktop is great. 1G for everything is fine.

What brand and model of cards? Right now I’m limited to hardware that is compatible with FreeNAS.

Mellanox ConnectX-2

I'm running Linux