Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ultra fast Thunderbolt NAS with Apple M1 and Linux

chrisbergeron.com

Post reply on HN
Print view View original

41–50 of 102 posts

Re: Ultra fast Thunderbolt NAS with Apple M1 and Linux

#41

Post by kllrnohj »

jasonjackson wrote: post #8
reacharavindh wrote: post #3

Not to take away anything from the fun of figuring out Thunderbolt on Linux, and such. But, I wonder if a NAS with a 40 Gbps or even 100Gbps NIC hooked up to a Thunderbolt to Ethernet adapter would achieve a better result without extra effort.

probably, but thunderbolt would cost 5x less for equivalent 40Gbps NIC

You can find tons of used 10-40Gbps NICs on ebay for $50 or less. For a simple point-to-point connection with both sides having PCI-E slots that's going to be easier & cheaper to set up than thunderbolt will be. It'll also be more reliable & more practical since getting 40Gbps over Thunderbolt requires extremely short cables, whereas you can get DAC cables for 40Gbps QSFP or Infiniband up to 7M or so. And there's no confusion around what the cable can actually do, unlike the nightmare that is finding the right USB-C cable with the right signal integrity requirements at the right distance for what you actually need/want.

Re: Ultra fast Thunderbolt NAS with Apple M1 and Linux

#42

Post by interactivecode »

cyberge99 wrote: post #40
xiconfjs wrote: post #4

this blog uses webp as image format without a fallback option. Had to switch to Firefox because Safari wouldn't display the pictures.

Thanks for pointing this out. I’ll add a fallback using the “picture” element (TIL). I also use some javascript to delay rendering until the browser window stays on an image. That could very well be introducing some wonkiness.

you should use the built in lazy loading https://caniuse.com/loading-lazy-attr or intersection observer for this https://developer.mozilla.org/en-US/docs/Web/API/Intersectio...

Re: Ultra fast Thunderbolt NAS with Apple M1 and Linux

#43

Post by jeroenhd »

SahAssar wrote: post #39
jeroenhd wrote: post #21

Earlier quoted context omitted.

That's very strange. Is this a versioning problem (i.e. Catalina not being supported anymore?) or is there some kind of hardware restriction? From what I can find, Catalina should still be supported for more than a year from now, right? Edit: well, that's bizarre. Caniuse.com [0] lists the same restriction, WebP is only available from Big Sur onwards. And then people wonder why nobody likes developing for Safari. [0]…

For media formats the support is often dependent on the OS. It's been that way forever (ask anyone deploying video in early html5 and getting it to work cross-os and cross-platform). I guess it's more expected with videos, but images are (usually) handled the same.

Maybe with hardware decoding, but software decoding is often done in browsers when hardware support is not available.

I suppose Apple doesn't want to include a software fallback for platforms where hardware isn't available to discourage developers from using the format. It's not like Apple would need to write any complicated decoding algorithms when there's already an open source implementation that's free to use.

Re: Ultra fast Thunderbolt NAS with Apple M1 and Linux

#44

Post by systemvoltage »

reacharavindh wrote: post #10
jeffbee wrote: post #5

Where by "ultra fast" it is meant that it is much slower than a directly-attached SSD that occupies the same port.

Local storage and Networked storage always have their places. Local solid state storage almost always gets you better performance because you negate the overhead of network. Unless, you’re talking fancy arrays connected to saturate 100G NICs. Local storage falls apart the moment you need that storage accessible from another machine…

Dumb question: Can you use Mac's shared folder for another machine to access files?

Re: Ultra fast Thunderbolt NAS with Apple M1 and Linux

#45

Post by dijit »

kilroy123 wrote: post #29
whalesalad wrote: post #18

Just yesterday I was thinking how badly I want an obscenely fast local datastore for Docker. I am managing a handful of clients right now who are all using Docker containers for their microservice environments and my local machine runs out of disk daily. What I really want is lightning quick network storage, but I don't think it would be feasible to roll out 10g networking in my current rental home.

Just curious what size is your hard drive? I had this pain with my last macbook. I ALWAYS ran out of disk space. Never again! I went with a 1 TB drive for my M1.

I offload docker to another machine and I'm still using about 400GiB of storage on my relatively new MacBook; and I basically never use this machine outside of work (and even then I didn't use it for like 4 months because I preferred linux). it has no local media or anything.

I'm shocked by how much disk I've used actually.

https://imgur.com/a/iHV4IKn

Re: Ultra fast Thunderbolt NAS with Apple M1 and Linux

#46

Post by insaneirish »

There's something weird here:

  root@nas-04:~ # ifconfig thunderbolt0 10.1.1.2

  root@nas-04:~ # ifconfig thunderbolt0
  thunderbolt0: flags=4163  mtu 1500
          inet 192.168.10.2  netmask 255.0.0.0  broadcast 10.255.255.255
The IP on the interface is different than the one that it was theoretically configured with. My suspicion is that perhaps the IP used was changed at some point and this is just an editing slip.

> Now that the interface has an IP, we have to add a static host route entry for it to communicate:

If both sides are configured with an appropriate netmask on the same subnet, there's no need for a specific host route.

Re: Ultra fast Thunderbolt NAS with Apple M1 and Linux

#47

Post by MayeulC »

pronoiac wrote: post #25

> In this post I discuss how you can upgrade a NAS Server by adding Thunderbolt 3 for lightning fast connectivity at 20 or 40Gbps. This is an interesting angle of attack, though I'll note that 10Gbps Ethernet was added as an option on the M1 Mac Minis.

here's an idea: use link aggregation yo make it 10 G faster, then.

I wonder what's the simplest option. could it be made to work with a virtual interface, like wireguard?

Re: Ultra fast Thunderbolt NAS with Apple M1 and Linux

#48

Post by SahAssar »

jeroenhd wrote: post #43
SahAssar wrote: post #39

Earlier quoted context omitted.

For media formats the support is often dependent on the OS. It's been that way forever (ask anyone deploying video in early html5 and getting it to work cross-os and cross-platform). I guess it's more expected with videos, but images are (usually) handled the same.

Maybe with hardware decoding, but software decoding is often done in browsers when hardware support is not available. I suppose Apple doesn't want to include a software fallback for platforms where hardware isn't available to discourage developers from using the format. It's not like Apple would need to write any complicated decoding algorithms when there's already an open source implementation that's free to use.

H264 didn't work in FF on MacOS and Linux (but it still worked on windows) until v34 (and it only started working because Cisco donated a license), Ogg Vorbis only works on MacOS 11.3 or later in safari (and it also depends on which container you use), HEVC in IE/EdgeHTML depended on hardware (with no fallback to software whatsoever), AV1 in FF65 only worked on Windows, AV1 in FF66 only worked on Windows and MacOS, chrome on android 2.3 required you to specify a m4v without mime-type, but doing it with mime-type worked fine on desktop chrome.

I'm just saying it's not unprecedented to have the same browser version supporting different audio/video/image formats depending on OS or hardware.

For image formats it's not as usual, but if someone is making a site only supporting webp then I'd assume they'd look up some support matrixes beforehand.

Re: Ultra fast Thunderbolt NAS with Apple M1 and Linux

#49

Post by FullyFunctional »

Funny I went down this rathole very shortly after discovering the "Thunderbolt networking" change in the Linux kernel. My hope was the use the Linux box (with 10 GbE) as a bridge/switch to give my Mac cheap 10 GbE access. While point-to-point did hit more or less the expected speed, bridging was horribly slow. Ended up just getting an Sonnet Solo10G SFP+ adapter (which is great).

Re: Ultra fast Thunderbolt NAS with Apple M1 and Linux

#50

Post by FullyFunctional »

yawniek wrote: post #33
whalesalad wrote: post #18

Just yesterday I was thinking how badly I want an obscenely fast local datastore for Docker. I am managing a handful of clients right now who are all using Docker containers for their microservice environments and my local machine runs out of disk daily. What I really want is lightning quick network storage, but I don't think it would be feasible to roll out 10g networking in my current rental home.

why don't you simply use one or many nvme thunderbolt/usb disks and move the caches there?

Not OP, but my files are shared with many hosts. Having local files are a PITA to manage coherent state.
Post reply on HN