Ask HN: How do you manage large files across multiple laptops/desktops?

#1

Post by kaiku » Wed, Mar 04, 2020, 8:03 PM UTC

My wife is a graphic designer and works on both a Mac desktop and laptop. The assets she generates for a single project can reach into gigabyte territory – multiple versions of a set of print layouts, RAW photos, etc. One problem she has is how to reliably sync large files between computers. She uses Dropbox currently, but sync over network can takes a long time or sometimes just doesn't finish depending on the file. And when she runs into space limits on one computer, she'll need to adjust Dropbox settings to selectively sync items.

I'm curious what people do in this case. I think she needs:

1) A large amount of storage, expandable as time goes on 2) Files accessible from multiple computers 3) Ability to access files securely when working from a coffee shop, traveling, etc.

Re: Ask HN: How do you manage large files across multiple laptops/desktops?

#2

Post by codingdave » Wed, Mar 04, 2020, 8:31 PM UTC

I use an external USB drive. You can get anywhere from the standard thumb drives, up to multi-terabyte larger drives. So when travelling I just take whichever drive I need and whichever laptop I need, and plug them together.

It isn't exactly syncing, but it solves the problem, at least for me.

Re: Ask HN: How do you manage large files across multiple laptops/desktops?

#3

Post by therealmarv » Wed, Mar 04, 2020, 8:37 PM UTC

Buy a large SSD like 2-8TB (just avoid the Samsung 860 QVO 4TB, it sucks more power during write than USB A standard allows) and buy a nice case like https://www.amazon.com/dp/B07D2BHVBD and mount it there. Work from the external SSD, it's fast enough in my opinion. Use the cloud only for backup purposes whenever the drive is attached (I use Insync for that but this is Google Drive centric).

Re: Ask HN: How do you manage large files across multiple laptops/desktops?

#4

Post by jedieaston » Wed, Mar 04, 2020, 8:53 PM UTC

Something that isn’t uncommon in some shops is using a big system/VM where the person does their work and just remotes into it from their mobile system as needed, so the file only actually is in one place.

Of course, it only works if you’re online, and VDI-ish solutions aren’t really a thing on OS X, so YMMV, but I remote into my desktop while I’m at school because my laptop doesn’t have much ram.

Re: Ask HN: How do you manage large files across multiple laptops/desktops?

#5

Post by pid_0 » Wed, Mar 04, 2020, 9:10 PM UTC

icloud, otherwise external storage

Re: Ask HN: How do you manage large files across multiple laptops/desktops?

#6

Post by ogou » Wed, Mar 04, 2020, 9:12 PM UTC

One thing Dropbox does well is transparently resuming a stalled or failed sync. The other solutions I can think of are more manual or the UX is very basic or obtuse. You could set up your own storage space on AWS and rsync (which has GUI tools available), but that's essentially roll-your-own Dropbox.

I work on large video files that I transport between locations and had an external drive that I used for a while but would end up forgetting or not wanting to unplug everything. I have settled on a high speed 64gb SD card that fits in any pocket or even my wallet. Out of all the fancy network tools I set up, this option has proved to be the most enduring.

Doesn't help with synchronizing which files are most current though. You might look at Transmit 5 from Panic. It has lots of useful syncing features now.

Re: Ask HN: How do you manage large files across multiple laptops/desktops?

#7

Post by kop316 » Wed, Mar 04, 2020, 9:27 PM UTC

Have you looked into Nextcloud?

If you primarily do it at home, the speeds of Nextxloud are typically limited by the network bandwidth, and the storage is limited by what server you have. I use it and manage 5 TB of data on it without problems.

Re: Ask HN: How do you manage large files across multiple laptops/desktops?

#8

Post by anonsivalley652 » Wed, Mar 04, 2020, 9:54 PM UTC

Try them all and then pick one (or two):

- SpiderOak One

- Mega (very cheap)

- Syncthing (open source)

Re: Ask HN: How do you manage large files across multiple laptops/desktops?

#9

Post by mceachen » Wed, Mar 04, 2020, 9:59 PM UTC

There are already a lot of suggestions to use an externally attached storage (or "sneakernet"). I think it's the simplest solution with best performance. I dual-boot off an EVO SSD in an external thunderport/USB-C enclosure, and performance is similar to the built-in HD. The enclosure is the size of a pack of gum. Make sure the USB-C to USB-C cable is actually USB 3.0 certified.

She should also verify her Time Machine configurations on both Macs are set up to back up her sneakernet drive fairly frequently. ... You only seem to need backups when you don't have them...

Re: Ask HN: How do you manage large files across multiple laptops/desktops?

#10

Post by anonsivalley652 » Wed, Mar 04, 2020, 10:00 PM UTC

therealmarv wrote: post #3

Buy a large SSD like 2-8TB (just avoid the Samsung 860 QVO 4TB, it sucks more power during write than USB A standard allows) and buy a nice case like https://www.amazon.com/dp/B07D2BHVBD and mount it there. Work from the external SSD, it's fast enough in my opinion. Use the cloud only for backup purposes whenever the drive is attached (I use Insync for that but this is Google Drive centric).

A Mac Mini expanded with two SSDs with syncthing is superior to sneakernet because it provides replication. And, if you really want to, add a 14 TiB HC520 placed in an external USB 3 case.

Add an rsync server to get backup. Forget the cloud, most solutions are overpriced and insecure. Use TarSnap if you have absolute have to backup reasonable quantities of critical data and call it a day.