Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Glusterfs or ceph on Digitalocean block storage?

news.ycombinator.com

Post reply on HN
Print view View original

1–9 of 9 posts

Ask HN: Glusterfs or ceph on Digitalocean block storage?

#1

Post by jaddison »

I'm curious to hear of experiences running distributed filesystems like glusterfs or ceph (others too?) on Digitalocean's block storage.

I've got a side project that will be continuously adding many 50kb to 20mb files, so easy expansion is necessary.

Any "at-scale" (ie. production, under real load) performance reports/benchmarks appreciated.

Reliability, recovery from node failure, etc comments welcome.

Re: Ask HN: Glusterfs or ceph on Digitalocean block storage?

#5

Post by taisgeal »

Don't use either. I tried and failed with both. Expensive hardware and a PhD required to go through all the config options. When I did get both to work, they were very slow and I kept getting replication issues and bricks that wouldn't heal.

This cost two months of my life and I was not flavur of the month with my employer.

I eventually tried lsyncd ( https://axkibe.github.io/lsyncd/) and it works like a charm. It took me about a week to get a decent setup going, but it has never let me down since, can handle large numbers of small files (similar to yours) and smaller numbers of large files. I use unison ( https://www.howtoforge.com/tutorial/unison-file-sync-between...) for web server pairs.

Re: Ask HN: Glusterfs or ceph on Digitalocean block storage?

#6

Post by taisgeal »

taisgeal wrote: post #5

Don't use either. I tried and failed with both. Expensive hardware and a PhD required to go through all the config options. When I did get both to work, they were very slow and I kept getting replication issues and bricks that wouldn't heal. This cost two months of my life and I was not flavur of the month with my employer. I eventually tried lsyncd ( https://axkibe.github.io/lsyncd/ ) and it works like a charm. It t…

and I am also Digital Ocean customer. It has meant I can completely remove a whole storage layer and has reduced my droplet bill overall by 20%.

Re: Ask HN: Glusterfs or ceph on Digitalocean block storage?

#8

Post by jaddison »

taisgeal wrote: post #5

Don't use either. I tried and failed with both. Expensive hardware and a PhD required to go through all the config options. When I did get both to work, they were very slow and I kept getting replication issues and bricks that wouldn't heal. This cost two months of my life and I was not flavur of the month with my employer. I eventually tried lsyncd ( https://axkibe.github.io/lsyncd/ ) and it works like a charm. It t…

Unless I'm misunderstanding, this isn't what I'm looking for. I'm looking for expandable storage - the ability to add GB/TB by adding disks. Replication is important, yes, but if I can't grow the volume, I'll just plain cry.

Re: Ask HN: Glusterfs or ceph on Digitalocean block storage?

#9

Post by taisgeal »

jaddison wrote: post #8
taisgeal wrote: post #5

Don't use either. I tried and failed with both. Expensive hardware and a PhD required to go through all the config options. When I did get both to work, they were very slow and I kept getting replication issues and bricks that wouldn't heal. This cost two months of my life and I was not flavur of the month with my employer. I eventually tried lsyncd ( https://axkibe.github.io/lsyncd/ ) and it works like a charm. It t…

Unless I'm misunderstanding, this isn't what I'm looking for. I'm looking for expandable storage - the ability to add GB/TB by adding disks. Replication is important, yes, but if I can't grow the volume, I'll just plain cry.

You can add storage by expanding the lsyncd droplet(remote) or by using Block Storage(local) on your web droplets and expanding that.

I have found this to be more cost effective than anything else I have tried.

Post reply on HN