AWS S3 open source alternative written in Go
91–100 of 139 posts
Re: AWS S3 open source alternative written in Go
#92Re: AWS S3 open source alternative written in Go
#93Earlier quoted context omitted.
Ran this in production and dealt with a lot of issues. I would caution people against it's use in anything critical or customer facing.
Could you please elaborate it? What were the issues you were facing?
- The restart times of the Riak process ranged from 10 minutes to 3+ hours, during which time the cluster was basically useless. Not a single suggestion from support sped up this process.
- Every single night from 0800 - 0900 UTC, the cluster would grind to a halt (as measured by canaries measuring upload/download cycle times). This continued even after we migrated all customer data and traffic off of the cluster.
- Riak-CS ships with garbage collection disabled despite it being a critical feature. I inherited a cluster that had been run for some months without gc enabled. Turning it on caused the cluster to catastrophically fail. Basho Support, over a period of close to a year, was unable to find a single solution that would get our cluster back to health. If our cluster were a house on a show like Hoarders, the garbage in it would be considered load bearing.
- We attempted to upgrade our way out of our un-garbage-collect-able mess, but the transfer crashed. Every. Single. Time.
- Even had transfers worked, all of the bloated manifests have to be copied in their entirety, so you can't gc the incoming data on the new cluster.
- Even while babying the cluster, it would become unusable at least once a month, requiring a restart of all nodes. The slowest node took 3+ hours to start, followed by another 3+ hours of transferring data. This was 6+ hours of system downtime every month.
- During these monthly episodes, we attempted to engage with support and try to debug the processes (we were a team of seasoned Erlang developers). We could attach Observer and/or use the REPL to grab stats, but not a single support resource was able or willing to engage.
- For giggles, once we had migrated all users off of the cluster, we attempted to let gc run. It never completed. Not once. We let this go on for a few months before nuking the entire cluster.
Now, I absolutely realize that we got ourselves into that mess by running the cluster without gc for an extended period. But in the grand scheme of things, this cluster wasn't storing a very large amount of data -- tens of TB spread over tens of millions of objects. Having the cluster get into a state where gc can never run and where this causes snowballing instability is unacceptable.
We switched to Ceph. We've never looked back.
Re: AWS S3 open source alternative written in Go
#94How easy is it to embed this into go tests? Right now I use goamz/s3test for that, but it has a lot of limitations.
Re: AWS S3 open source alternative written in Go
#95Re: AWS S3 open source alternative written in Go
#96Or, run Riak with their S3 compatibility layer. Riak is extremely stable and the work Basho has done to make a truly robust distributed database is significant. http://docs.basho.com/riak/cs/2.1.1/
Re: AWS S3 open source alternative written in Go
#97I love the website. I'm a lone developer who doesn't know any HTML, how would I go about getting such a nice design for my own projects? (Or how much would it cost)
Re: AWS S3 open source alternative written in Go
#98Earlier quoted context omitted.
Ran this in production and dealt with a lot of issues. I would caution people against it's use in anything critical or customer facing.
Could you please elaborate it? What were the issues you were facing?
Re: AWS S3 open source alternative written in Go
#99I love the website. I'm a lone developer who doesn't know any HTML, how would I go about getting such a nice design for my own projects? (Or how much would it cost)
Wappalyzer ( https://wappalyzer.com/ ) tells me they're using Bootstrap ( http://getbootstrap.com/ ) (probably customized a bit). HTML isn't very difficult (just another markup language) and if you're not inclined toward design (I am also not) there are a plethora of CSS frameworks to choose from (like Bootstrap) that will get you up and running with something not completely ugly. Personally I like Bulma ( http://bul…
Re: AWS S3 open source alternative written in Go
#100Earlier quoted context omitted.
Wappalyzer ( https://wappalyzer.com/ ) tells me they're using Bootstrap ( http://getbootstrap.com/ ) (probably customized a bit). HTML isn't very difficult (just another markup language) and if you're not inclined toward design (I am also not) there are a plethora of CSS frameworks to choose from (like Bootstrap) that will get you up and running with something not completely ugly. Personally I like Bulma ( http://bul…
Really my design sense isn't great, given time I can hack together something with bootstrap, but I do think I lack the designer training and probably instincts