Amazon Elastic Container Service now supports Amazon EFS file systems
1–10 of 31 posts
Re: Amazon Elastic Container Service now supports Amazon EFS file systems
#2FINALLY!!!
edit: thanks a lot ECS team
Re: Amazon Elastic Container Service now supports Amazon EFS file systems
#3This is great news!
Yes, these containers are supposed to be stateless, but I was tasked with converting an app at my previous job over to using ECS on Fargate and we hit so many issues because of the limits on storage per container instance. We ended up having to tweak the heck out of nginx caching configurations and other processes that would generate any "on disk" files to get around the issues. Having EFS available would have made solving some of those problems so much easier.
I've also been wanting to use ECS on Fargate for running scheduled tasks with large files (50gb+) but it wasn't really possible given the previous 4gb limit on storage.
Re: Amazon Elastic Container Service now supports Amazon EFS file systems
#4This is the single biggest blocker to running something like Postfix in ECS. This is a huge, huge win.
Re: Amazon Elastic Container Service now supports Amazon EFS file systems
#5This is great news! Yes, these containers are supposed to be stateless, but I was tasked with converting an app at my previous job over to using ECS on Fargate and we hit so many issues because of the limits on storage per container instance. We ended up having to tweak the heck out of nginx caching configurations and other processes that would generate any "on disk" files to get around the issues. Having EFS availab…
containers shouldn't necessarily be stateless; most existing codes don't know how or want to talk to services via RPC interfaces. In some sense, a mounted remote filesystem is just a standard API the OS provides you to access state in a convenient way that happens to be high performance, indexed, etc.
Re: Amazon Elastic Container Service now supports Amazon EFS file systems
#6This is going to make a lot of container workloads that were possible, but inconvenient to setup, suddenly trivial to deploy. Very nice!
Re: Amazon Elastic Container Service now supports Amazon EFS file systems
#7How's the performance on EFS? Has anyone used it in production that is willing to share their experience?
We evaluated it for a relatively simple use case, and the performance seemed abysmal, so we didn't select it. I'm hoping that we made a mistake in our evaluation protocol, which would give me an excuse to give it another try.
Re: Amazon Elastic Container Service now supports Amazon EFS file systems
#8How's the performance on EFS? Has anyone used it in production that is willing to share their experience? We evaluated it for a relatively simple use case, and the performance seemed abysmal, so we didn't select it. I'm hoping that we made a mistake in our evaluation protocol, which would give me an excuse to give it another try.
It's terrible. Very slow when we tried to use it. There are ways to work around this, and ways to tune the performance, but honestly it was not worth it for our use case and instead we found a way to make EBS work.
EFS is a great way to get a lot of iowait on your cpu graphs. Would not recommend it for anything that had to be fast.
Re: Amazon Elastic Container Service now supports Amazon EFS file systems
#9How's the performance on EFS? Has anyone used it in production that is willing to share their experience? We evaluated it for a relatively simple use case, and the performance seemed abysmal, so we didn't select it. I'm hoping that we made a mistake in our evaluation protocol, which would give me an excuse to give it another try.
It's terrible. Very slow when we tried to use it. There are ways to work around this, and ways to tune the performance, but honestly it was not worth it for our use case and instead we found a way to make EBS work. EFS is a great way to get a lot of iowait on your cpu graphs. Would not recommend it for anything that had to be fast.
> we found a way to make EBS work.
Can you say more about what you did with EBS? It seems like it would be necessary to make some compromises in availability and disaster recovery because any given EBS volume is restricted to the availability zone where it was created.
Re: Amazon Elastic Container Service now supports Amazon EFS file systems
#10How's the performance on EFS? Has anyone used it in production that is willing to share their experience? We evaluated it for a relatively simple use case, and the performance seemed abysmal, so we didn't select it. I'm hoping that we made a mistake in our evaluation protocol, which would give me an excuse to give it another try.
It is highly dependent on your needs. It's NFS, and performs accordingly (though EFS has been rock solid in a few different scenarios from an availability standpoint and a baseline performance, assuming you use dedicated IOPS).
Should you run a database on EFS? No. Can you use it to back media files for a web application that are cached using a CDN, or for data files used for processing or temporary storage? Yes, it shines in those use cases... and it's cheaper than dedicating the time required to maintain your own NFS cluster.
Even Gluster or Ceph is, IMO, not worth the effort unless you (a) know how to run and maintain it, and (b) absolutely need the potential speed up that you can get, assuming a well-configured and well-maintained system.