Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Amazon Elastic File System

aws.amazon.com

Post reply on HN
Print view View original

171–180 of 217 posts

Re: Amazon Elastic File System

#171

Post by goldenkey »

jonahx wrote: post #156
chuckcode wrote: post #16

I wish AWS would be a little more technical in their product descriptions and announcements. If there was ever an audience that wants technical data about how this type of technology will scale and compares to existing technology like glusterfs it would be the AWS users. Instead the performance specifications are ".. and is designed to provide the throughput, IOPS, and low latency needed for a broad range of workload…

As a developer (but not a sysadmin) I feel the opposite. Not saying your criticism is wrong, just giving another view from a AWS newbie perspective. When I read most AWS product descriptions, I cannot understand what real world situations they are for. I either could not say, or could only vaguely say, what the product is in my own words. There seems to be a great scaffolding of assumed knowledge about the AWS system…

heroku uses ec2 under the hood. heroku is heresy for any competent programmer. despite aws being packaged services at least both managed db and ec2 show you your actual servers and let you choose specs.

Re: Amazon Elastic File System

#172

Post by sciurus »

seanwilson wrote: post #160
jonahx wrote: post #156

Earlier quoted context omitted.

As a developer (but not a sysadmin) I feel the opposite. Not saying your criticism is wrong, just giving another view from a AWS newbie perspective. When I read most AWS product descriptions, I cannot understand what real world situations they are for. I either could not say, or could only vaguely say, what the product is in my own words. There seems to be a great scaffolding of assumed knowledge about the AWS system…

I feel similarly about AWS and I've been using it for a while. AWS is very powerful and flexible but the number of options and different services can be bewildering. I've suggested AWS to several developers I know that have never used it before and they were turned off from not being able to understand what was being offered and how much it would cost them. For example, compare the Heroku pricing page and the AWS pri…

I think this just illustrates your point, but EC2 and Heroku Dynos are two different types of services (IAAS vs PAAS). The appropriate comparisons are AWS Elastic Beanstalk to Heroku Dynos and AWS RDS to Heroku Postgres.

https://aws.amazon.com/elasticbeanstalk/

https://aws.amazon.com/rds/

Re: Amazon Elastic File System

#173

Post by sciurus »

tracker1 wrote: post #145
mrmondo wrote: post #133

Earlier quoted context omitted.

Until you see technical details on any file, network or storage system you should always assume your writes (or data for that matter) are/is not safe. It's disappointing to see a product launched and hyped without the appropriate details required to make an informed decision around its use.

I think that you are correct, that some technical details are definitely needed for comparison... To me, this matches up against Azure Files (which is CIFS/SAMBA based)... Though, even if redundancy isn't a factor, it's nice that you can have network file shares without having to run your own dedicated instance in a given cloud. There are plenty of situations where having a common networked filesystem makes sense acr…

Aren't seeding static content and user-uploaded files both use-cases that are well-handled by an object store like S3?

Re: Amazon Elastic File System

#174

Post by seanwilson »

sciurus wrote: post #172
seanwilson wrote: post #160

Earlier quoted context omitted.

I feel similarly about AWS and I've been using it for a while. AWS is very powerful and flexible but the number of options and different services can be bewildering. I've suggested AWS to several developers I know that have never used it before and they were turned off from not being able to understand what was being offered and how much it would cost them. For example, compare the Heroku pricing page and the AWS pri…

I think this just illustrates your point, but EC2 and Heroku Dynos are two different types of services (IAAS vs PAAS). The appropriate comparisons are AWS Elastic Beanstalk to Heroku Dynos and AWS RDS to Heroku Postgres. https://aws.amazon.com/elasticbeanstalk/ https://aws.amazon.com/rds/

Yes, a link to Elastic Beanstalk would be a fairer comparison of what is being offered by Heroku but my point was most AWS pages are like that and aren't easy to follow, especially if you're new to AWS.

Re: Amazon Elastic File System

#175

Post by karpodiem »

tracker1 wrote: post #152
616c wrote: post #132

Earlier quoted context omitted.

> - (Not mentioned) Both Linux and Windows have built-in NFS clients. Does anyone use the NFS for Windows? Is it reliable enough I can move servers away from CIFS alogether if I want to access a Linux box from Windows. For instance if I click the folder's Properties in Windows, will I see the proper Unix perms and metadata and be able to edit them?

FYI: In windows (desktop versions) it's only now in the "Enterprise" version, and apparently will be removed in Windows 10. Not sure if there are limitations to which server versions are currently supported. CIFS/SAMBA in Linux has been pretty good, imho for the past couple years... though I still get some occasional wonky behavior from my NAS box.

Where have you heard that they are removing the NFS client in Win10?

Re: Amazon Elastic File System

#176

Post by wpietri »

rjurney wrote: post #129
vially wrote: post #94

Earlier quoted context omitted.

Would you mind explaining how exactly "Apple consistently leads and innovates ahead of other companies and open source"?

Are you kidding? Apple reinvents every space they enter with profound invention. Do you remember computers before the Mac? Notebooks before the Macbook? Music players before the iPod? Phones before the iPhone? Tablets before the iPad? And in a month you'll think: watches before the Apple Watch?

I think their skill is more profound commercialization. I do in fact remember all of those things. I even owned all of those things before Apple entered those markets. Apple invented very little for any of them.

Their strength lies in taking existing technologies, rebuilding them with a strong user focus, and then marketing the hell out of them. So much so that many people apparently forget what came before.

Re: Amazon Elastic File System

#177

Post by chuckcode »

jonahx wrote: post #156
chuckcode wrote: post #16

I wish AWS would be a little more technical in their product descriptions and announcements. If there was ever an audience that wants technical data about how this type of technology will scale and compares to existing technology like glusterfs it would be the AWS users. Instead the performance specifications are ".. and is designed to provide the throughput, IOPS, and low latency needed for a broad range of workload…

As a developer (but not a sysadmin) I feel the opposite. Not saying your criticism is wrong, just giving another view from a AWS newbie perspective. When I read most AWS product descriptions, I cannot understand what real world situations they are for. I either could not say, or could only vaguely say, what the product is in my own words. There seems to be a great scaffolding of assumed knowledge about the AWS system…

I completely agree that the AWS descriptions and documentations are written with the style that actual best practices are left as an exercise for the reader. Without much nurturing the community is left to develop solutions like boto-rsync for moving files into S3 and glusterfs to overcome a 1Tb limit on drives. I'm not sure if that is intentional or a side effect of something else but it certainly turns off a lot of newbies for whom it seems overwhelming and a lot of technical folks as it is difficult to get a grasp on exactly what to use and how in AWS.

Re: Amazon Elastic File System

#178

Post by kondro »

andrew311 wrote: post #138

EFS is a great addition to AWS. We have SAN as a service via EBS, now we get NFS as a service. Great. The question (for me) now becomes "where do we go from here?" Infinite NFS is great, but what I've always wanted is infinite EBS that is fully integrated from file system to SAN. In other words, something that behaves like a local file system (without the gotchas of NFS like a lack of delete on close), but I don't ha…

http://aws.amazon.com/storagegateway/ is a little like what you require (if you're unfamiliar).

It's not unlimited, but currently its 32TB/volume and you're charged for the storage you use, rather than what is provisioned.

It supports encryption at rest.

Re: Amazon Elastic File System

#179

Post by andrew311 »

rgbrenner wrote: post #170
andrew311 wrote: post #138

EFS is a great addition to AWS. We have SAN as a service via EBS, now we get NFS as a service. Great. The question (for me) now becomes "where do we go from here?" Infinite NFS is great, but what I've always wanted is infinite EBS that is fully integrated from file system to SAN. In other words, something that behaves like a local file system (without the gotchas of NFS like a lack of delete on close), but I don't ha…

something that behaves like a local file system (without the gotchas of NFS like a lack of delete on close), but I don't have to snapshot and create new volumes and issue file system expansion commands to grow a volume. I want seamless and automatic growth. Wouldn't EBS w/ thin provisioning get you most of this? Just create a massive volume, and you get billed for the space actually used. (and the volume size could a…

Yes, good point. This would provide effectively "infinite" backing storage. There might be some hurdles to overcome, though. For example, when you delete a file, will EBS know that the blocks are now free and thus can be decommissioned? This might mean the whole stack needs to support things like TRIM. I'm not sure the rest of the stack is smart enough yet. I'd love to hear from a storage/FS expert on this.

Edit: coincidentally, I just saw this article about XFS which observes the following:

"Over the next five or more years, XFS needs to have better integration with the block devices it sits on top of. Information needs to pass back and forth between XFS and the block device, [says Dave Chinner]. That will allow better support of thin provisioning." https://lwn.net/Articles/638546/

Re: Amazon Elastic File System

#180

Post by derefr »

seanwilson wrote: post #174
sciurus wrote: post #172

Earlier quoted context omitted.

I think this just illustrates your point, but EC2 and Heroku Dynos are two different types of services (IAAS vs PAAS). The appropriate comparisons are AWS Elastic Beanstalk to Heroku Dynos and AWS RDS to Heroku Postgres. https://aws.amazon.com/elasticbeanstalk/ https://aws.amazon.com/rds/

Yes, a link to Elastic Beanstalk would be a fairer comparison of what is being offered by Heroku but my point was most AWS pages are like that and aren't easy to follow, especially if you're new to AWS.

The "you" in the comparison changes as well, though. Indeed, the copy for (say) EC2 isn't comprehensible to the same people that the copy for Elastic Beanstock is. But different people will be reading the copy for those two things. The person AWS is selling EC2 to isn't a newbie developer wanting to run their app; it's someone wanting to build out something like Heroku: someone with lots of system-architecture experience.

Or, to put it more simply: the intended audience for AWS ad copy is exactly the set of people who are currently trying to compare some specific AWS service to similar services of competitors. AWS is never trying to "create a need" for something nobody was already asking for; they're just trying to serve the needs people already do have better/cheaper/more flexibly.

Post reply on HN