Anyone know if this can be used directly from CloudFormation?
Not as useful if you're invested in CF, but Terraform can be better about getting new features. Just FYI.
AWS Transfer for SFTP – Fully Managed SFTP Service for Amazon S3
81–90 of 158 posts
Re: AWS Transfer for SFTP – Fully Managed SFTP Service for Amazon S3
#82I'd be curious how this handles all the posix cases not well suited to object storage. Renaming a folder than has a million files/folders inside is a single operation in SFTP, but 2 million operations on S3. Does it handle writing at arbitrary offsets within a file? Does it download the file first then let you start writing? What about just writing a few bytes at the beginning of a large existing file and then closin…
I don't work on the S3 team so I can't answer all these questions but relative to the "2 million operations on S3" question I can point out that S3 now has batch operation support: https://aws.amazon.com/about-aws/whats-new/2018/11/s3-batch-...
This is something that AWS needs to change. I work with millions of large files and have to keep a very large (PBs) local storage array just to make sure that things are right before uploading to S3 so that I don’t have to pay and wait for arch changes like this.
Re: AWS Transfer for SFTP – Fully Managed SFTP Service for Amazon S3
#83> SFTP (Secure File Transfer Protocol) It's SSH File Transfer Protocol. When you say Secure File Transfer Protocol many people think about FTP over SSL if you don't emphasize it's about SSH.
> many people think about FTP over SSL if you don't emphasize it's about SSH Huh? Sure there's always potential for confusion but every time I heard anything about FTP over SSL (which no one seems to actually use) it's been called "FTPS"
I agree FTPS is the right acronym for this but I had to correct people about this all the time. So many people actually have no idea SSH does more than just letting you execute command line programs on a remote server and FTP is not the only/best protocol to access remote file systems over the Internet.
Re: AWS Transfer for SFTP – Fully Managed SFTP Service for Amazon S3
#84Charged by the hour... pay for an instance.. Tsk tsk.. The thing I love about S3 and cloud services in general is when I pay per request and can scale through the roof. Whenever a services is meter by number of instances my interest fades, and I look for other solutions.. S3 has this very handsoff feeling to it :)
AWS should investigate a new concept called serverless.
Re: AWS Transfer for SFTP – Fully Managed SFTP Service for Amazon S3
#85Presumably this will handle large file uploads with aplomb? Multipart upload with s3 can be a pain (when you want someone else to be doing the uploading).
Re: AWS Transfer for SFTP – Fully Managed SFTP Service for Amazon S3
#86Earlier quoted context omitted.
The AWS pricing page for this service says it costs about $225/month for a lightly used instance. I implemented the same kind of thing on AWS using a nano-sized instance for about $10/month. The instance is managed with an Ansible Role for automated SFTP server management. I connected it with an off-the-shelf AWS Lamda function which listens for S3 PUT events and copies files to the SFTP server as needed. My solution…
$200 a month is nothing for a business. Anything that we don't have to manage ourselves or worry about reliability, scalability, and we can just use our AWS business support plan is a win.
In large business, sure. In small business and education, a $200/mo commitment could easily require approval by the owner or the department chair.
Re: AWS Transfer for SFTP – Fully Managed SFTP Service for Amazon S3
#87Earlier quoted context omitted.
$200 a month is nothing for a business. Anything that we don't have to manage ourselves or worry about reliability, scalability, and we can just use our AWS business support plan is a win.
In large business, sure. In small business and education, a $200/mo commitment could easily require approval by the owner or the department chair.
The alternative is developer time. Nothing about managed services is ever less expensive if you don’t account for developer/Devops/netops time saved.
A small company has even more of reason to want as many managed services as possible. You can avoid hiring netops if you both have a third party managed service provider to manage your network and you have developers/architects who know enough to fill in the gaps.
Re: AWS Transfer for SFTP – Fully Managed SFTP Service for Amazon S3
#88Charged by the hour... pay for an instance.. Tsk tsk.. The thing I love about S3 and cloud services in general is when I pay per request and can scale through the roof. Whenever a services is meter by number of instances my interest fades, and I look for other solutions.. S3 has this very handsoff feeling to it :)
Well, its likely because the sftp server itself is always running. If they had an sftp lambda function... sure.
Re: AWS Transfer for SFTP – Fully Managed SFTP Service for Amazon S3
#89Anyone know if this can be used directly from CloudFormation?
New services rarely launch with CF support, if you want to programmatically create SFTP servers TODAY you could write a Lambda that uses the SDK and reference that Lambda with a CF Custom Resource.
Re: AWS Transfer for SFTP – Fully Managed SFTP Service for Amazon S3
#90Earlier quoted context omitted.
That’s quite literally AWS’s playbook. Fill the vendor hall with tech, learn it, see what sticks and then crush it. This is why I’m bullish on cloud agnostic tech. These practices don’t typically fair well in the enterprise space. This is why companies like MSFT are interesting to me. They partner and rarely kill. Amazon is the complete opposite.
You can't even sell merchandise on Amazon without significant risk they will like your numbers enough to muscle you out: https://www.amazon.com/AmazonBasics/b?node=10112675011
Even apart from AmazonBasics, if you've found established a niche product that sells well as a merchant on Amazon, you can be sure that Amazon will swoop in and undercut you ASAP.