Seems like a service that should be offered by AWS.
Localstack – Local AWS Emulator
121–130 of 132 posts
Re: Localstack – Local AWS Emulator
#122I prefer mocking my AWS calls, so much more lightweight and means exert developer doesn’t have to install local stack
Interesting! Staying lightweight and zero-dependency is certainly a good argument. Out of curiosity - which programming language(s) are you / your team developing in? Do you have a notion of how much (%) of your time, approximately, goes into maintaining the mocks - does it scale / pay off over time?
We use Typescript and jest for the mocks. Probably about 10%, they’re very simple mocks, all they do is return success and in a few unit tests we make them return failure and then test how our app responds.
I can’t really say if it pays off but it has caught some issues, would like to expand them more and implement more logic in then to check arguments etc but haven’t been able to yet.
This article is similar to what we’ve done
https://dev.to/elthrasher/mocking-aws-with-jest-and-typescri...
We also have a pipeline that does e2 work tests and canary releases to hopefully limit the blast radius. If we encounter >10 errors in a 1 minute period we roll back and the team are alerted
Re: Localstack – Local AWS Emulator
#123Earlier quoted context omitted.
> However for my own projects, we consider local development like this an anti-pattern. IaC tools like Terraform/CDK/SST make it easy to spin up environments for each developer in the cloud. But what do you test your IAC against? If you consider using something like Terratest, then combining it with LocalStack means you can run Terratest nearly instantly and test your IAC. Plus developers being able to run "AWS" loca…
We test it against real AWS deployments. With a proper multi-account strategy + SSO, there are no AWS credentials to leak + blast radius is limited. The feedback loop is definitely a problem but projects like SST return the instant feedback loop to you.
The credentials are the API keys and secrets, not the login and passwords.
One improper setting in IAM and an accidental pastebin or GitHub means half the automated hacks have access to the same thing the IAM user does.
I did this once on accident by hitting control-V instead of control-c and overwriting my censored version before pasting into GitHub so other people could save the 4 hours of tedious scripting I had to do. GitHub sends an email within minutes, but I was already miles away from a computer when I got the email...
Re: Localstack – Local AWS Emulator
#124Earlier quoted context omitted.
But AWS is huge. They employ thousands of engineers. Granted most of the complexity is due to its huge huge scale, but still it has too many features to do what youre asking. And then there are AWS exclusives like DynamoDB. Yes there are open source public/private cloud alternatives like OpenStack but they won't solve the problem of lock in and you'd still have to change all your tooling.
> they won't solve the problem of lock in Conflating lock in to an open source tool with lock in to a vendor who you literally have to keep paying money to survive is a shitty tactic used by companies like Amazon. Using open stack absolutely solves the problem of lock in from a business perspective. You can’t be cancelled, the price can’t be changed, etc.
Exactly, open source solves the problem with vendor lock-in in your tooling. It's more needed than ever now that the world is moving towards proprietary SaaS black boxes.
Re: Localstack – Local AWS Emulator
#125I’ve been saying for a couple of years whoever builds a tool to one click deploy an AWS alike to run on local bare metal has built the next billion dollar a year company. I know of a lot of big shops that are desperate to get out of the cloud due to 7-8 figure AWS bills but their software and engineering is too tightly wound into AWS tooling.
I worked on a system called Quartz at Bank of America. It wasn't an AWS clone, but was targeting the same flexibility for the development and deployment of applications on 'programmable' infrastructure. The team that built it had done similar projects before at J. P. Morgan and Goldmans. It was a set of infrastructure and services all accessed through Python APIs, with YAML config, all stored in a single distributed…
Wow, my brother works on Quartz. But I think that half of BofA does. Higgins and Kirat formed their own company to bring the Quartz/Athena paradigm to the masses.
Re: Localstack – Local AWS Emulator
#126I’ve been saying for a couple of years whoever builds a tool to one click deploy an AWS alike to run on local bare metal has built the next billion dollar a year company. I know of a lot of big shops that are desperate to get out of the cloud due to 7-8 figure AWS bills but their software and engineering is too tightly wound into AWS tooling.
But why? Because they can run an on-prem production "AWS"? Or because it's easier for testing? If the former, well, sure, but why stop there? Why not an easy on-prem Heroku clone? If the latter, I don't think that goes far enough. Simply emulating AWS (as LocalStack does) is barely usable for testing. You really would want a GUI that lets you see everything in the LocalStack config and data stores. Running a fake DynamoDB is okay, but it's painful if you need to use DynamoDB queries to see the data.
Re: Localstack – Local AWS Emulator
#127Localstack is great but I wish it wasn't needed. Engineers have become too comfortable stringing cloud services together with minimal code in a way that makes them unable to be run locally. This is also a hard problem to fix after the fact. If you make a mock of the cloud service, how do you know you captured all of the functionality? How do you know your mock won't pass all of the tests, but the real cloud server will behave differently and break production? Greenfield you can anticipate this and explicitly test all behavior locally.
Re: Localstack – Local AWS Emulator
#128Earlier quoted context omitted.
> However for my own projects, we consider local development like this an anti-pattern. IaC tools like Terraform/CDK/SST make it easy to spin up environments for each developer in the cloud. But what do you test your IAC against? If you consider using something like Terratest, then combining it with LocalStack means you can run Terratest nearly instantly and test your IAC. Plus developers being able to run "AWS" loca…
We test it against real AWS deployments. With a proper multi-account strategy + SSO, there are no AWS credentials to leak + blast radius is limited. The feedback loop is definitely a problem but projects like SST return the instant feedback loop to you.
And I think this can be a good idea, but only sometimes. Mostly I think you should just use LocalStack.
Re: Localstack – Local AWS Emulator
#129I’ve been saying for a couple of years whoever builds a tool to one click deploy an AWS alike to run on local bare metal has built the next billion dollar a year company. I know of a lot of big shops that are desperate to get out of the cloud due to 7-8 figure AWS bills but their software and engineering is too tightly wound into AWS tooling.
So are you saying that the tooling needs to deploy to their datacenters or AWS, or just deploying to their own datacenters needs to be as simple as using AWS? If it’s both, use Kubernetes. If it’s deploying to their own datacenters, then maybe a new product could disrupt that space. You would still need to have a team competent in datacenter operations and hardware procurement though.
Re: Localstack – Local AWS Emulator
#130Earlier quoted context omitted.
As an Amazon engineer who frequently goes oncall, this is Amazon’s true competitive advantage. The company works because every night there’s 1000s of engineers sleeping with a pager by their head.
Why do that for general customer support instead of hiring in different time zones?
Because minutes after a metric drops that indicates the possibility of an issue an engineer who probably wrote the code or has the code checked out on their laptop and is intimately familiar with it working on the problem. No waiting for a support team to triage, find the owner, and get ahold of them.