This is great, would be useful to test scaling at some level too!
Localstack – Local AWS Emulator
111–120 of 132 posts
Re: Localstack – Local AWS Emulator
#112I’ve been saying for a couple of years whoever builds a tool to one click deploy an AWS alike to run on local bare metal has built the next billion dollar a year company. I know of a lot of big shops that are desperate to get out of the cloud due to 7-8 figure AWS bills but their software and engineering is too tightly wound into AWS tooling.
Need more details. Matching AWS service for service would be overkill I imagine, so what services does this need to have. Is hardware virtualisation is must-have. The way I made systems/applications before AWS existed was to create bootable images. I could already use a hypervisor, Xen, because the OS fully supported both guest and host, all virtualisation modes, before Linux did, and before AWS existed. But because…
> The ever-increasing complexity of AWS, including the "tooling", is, IMO, How To Create Lock-In 101
That is an interesting point. Complexity creates lock-in. Why? Because when you are interacting with a complex system you depend on it working in the complex ways it does. It is unlikely that anybody else could duplicate those features of the AWS your application is depending on.
This all runs counter to the idea of "encapsulation". You should be able to use a system via a well-defined interface. Once the interface is well-defined, other providers can provide their own implementation of the same interface.
So, AWS is basically bad software engineering, lacking encapsulation?
Re: Localstack – Local AWS Emulator
#113I’ve been saying for a couple of years whoever builds a tool to one click deploy an AWS alike to run on local bare metal has built the next billion dollar a year company. I know of a lot of big shops that are desperate to get out of the cloud due to 7-8 figure AWS bills but their software and engineering is too tightly wound into AWS tooling.
Nutanix is strong contender for this use case. We see lot of workloads moving from Cloud to onprem on nutanix stack.
P.S I work on engineering side in nutanix.
https://a16z.com/2021/05/27/cost-of-cloud-paradox-market-cap...
Re: Localstack – Local AWS Emulator
#114I’ve been saying for a couple of years whoever builds a tool to one click deploy an AWS alike to run on local bare metal has built the next billion dollar a year company. I know of a lot of big shops that are desperate to get out of the cloud due to 7-8 figure AWS bills but their software and engineering is too tightly wound into AWS tooling.
Eucalyptus built a clone. It wasn’t one click but it also wasn’t hard to deploy. It really never made much of a dent. I’m pretty sure it ended up where all software goes to die: HP.
The website is still up and points to a fork on GitHub:
- https://docs.eucalyptus.cloud/eucalyptus/5/install_guide/int...
- https://github.com/corymbia/eucalyptus/
There are quite a few moving parts. I think I got stuck around just comprehending the networking bits.
Re: Localstack – Local AWS Emulator
#115Localstack is our favorite for developing our application on AWS environment. It saves us from redeploys and AWS env based issues by simulating services on locally
Re: Localstack – Local AWS Emulator
#116Earlier quoted context omitted.
I worked on a system called Quartz at Bank of America. It wasn't an AWS clone, but was targeting the same flexibility for the development and deployment of applications on 'programmable' infrastructure. The team that built it had done similar projects before at J. P. Morgan and Goldmans. It was a set of infrastructure and services all accessed through Python APIs, with YAML config, all stored in a single distributed…
I would not be shocked if they just had some abstractions built around Ansible[0], it definitely fits the description [0]: https://docs.ansible.com/ansible/latest/user_guide/playbooks... Intro to playbooks - Ansible Documentation
It’s very different, much more like AWS or Azure. The underlying infra is there, you just allocate resources using config and APIs. For example there’s a server farm for running server applications such as batch processing or web applications. To stand up a new application you develop the app on a local instance of the Quartz Python runtime, check it into version control, then to deploy it by configuring a YAML file in version control that defines which server farm it runs on, the path to the code in VC, the schedule and various resource limits. That’s about it. Job done.
Re: Localstack – Local AWS Emulator
#117Earlier quoted context omitted.
As an Amazon engineer who frequently goes oncall, this is Amazon’s true competitive advantage. The company works because every night there’s 1000s of engineers sleeping with a pager by their head.
Why do that for general customer support instead of hiring in different time zones?
cultural cohesion?
we have often delayed calls because it's often hard to explain it to some "foreigner"
Re: Localstack – Local AWS Emulator
#118Hey all - LocalStack co-founder here. Great to see this article trending. :) Happy for any additional suggestions and questions you may have. Looking forward to getting your feedback!
Localstack Pro looks quite nice and comprehensive, but the CI pricing seems to really discourage proper continuous integration best practices. At 10 cents per run you're encouraging developers to run their CI pipelines as little as possible, while ideally you'd run CI on each commit and you'd commit often, probably 10s is times a day. In that case Localstack Pro for CI would get costly very quickly. Why not have CI p…
how did that 'mining coins on gh-actions' thing turned out again?
Re: Localstack – Local AWS Emulator
#119Earlier quoted context omitted.
But AWS is huge. They employ thousands of engineers. Granted most of the complexity is due to its huge huge scale, but still it has too many features to do what youre asking. And then there are AWS exclusives like DynamoDB. Yes there are open source public/private cloud alternatives like OpenStack but they won't solve the problem of lock in and you'd still have to change all your tooling.
I think it would not be impossible for a talented, well-funded group to pick the top 20% of AWS's services and then implement them with the top 80% of features, with an API-compatible layer. Even "just" to emulate a single region with 3 AZs would be useful for testing and for prod for many people. I know of several large organizations that to this day haven't scaled their CI beyond one region.
This makes sense; I would guess that 90% of AWS customers are only using the top 10% of available AWS features.
Same deal with Azure and GCP, although note that Azure has Azure Stack, which is a self-hosted version of Azure (I guess it's still expensive though, and I don't know how many services it includes).
Re: Localstack – Local AWS Emulator
#120Earlier quoted context omitted.
> they won't solve the problem of lock in Conflating lock in to an open source tool with lock in to a vendor who you literally have to keep paying money to survive is a shitty tactic used by companies like Amazon. Using open stack absolutely solves the problem of lock in from a business perspective. You can’t be cancelled, the price can’t be changed, etc.
Everybody is focusing on what happens on top of the CPU, but very few people talk about what happens beneath it. Datacenters are fucking expensive to build and run, buying network and compute hardware is a pain in the ass, maintaining sufficient capacity to sustain growth while operating on a 3-6 month deployment lead time is atrociously expensive, and that's without all of the financial calculation around depreciati…
This. My company is transitioning to the cloud for this reason.