Earlier quoted context omitted.
I was a guy that built server clusters during the early 00's, for my own and others' web and other projects. When AWS really took off, it was like a spend all your money mania, and devs and companies treated my skills like dirt. I got a job writing facial recognition edge servers, with high performance many claim are impossible numbers (25M face compares per second per core) and my employer found itself a leader in t…
Do you have any blog posts or something you could share on how facial recognition works? Specifically what a "face compare" is. That sounds interesting!
I took all my projects off the cloud, saving thousands of dollars
361–370 of 406 posts
Re: I took all my projects off the cloud, saving thousands of dollars
#362yet another obsessive take on "cloud is bad and expensive" eh? I think they vastly forget the value of some SaaS offerings in terms of time saving for small companies. running and managing numerous DBs, k8s clusters, ci/cd pipelines and stateless container systems is simply impossible with a team of 1-2 people. sure if the setup is simple and only requires a few classic components, this is way cheaper and for a 99.9%…
I'm sorry but whereever I've seen aws at work there was a sprawling terraform codebase to manage it.
This no different than puppet or ansible on bare metal complexoty-wise, you just pat extra for the shiny tools
Re: I took all my projects off the cloud, saving thousands of dollars
#363This is all correct. I've been running my own servers for many years now, keeping things simple and saving a lot of money and time (setting anything up in AWS or Azure is horribly complicated and the UIs are terrible). One thing to remember is that you do need to treat your servers as "cloud servers" in the sense that you should be able to re-generate your entire setup from your configuration at any time, given a bun…
I was a guy that built server clusters during the early 00's, for my own and others' web and other projects. When AWS really took off, it was like a spend all your money mania, and devs and companies treated my skills like dirt. I got a job writing facial recognition edge servers, with high performance many claim are impossible numbers (25M face compares per second per core) and my employer found itself a leader in t…
> to get sales the company ended up moving everything into AWS because customers did not trust anything else
This is a hidden cost of self-hosting for many in b2b. It's not just convincing management, it's convincing your clients.
Re: I took all my projects off the cloud, saving thousands of dollars
#364Great write-up :) I find hosting VPS on DigitialOcean/Hetzner to be the best middle-ground: you save a lot, but it still convenient; you just SSH into the thing, no need to think about physical cables, cooling, electricity and all the being physical baggage
Re: I took all my projects off the cloud, saving thousands of dollars
#365Earlier quoted context omitted.
I was a guy that built server clusters during the early 00's, for my own and others' web and other projects. When AWS really took off, it was like a spend all your money mania, and devs and companies treated my skills like dirt. I got a job writing facial recognition edge servers, with high performance many claim are impossible numbers (25M face compares per second per core) and my employer found itself a leader in t…
> to get sales the company ended up moving everything into AWS because customers did not trust anything else This is a hidden cost of self-hosting for many in b2b. It's not just convincing management, it's convincing your clients.
That's interesting. Except for $GIGANTIC_CO (like, BofA, or the government), i'd expect a SLA that describes service resiliency and not "well, our service will be up because we're on AWS".
Why would you need to disclose your hosting provider? is that really a concern for hosted services (and if it is, why isn't the customer hosting it in their cloud?)
Re: I took all my projects off the cloud, saving thousands of dollars
#366Earlier quoted context omitted.
Multiple small boxes is actually better than one giant box, for a whole lot of reasons. Scaling isn't the issue.
This needs some context. I can think of several cases where this wouldn’t be the be right.
Here's a basic comparison:
Many small boxes
Pros:
- Cost. Small boxes cost less. Spend less or spend more as needed. If one dies, cheaper to replace.
- Efficiency. Scaling *down* saves money when load is low. Can schedule specific loads to specific boxes.
- Redundancy. Multiple VMS, OSes, network paths, etc. No single point of failure.
- Zero-downtime. Rolling deployments, upgrades means changes with no user impact.
- System bandwidth. More network links, cpus, kernels, disks, etc = more bandwidth, capacity.
- Performance resilience. A heavily loaded app on one server doesn't affect others.
- Immutability. "Pets" rather than "cattle" uses automation to reduce maintenance/instability.
- Scalability. When you run out of resources, adding more is easy, zero impact.
Cons:
- Does not work with applications that require large memory/cpu.
- Inefficient for apps that require shared filesystem access (as opposed to database).
- Requires smarter architecture to reduce long tail of cross-host calls.
- More transient network path failures, troubleshooting issues.
One big box
Pros:
- Allows applications which require large memory/cpu.
- More efficient for apps that share a filesystem.
- Simpler architecture.
- Fewer network path failures.
Cons:
- Large cost that you can't easily reduce as needed.
- Waste (in unused resources) unless load is constant.
- Single point of failure, for reliability and security.
- Upgrades require reboots. App goes down; possibility the server might not boot up properly.
- Single network, cpu, kernel, disks(s), etc become bottlenecks.
- A single heavily-loaded process, excess interrupts, etc can bring down entire system performance.
- Often treated as "pet" rather than "cattle"; creates more maintenance, instability.
- Not scalable.Re: I took all my projects off the cloud, saving thousands of dollars
#367Earlier quoted context omitted.
You're falling into the false dichotomy that always comes up with these topics: as if the choice is between the cloud and renting rack space while applying your own thermal paste on the CPUs. In reality, for most people, renting dedicated servers is the goldilocks solution (not colocation with your own hardware). You get an incredible amount of power for a very reasonable price, but you don't need to drive to a datac…
I’d add this. Servers used to be viewed as pets; the system admins spent a lot time on snow flake configurations and managing each one. When we started standing up tens of servers to host the nodes of our app (early 2000s); the simple admin overhead was huge. One thing I have not seen mentioned here was how powerful ansible and similar tools were at simplifying server management. Iirc being able to provision and stan…
Also, it was a very very different landscape.
You were commonly given a network uplink and a list of public IP addresses you were to set up on your box or boxes. IPMI/BMC were not a given on a server so if you broke it, you needed to have remote hands and probably brains too.
Virtualisation was in the early days and most of the services were co-hosted on the server.
Software defined networks and Open vSwitch were also not a thing back then. There were switches with support for VLANs and you might have had a private network to link together frontend and backend boxes.
Servers today can be configured remotely. They have their own management interfaces so you can access the console and install OS remotely. The network switches can be reconfigured on the fly, making the network topology reconfigurable online. Even storage can be mapped via SAN. The only hands on issue is hardware malfunction.
If I was to compare with today, it was like having a wardrobe of Raspberry Pies on a dumb switch, plugging in cables when changes were needed.
Re: I took all my projects off the cloud, saving thousands of dollars
#368Re: I took all my projects off the cloud, saving thousands of dollars
#369Earlier quoted context omitted.
Databases are rarely the bottleneck during access volume spikes in my experience. It’s really impressive how far some beefy servers and a read copy can take you. But if that is your bottleneck you should be upgrading your DB system regardless of whether you’re on cloud or bare metal.
But if your one source of truth is a single huge database, maybe on a mainframe, as is often the case with finance, and eventual consistency is a huge no go, the mainframe can become a limit regardless when you're being "hammered" by 3+ million users all wanting to check their account balance. Yes, you can solve the problem with sharding and other tricks, but for many banks, the mainframe is still their main data sto…
I agree with that, but I don’t understand how that’s particularly relevant to a bare metal vs cloud discussion.
Wouldn’t you need to do the work to shard regardless of where you’re running?
Re: I took all my projects off the cloud, saving thousands of dollars
#370Earlier quoted context omitted.
But if your one source of truth is a single huge database, maybe on a mainframe, as is often the case with finance, and eventual consistency is a huge no go, the mainframe can become a limit regardless when you're being "hammered" by 3+ million users all wanting to check their account balance. Yes, you can solve the problem with sharding and other tricks, but for many banks, the mainframe is still their main data sto…
I agree with that, but I don’t understand how that’s particularly relevant to a bare metal vs cloud discussion. Wouldn’t you need to do the work to shard regardless of where you’re running?
Banks running on mainframes don't shard. They just throw more hardware at the problem. The modern mainframes capacity for raw IO is insane, and much higher than what you can achieve with any PC setup. More hardware, more bandwidth, client facing applications in the cloud, database on the mainframe.
The major difference lies in infrastructure, particularly networking infrastructure. With cloud providers like Azure, AWS, etc, you can provision your vnet layout, and scale "indefinitely" on the same infrastructure. You don't need to provision new hosts, setup new secrets, or anything like that.
If a data center goes down, you can relatively easy switch to another one, though most financial institutions I know of uses hot/cold setups as hot/hot is essentially twice the money, and they rarely go down for long.
Of course it's all just regular servers underneath, so anything possible with AWS and Azure is also possible with other cloud providers, but the tooling simply isn't there (yet?).
Another issue is ISO auditor compliance. Being a regulated industry, finance (in EU at least) needs certain compliance to be fulfilled, not only regarding the services you consume, but also stuff like the physical locations, or being able to physically inspect the data center if auditors require it.
Microsoft and Amazon has this nailed. I've yet to experience a EU data center not run by FAANG meet the requirements, not that they can't. My best hope so far is probably "Lidl cloud" (forgot the name).