Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Have you ever switched cloud?

news.ycombinator.com

Post reply on HN
Print view View original

251–260 of 268 posts

Re: Ask HN: Have you ever switched cloud?

#251

Post by dx034 »

Spivak wrote: post #141
dx034 wrote: post #42

Earlier quoted context omitted.

I'm still not a fan of the second way. If you develop all your software to tightly integrate with AWS, you might save time developing the software but create a huge amount of technical debt. Managing your own infrastructure (with dedicated servers, so no hardware management) isn't too hard, even if you're a small shop. And managing a fleet of AWS services isn't necessarily less work. Maybe there's a reason all ads fo…

> Managing your own infrastructure (with dedicated servers, so no hardware management) isn't too hard. If you have infra skills then absolutely, it's way simpler to manage. But infra people like don't really fit in "small shops" because the price tag for one of us is (depending on the cost of living) anywhere between a quarter and half a mil total comp. And if you ever want them to take a vacation not on-call you'll…

Maybe in SV the comp is that high. I only have insights into European markets, but here the comp doesn't differ too much from programmers. And an AWS expert won't be cheaper than someone with knowledge how to manage infrastructure.

And yes, you'll need three people, but not full time. From experience I can say that even with a hundred servers, it tends to be a small chunk of each of their time. And if you have a redundant system and don't deploy on Fridays, the chance that someone has to respond to a call on the weekend is pretty much 0.

Re: Ask HN: Have you ever switched cloud?

#252

Post by TrickardRixx »

dijit wrote: post #211
TrickardRixx wrote: post #75

Earlier quoted context omitted.

Very similar experience here. I work on a two person "DevOps" team. Without AWS ECS we would have to have a much higher headcount. I get to spend most of my time solving real problems for the engineers on the product team rather than sysadmin work.

What are “real problems” for the engineers or product team?

Things like automating manual workflows, building small infrastructure debugging tools, or providing infrastructure consultation to an engineer trying to decouple two parts of a legacy code base.

Re: Ask HN: Have you ever switched cloud?

#253

Post by dx034 »

tomatowurst wrote: post #81
dx034 wrote: post #38

Earlier quoted context omitted.

I also found that running a PostgreSQL database is really simple. Especially if most of your workload is read only, a few dedicated servers at several providers with a PostgreSQL cluster can deliver 100% uptime and more than enough performance for pretty much any use case. At the same time, this will still be cheaper than one managed database at any cloud provider. I've been running a PostgreSQL cluster with signific…

can you share more details, because im in teh process of doing the same since having a few terabytes of postgresql / dynamodb is stupid expensive. for a lot big organizations its a matter of accountability. if they say AWS went down vs our dedicated servers went down, it matters a lot for insurance, clients. what i dont get are 4 man startups paying thousands to AWS ... because everybody does it.

As I said, if most queries are read-only it's really simple. Streaming replication works very well out of the box, just make sure you keep enough WAL segments on master so that slaves can catch up after some downtime.

I have a 1-1 relationship between application servers and databases. The application queries replication delay and marks itself as unhealthy and reports an error if the delay is too high. You can also do that via postgres (max_replication_delay), but I found this way to allow for more graceful failovers.

With streaming replication, servers are completely identical, so you can easily provision a new server. Failover is done by just one command on a slave. I don't have automatic failover as I only needed to use that once in several years (and that was on purpose), I'd rather accept downtime than having an unwanted failover.

With that setup you can always failover and can scale read operations really well. There are solutions for postgres if you need more complicated setups, but I never looked into them.

If you're in Europe, it's really cheap to get a dedicated machine from Hetzner with a few TB of NVMe. Just pay the extra money for 10gbit link, otherwise replication will take forever. But there are also some decent providers in the US, it's just more expensive. But with Hetzner, a two machine setup will be I'd just be careful with using block storage, I often found that to be a bottleneck with database servers. Local storage is almost always much faster.

But in the end it depends on your use case. In the end, your database will usually go down because of a bug in the application or some misconfiguration. Both can happen on any service. It's really so rare these days to lose a server without notice. And Postgres is really stable, I've never seen it crash.

Re: Ask HN: Have you ever switched cloud?

#254

Post by thatwasunusual »

xtracto wrote: post #149
thatwasunusual wrote: post #3

Switched from Amazon to Google because I hate Amazon more than Google. Feature-wise, I'm just as happy. However, I trust Google more, but that probably boils to my hatred again. :)

Interesting that you trust Google more in this regard. Given Google's terrible history of deprecating products, I would not trust any "real world" business to any of their services. Also, at some point I was playing with Serverless in both Google and AWS. Google's Serverless examples were broken (Google cloud was returning 500 errors) while the same stuff in AWS worked smoothly. That left me with a bad taste.

> Interesting that you trust Google more in this regard. Given Google's terrible history of deprecating products, I would not trust any "real world" business to any of their services.

That doesn't apply to me, because I have never used any of Google's deprecated products. I assume that's because they haven't (or have they?) deprecated any of their cloud services.

> Also, at some point I was playing with Serverless in both Google and AWS. Google's Serverless examples were broken (Google cloud was returning 500 errors) while the same stuff in AWS worked smoothly. That left me with a bad taste.

Good for you!

I have used Google Cloud Run for more than a year now, and can't be more happy. Never had problems with AWS either, which means that there are at least two cloud providers providing the same-ish service that lots of people can enjoy.

Re: Ask HN: Have you ever switched cloud?

#255

Post by klysm »

martingoodson wrote: post #161

I once worked with a CTO who decided to move from AWS to GCP (and also move from Spark to Hive and python to scala, at the same time). That guy was an idiot. (At one point somebody accidentally spent £30,000 in data transfer costs with one key press.) The project was never completed and the CTO just moved on to another fancy CTO position.

Why on earth would you go from spark to hive?

Re: Ask HN: Have you ever switched cloud?

#256

Post by ramshanker »

dijit wrote: post #210
netcoyote wrote: post #184

Earlier quoted context omitted.

One of the things I'm immensely curious about is how you handle security/networking/firewalls when working with Hetzner or other bare metal providers? It seems they don't provide network gear for firewalls and to protect against DDOS attacks? Do you just use iptables? Or do you build out more complex solutions like software routers running on Linux/BSD? I work in online gaming, and we're constantly seeing attacks on…

Hey! I also make games and we solved the issue by doing stateless ACLs on the border switches and having a really fat pipe. You can add a magic header to traffic and drop anything that doesn’t contain the header. Since this is done in hardware it operates at line speed. In our case 100GBit/s.

This seems such a good trick. We could even do it at cloudflare rule level I guess.

Re: Ask HN: Have you ever switched cloud?

#257

Post by lapser »

jjav wrote: post #234
lapser wrote: post #225

Earlier quoted context omitted.

AWS gives away literally tens of thousands of pounds to start ups. Granted, it's to lock them in down the line but getting free credits gets you further than not having credits.

AWS credits are a toxic trap. BTDT. They'll get you far enough that you are hopefully (from AWS perspective) locked in pretty tight. And then the high monthly costs start to hit you hard.

Sure, that's true. But I've been in companies where they've had major investments and have a pretty significant number of users, and they still had stupid amounts of credits. One of my previous companies was running a third of their infra on AWS credits, and they still long runway in terms of free credits.

Meanwhile, they had to shake off a £1 million a year contract for the next 5 years for 2 DCs. With AWS they were using less than half of that per year (this includes the credits they had). But even if it wasn't cheaper, requesting a new server took days, not minutes. Scaling was not possible. I'll take the credit and potentially get trapped than having to deal with an inflexible mess that is in-house managed infrastructure.

At least bigger orgs are able to afford it by (hopefully) building a cloud on top of their infrastructure, but outside that, the majority should of companies be looking into the cloud. Whether that be AWS, GCP, or the smaller Clouds like DO, it doesn't matter.

Re: Ask HN: Have you ever switched cloud?

#258

Post by lolinder »

holografix wrote: post #242
lolinder wrote: post #66

Earlier quoted context omitted.

> Really, I can't see a compelling reason to be in the cloud for the majority of mid-level workloads like ours. I work on a very small team. We have a few developers who double as ops. None of us are or want to be sysadmins. For our case, Amazon's ECS is a massive time and money saver. I spent a week or two a few years ago getting all of our services containerized. Since then we have not had a single full production…

To the peeps running ECS? Why not just straight up AKS or GKE? Have you compared ECS to Cloud Run on GCP?

In my case, mostly because it was easier to get buy-in from the rest of the team on ECS than Kubernetes.

Re: Ask HN: Have you ever switched cloud?

#259

Post by jjav »

lapser wrote: post #257
jjav wrote: post #234

Earlier quoted context omitted.

AWS credits are a toxic trap. BTDT. They'll get you far enough that you are hopefully (from AWS perspective) locked in pretty tight. And then the high monthly costs start to hit you hard.

Sure, that's true. But I've been in companies where they've had major investments and have a pretty significant number of users, and they still had stupid amounts of credits. One of my previous companies was running a third of their infra on AWS credits, and they still long runway in terms of free credits. Meanwhile, they had to shake off a £1 million a year contract for the next 5 years for 2 DCs. With AWS they were…

> But I've been in companies where they've had major investments and have a pretty significant number of users, and they still had stupid amounts of credits.

Your companies have been wiser and more frugal than mine!

In every case, I've seen the credits run out before there was a penny of revenue coming in.

Re: Ask HN: Have you ever switched cloud?

#260

Post by nijave »

dijit wrote: post #217
bushbaba wrote: post #171

Earlier quoted context omitted.

Doubtful. I've personally had projects pull many hundreds of gigabits per second of s3 throughput. How you architect and design has a large influence to your analytics performance.

“Many hundreds” Last time I talked to a technical person at AWS the limit was 5GBits. Wonder what you’re doing differently. Perhaps that changed.

There's another benchmark somewhere showing S3 can max out a 100Gbps instance.

https://github.com/dvassallo/s3-benchmark

Another potential issue is ListBucket rate limiting. If you have lots of small objects, you'll spend most of the time waiting to discover the names than transferring data

Post reply on HN