Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Almost every infrastructure decision I endorse or regret

cep.dev

Post reply on HN
Print view View original

451–460 of 644 posts

Re: Almost every infrastructure decision I endorse or regret

#451

Post by sseagull »

pinkgolem wrote: post #261
chii wrote: post #249

Earlier quoted context omitted.

i would imagine that cloud infrastructure has the ability for fast scale up, unlike self-owned infrastructure. For example, how long does it take to rent another rack that you didnt plan for? And not to mention that the cost of cloud management platforms that you have to deploy to manage these owned assets is not free. I mean, how come even large consumers of electricity does not buy and own their own infrastructure…

>I mean, how come even large consumers of electricity do not buy and own their own infrastructure to generate it? They sure do? BASF has 3 power plants in Hamburg, Disney operate Reedy Creek Energy with at least 1 power plant and I could list a fair bit more... >For example, how long does it take to rent another rack that you didnt plan for? I mean, you can also rent hardware a lot cheaper then on AWS. There certainl…

Some universities also have their own power plants. It’s also becoming more common to at least supplement power on campus with solar arrays.

Re: Almost every infrastructure decision I endorse or regret

#452

Post by Jedd »

rswail wrote: post #257
Jedd wrote: post #92

Earlier quoted context omitted.

> Atlassian also just launched an integration with OpsGenie. Given Atlassian bought OpsGenie in 2018, this either somewhere between quite late and unsurprising .

Two different measurements (time and Atlassian development processes) that are orthogonal. Anything Atlassian does is mostly quite late and its integration story is so pathetic that it's unsurprising. Try to have a bitbucket pipeline that pushes to confluence. Seems like a basic integration to have, after all, Confluence has an API (well, actually it has 3 different ones) so surely Atlassian would make a basic thing…

Oh, I am no great fan. Plus I have a nascent blog post on the subject of 'can you believe ...?' items around this subject.

I suppose it comes back to the comparative priorities (as evaluated by recurrent revenue) of ticking rfq boxes vs solving actual problems.

Re: Almost every infrastructure decision I endorse or regret

#453

Post by swores »

pinkgolem wrote: post #278
swores wrote: post #276

Earlier quoted context omitted.

Could you please explain what you mean by "physical backbone connection", as I can't think of a meaning that fits the context. If you mean dealing with the physical dedicated servers that can be rented from Hetzner, that's what the person you replied to was talking about being not so difficult. If you mean everything else at the data centre that makes having a server there worthwhile (networking, power, cooling, etc.…

With "routing" I meant the backbone connection, which is included in the hetzner price. Aka if I add up power (including backup) + backbone connection rental + server deprication I can not do it for the hetzner price.. That was quite imprecise, sorry about that.

No worries, easy to not foresee every possible way in which strangers could interpret a comment!

But I think that people (at least jwr, and probably even nyc_data_geek saying "on prem") are talking about cloud (like AWS) vs. renting (or buying) servers that live in a data centre run by a company like Hetzner, which can be considered "on prem" if you're the kind of data centre client who has building access to send your own staff there to manage your servers (while still leaving everything else, possibly even legal ownership and therefore deprecation etc. to the data centre owner).

What you're thinking of - literally taking responsibility for running your own mini data centre - I think is hardly ever considered (at least in my experience), except by companies at the extremes of size. If you're as big as Facebook (not sure where the line is but obviously including some companies not AS big as Meta but still huge) then it makes sense to run your own data centres. If you're a tiny business getting less than thousands of website visits a day and where the website (or whatever is being hosted) isn't so important that a day of downtime every now and then isn't a big deal, then it's not uncommon to host from the company's office itself (just using a spare old PC or second hand cheap 1U server, maybe a cheap UPS, and just connected to the main internet connection that people in the office use, and probably managed by a single employee, or company owner, who happens to be geeky enough to think it's one or both of simple or fun to set up a basic LAMP server, or even a Windows server for its oh-so-lovely GUI).

Re: Almost every infrastructure decision I endorse or regret

#455

Post by ffsm8 »

davedx wrote: post #349
ffsm8 wrote: post #313

Earlier quoted context omitted.

> Automating a database backup and testing it works takes half a day unless you’re doing something weird True story bro I'm sure that's possible if you're storing the backup on the same server you're restoring on and everything is on top of the line nvme storage. Otherwise your backup just started to run and will need another few days to finish. And that's only if you're running single master. You're massively undere…

I’ve implemented such a process for companies multiple times, bro. I know what I’m talking about.

I've done it before,too. For toy project, it's easy as you said. It's not once you're at scale. It's hilarious that people are down voting my comment. I guess there are a lot of juniors suffering from the dunning Kruger syndrome around right now

Re: Almost every infrastructure decision I endorse or regret

#456

Post by Shorel »

hhsectech wrote: post #319
smitty1e wrote: post #307

Earlier quoted context omitted.

My funny story is built on the idea that AWS is Hotel California for your data. A customer had an interest in merging the data from an older account into a new one, just to simplify matters. Enterprise data. Going back years. Not even leaving the region. The AWS rep in the meeting kinda pauses, says: "We'll get back to you on the cost to do that." The sticker shock was enough that the customer simply inherited the ol…

Eh? I've never had a problem moving data out of AWS. Have people lost the ability to write export and backup scripts?

Just in network costs, there's a huge asymmetry. Uploading data to AWS is free. Downloading data from them, you have to pay.

When you have enough data, that cost is quite significant.

Re: Almost every infrastructure decision I endorse or regret

#457

Post by [deleted] »

jwr wrote: post #266
pinkgolem wrote: post #254

Earlier quoted context omitted.

Keep in mind, there is an in between.. I would have a hard time doing servers as cheap as hetzner for example including the routing and everything

I do that. In fact I've been doing it for years, because every time I do the math, AWS is unreasonably expensive and my solo-founder SaaS would much rather keep the extra money. I think there is an unreasonable fear of "doing the routing and everything". I run vpncloud, my server clusters are managed using ansible, and can be set up from either a list of static IPs or from a terraform-prepared configuration. The same…

[deleted]

Re: Almost every infrastructure decision I endorse or regret

#458

Post by CubsFan1060 »

storyinmemo wrote: post #404
mciancia wrote: post #380

Earlier quoted context omitted.

But there is no paid egress when we are moving data between account within one region, rigth?

There is. You pay a price for any cross-VPC traffic.

This isn't true, at least not anymore.

You can peer two vpc's and as long as you are transferring within the same (real) AZ, it's free: https://aws.amazon.com/about-aws/whats-new/2021/05/amazon-vp...

Even peered VPC's only pay "normal" prices: https://aws.amazon.com/ec2/pricing/on-demand/#Data_Transfer

"Data transferred "in" to and "out" from Amazon EC2, Amazon RDS, Amazon Redshift, Amazon DynamoDB Accelerator (DAX), and Amazon ElastiCache instances, Elastic Network Interfaces or VPC Peering connections across Availability Zones in the same AWS Region is charged at $0.01/GB in each direction."

Re: Almost every infrastructure decision I endorse or regret

#459

Post by CubsFan1060 »

jacurtis wrote: post #126
infecto wrote: post #65

Earlier quoted context omitted.

The problem you have here is by the time you reach the size of this DB, you are on a special discount rate within AWS.

Discount rates are actually much better too on the bigger instances. Therefore the "sticker price" that people compare on the public site is no where close to a fair comparison. We technically aren't supposed to talk about pricing publically, but I'm just going to say that we run a few 8XL and 12Xl RDS instances and we pay ~40% off the sticker price. If you switch to Aurora engine the pricing is absurdly complex (its…

The new Aurora pricing model helps, and is honestly the only reason we're able to use it. It caps costs: https://aws.amazon.com/blogs/aws/new-amazon-aurora-i-o-optim...

Re: Almost every infrastructure decision I endorse or regret

#460

Post by banku_brougham »

smitty1e wrote: post #307
jumploops wrote: post #280

Earlier quoted context omitted.

Funny story time. I was once part of an acquisition from a much larger corporate entity. The new parent company was in the middle of a huge cloud migration, and as part of our integration into their org, we were required to migrate our services to the cloud. Our calculations said it would cost 3x as much to run our infra on the cloud. We pushed back, and were greenlit on creating a hybrid architecture that allowed us…

My funny story is built on the idea that AWS is Hotel California for your data. A customer had an interest in merging the data from an older account into a new one, just to simplify matters. Enterprise data. Going back years. Not even leaving the region. The AWS rep in the meeting kinda pauses, says: "We'll get back to you on the cost to do that." The sticker shock was enough that the customer simply inherited the ol…

Is R2 a sensible option for hosting data? I understand egress is chesp.
Post reply on HN