Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Use one big server

specbranch.com

Post reply on HN
Print view View original

391–400 of 601 posts

Re: Use one big server

#391

Post by SicSemperUranus »

quickthrower2 wrote: post #379

I am using Firebase on a project and I regret it. There are some Firebase specific annoyances to put up with, like the local emulator is not as nice and "isomorphic" as say running postgresql locally. But the main problem (and I think this is shared by what I call loosely "distributed databases") is you have to think really hard about how the data is structured. You can't structure it as nicely from a logical perspec…

> Let me rent real servers, but expose it in a "serverless" "cloud-like" way, so I don't have to upgrade the OS and all that kind of stuff.

I think you're describing platform-as-a-service? It does exist, but it didn't eat cloud's lunch, rather the opposite I expect.

It's hard to sell a different service when most technical people in medium-big companies are at the mercy of non-technical people who just want things to be as normal as possible. I recently encountered this problem where even using Kubernetes wasn't enough, we had to use one of the big three, even though even sustained outages wouldn't be very harmful to our business model. What can I say, boss want cloud.

Re: Use one big server

#392

Post by kjeetgill »

PeterCorless wrote: post #125

We have a different take on running "one big database." At ScyllaDB we prefer vertical scaling because you get better utilization of all your vCPUs, but we still will keep a replication factor of 3 to ensure that you can maintain [at least] quorum reads and writes. So we would likely recommend running 3x big servers. For those who want to plan for failure, though, they might prefer to have 6x medium servers, because…

Well said. Caring about vertical scale doesn't mean you have to throw out a lot of the lessons learned about still being horizontally scalable or high availability.

Re: Use one big server

#393

Post by mr_toad »

cfors wrote: post #56

Yep, there's a premium on making your architecture more cloudy. However, the best point for Use One Big Server is not necessarily running your big monolithic API server, but your database. Use One Big Database. Seriously. If you are a backend engineer, nothing is worse than breaking up your data into self contained service databases, where everything is passed over Rest/RPC. Your product asks will consistently want t…

> Use One Big Database.

It’s never one big database. Inevitably there are are backups, replicas, testing environments, staging, development. In an ideal unchanging world where nothing ever fails and workload is predictable then the one big database is also ideal.

What happens in the real world is that the one big database becomes such a roadblock to change and growth that organisations often throw away the whole thing and start from scratch.

Re: Use one big server

#394

Post by luhn »

quickthrower2 wrote: post #379

I am using Firebase on a project and I regret it. There are some Firebase specific annoyances to put up with, like the local emulator is not as nice and "isomorphic" as say running postgresql locally. But the main problem (and I think this is shared by what I call loosely "distributed databases") is you have to think really hard about how the data is structured. You can't structure it as nicely from a logical perspec…

In my opinion the best argument for RDBMSs came, ironically, from Rick Houlihan, who was at that time devrel for DynamoDB. Paraphrasing from memory, he said "most data is relational, because relationships are what give data meaning, but relational databases don't scale."

Which, maybe if you're Amazon, RDBMSs don't scale. But for a pleb like me, I've never worked on a system even close the scaling limits of an RDBMS—Not even within an order of magnitude of what a beefy server can do.

DynamoDB, Firebase, etc. require me to denormalize data, shape it to conform to my access patterns—And pray that the access patterns don't change.

No. I think I'll take normalized data in an RDBMS, scaling be damned.

Re: Use one big server

#395

Post by pc86 »

bergkvist wrote: post #356
jethro_tell wrote: post #338

Earlier quoted context omitted.

if you have multiple micro services updating the database you need to have a database access layer service as well. there's some real value with abstraction and microservices but you can try to run them against a monolithic database service

No amount of abstraction is going to save you from the problem of 2 processes manipulating the same state machine.

In this example, it's the job of the "database access layer service" to manage those processes and prevent issues.

But, terrible service name aside, this is a big reason why two services accessing the same database is a capital-H Huge anti-pattern, and really screams "using this project to learn how to do microservices."

Re: Use one big server

#396

Post by cestith »

Koshkin wrote: post #142
cestith wrote: post #135

Earlier quoted context omitted.

Why containers when you can use unikernel applications?

But can unikernel applications share a big server (without themselves running inside VMs)?

Unikernels are VM guests. They just need a hypervisor, not to run inside another VM.

Re: Use one big server

#397

Post by wheresmycraisin »

One thing that has helped me grow over the last few years building startups is: microservices software architecture and microservice deployment are two different things.

You can logically break down your software into DDD bounded contexts and have each own its data, but that doesn't mean you need to do Kubernetes with Kafka and dozens of tiny database instances, communicating via json/grpc. You can have each "service" live in its own thread/process, have it's own database (in the "CREATE DATABASE" sense, not the instance sense), communicate via a simple in-memory message queue, and communicate through "interfaces" native to your programming language.

Of course it has its disadvantages (need to commit to a single softare stack, still might need a distributed message queue if you want load balancing, etc) but for the "boring business applications" I've been implementing (where DDD/logical microservices makes sense) it has been very useful.

Re: Use one big server

#398

Post by [deleted] »

cogman10 wrote: post #385
n0ts0gr33n wrote: post #368

Earlier quoted context omitted.

I'd be curious to know what your company does which generates this volume of data (if you can disclose), what database you are using and how you are planning to solve this issue.

Finance. MSSQL. There are multiple plans on how to fix this problem but they all end up boiling down to carving out domains and their owners and trying to pull apart the data from the database. What's been keeping the lights on is "Always On" and read only replicas. New projects aren't adding load to the db and it's simply been a slow going getting stuff split apart. What we've tried (and failed at) is sharding the d…

[deleted]

Re: Use one big server

#399

Post by anon73044 »

bombcar wrote: post #45
saulrh wrote: post #28

Earlier quoted context omitted.

> competitively provide residential > Internet service to offset costs. I uh. Providing residential Internet for an apartment complex feels like an entire business in and of itself and wildly out of scope for a small business? That's a whole extra competency and a major customer support commitment. Is there something I'm missing here?

You're missing "apartment complex" - you as the service provider contract with the apartment management company to basically cover your costs, and they handle the day-to-day along with running the apartment building. Done right, it'll be cheaper for them (they can advertise "high speed internet included!" or whatever) and you won't have much to do assuming everything on your end just works. The days where small ISPs…

I feel like this would open up the company to too much liability. Too many of your apartment users are torrenting/streaming/ too many DMCA filings to deal with when my main business is "hypothetically" being a top 3 nation wide payroll provider.

Re: Use one big server

#400

Post by filearts »

fleddr wrote: post #331

Our industry summarized: Hardware engineers are pushing the absolute physical limits of getting state (memory/storage) as close as possible to compute. A monumental accomplishment as impactful as the invention of agriculture and the industrial revolution. Software engineers: let's completely undo all that engineering by moving everything apart as far as possible. Hmmm, still too fast. Let's next add virtualization an…

Software engineers don't want to be managing physical hardware and often need to run highly available services. When a team lacks the skill, geographic presence or bandwidth to manage physical servers but needs to deliver a highly-available service, I think the cloud offers legitimate improvements in operations with downsides such as increased cost and decreased performance per unit of cost.

Seems like a fair trade-off to make.

Post reply on HN