271–280 of 601 posts
The problem with "one big server" is, you really need good IT/ops/sysadmin people who can think in non-cloud terms. (If you catch them installing docker on it, throw them into a lava pit immediately).
Earlier quoted context omitted.
OP mentioned joining, so they were definitely talking about a single database
Not suggesting it, but for the sake of knowledge you can join tables living in different databases, as long as they are on the same server (e.g. mysql, postgresql, SQL server supports it - doesn't necessarily come for free)
Yep, there's a premium on making your architecture more cloudy. However, the best point for Use One Big Server is not necessarily running your big monolithic API server, but your database. Use One Big Database. Seriously. If you are a backend engineer, nothing is worse than breaking up your data into self contained service databases, where everything is passed over Rest/RPC. Your product asks will consistently want t…
I'm glad this is becoming conventional wisdom. I used to argue this in these pages a few years ago and would get downvoted below the posts telling people to split everything into microservices separated by queues (although I suppose it's making me lose my competitive advantage when everyone else is building lean and mean infrastructure too). In my mind, reasons involve keeping transactional integrity, ACID compliance…
My hunch is that computers caught up. Back in the early 2000's horizontal scaling was the only way. You simply couldn't handle even reasonably mediocre loads on a single machine.
As computing becomes cheaper, horizontal scaling is starting to look more and more like unnecessary complexity for even surprisingly large/popular apps.
I mean you can buy a consumer off-the-shelf machine with 1.5TB of memory these days. 20 years ago, when microservices started gaining popularity, 1.5TB RAM in a single machine was basically unimaginable.
The entire situation ended up having conference call with multiple department directors who were deciding which server from other systems to cannibalize (even if it is underpowered) to get the system going.
Since that time I'm quite skeptical about "one", and to me this is one of big benefits of cloud provides, as, most likely, there is another instance and stockouts are more rare.
Earlier quoted context omitted.
Just FYI, you can have one big database, without running it on one big server. As an example, databases like Cassandra are designed to be scaled horizontally (i.e. scale out, instead of scale up). https://cassandra.apache.org/_/cassandra-basics.html
There are trade-offs when you scale horizontally even if a database is designed for it. For example, DataStax's Storage Attached Indexes or Cassandra's hidden-table secondary indexing allow for indexing on columns that aren't part of the clustering/partitioning, but when you're reading you're going to have to ask all the nodes to look for something if you aren't including a clustering/partitioning criteria to narrow…
https://stackoverflow.blog/2022/03/14/how-sharding-a-databas...
Earlier quoted context omitted.
Depends what you mean by a database I guess. I take it to mean an RDBMS. RDBMSs provide guarantees that web searching doesn't need. You can afford to lose a pieces of data, provide not-quite-perfect results for web stuff. It's just wrong for an RDBMS.
What if you are using the database as a system of record to index into a real search engine like Elasticsearch? For a product where you have tons of data to search from (ie text from web pages)
Elasticsearch is for search.
If you're asking about "what if you use an RDBMS as a pointer to Elasticsearch" then I guess I would ask: why would you do this? Elasticsearch can be used as a system of record. You could use an RDBMS over top of Elasticsearch without configuring Elasticsearch as a system of record, but then you would be lying when you refer to your RDBMS as a "system of record." It's not a "system of record" for your actual data, just a record of where pointers to actual data were at one point in time.
I feel like I must be missing what you're suggesting here.
Earlier quoted context omitted.
This is a huge one -- value in outsourcing blame. If you're down because of a major provider outage in the news, you're viewed more as a victim of a natural disaster rather than someone to be blamed.
I hear this repeated so many times at my workplace, and it's so totally and completely uninformed. Customers who have invested millions of dollars into making their stack multi-region, multi-cloud, or multi-datacenter aren't going to calmly accept the excuse that "AWS Went Down" when you can't deliver the services you contractually agreed to deliver. There are industries out there where having your service casually g…
Similar to security, the last few 9s of availability come at a heavily increasing (log) complexity / price. The cutoff will vary case by case, and I’m sure the decision on how many 9s you need is often irrational (CEO says it can never go down! People need their pet food delivered on time!).
It blows my mind people are spending $2000+ per month for a server they can get used for $4000-5000 one time only cost. VMWare + Synology Business Backup + Synology C2 backup is our way of doing business and never failed us for over 7 years. Why do people spend so much money for cloud while they can host it themselves less than 5% of the cost? (2 year usage assumed).
I've tried it all except this, including renting bare metal. Nowadays I'm in the cloud but not cloudy camp. Still, I'm intrigued. Apart from the $4-5k server, what are your running costs? Licenses? Colocation? Network?
They have been around forever and their $400 deal is good, but that is for 42U, 1G and only 15 amps. With beefier servers, you will need more current (both BW and amperage) if you intend on filling the rack.
Earlier quoted context omitted.
> competitively provide residential > Internet service to offset costs. I uh. Providing residential Internet for an apartment complex feels like an entire business in and of itself and wildly out of scope for a small business? That's a whole extra competency and a major customer support commitment. Is there something I'm missing here?
It depends on the scale - it does not have to be a major undertaking. You are right, it is a whole extra competency and a major customer support commitment , but for a lot of the entrepreneurial folk on HN quite a rewarding and accessible learning experience. The first time I did anything like this was in late 1984 in a small town in Iowa where GTE was the local telecommunication utility. Absolutely abysmal Internet…
Earlier quoted context omitted.
This is absolutely true - when I was at Bitbucket (ages ago at this point) and we were having issues with our DB server (mostly due to scaling), almost everyone we talked to said "buy a bigger box until you can't any more" because of how complex (and indirectly expensive) the alternatives are - sharding and microservices both have a ton more failure points than a single large box. I'm sure they eventually moved off t…
One issue I've seen with this is that if you have a single, very large database, it can take a very, very long time to restore from backups. Or for that matter just taking backups. I'd be interested to know if anyone has a good solution for that.