Totally agree.
Now every system design interview expects you to build some monstrous stack with layers of caching and databases for a hypothetical 1M DAU (daily active users) app.
Mess in the head.
What the hell have you built
121–130 of 239 posts
Re: What the hell have you built
#122I don't really get this line of argument Or at least it's not engaging with the obvious counterargument at all - that: "You may not need the scale now, but you may need it later". For a startup being a unicorn with a bajillion users is the only outcome that actually counts as success. It's the outcome they sell to their investors. So sure, you can make a unscalable solution that works for the current moment. Most lik…
>So sure, you can make a unscalable solution that works for the current moment. You're making two assumptions - both wrong: 1) That this is an unscalable solution - A monolith app server backed by Postgres can take you very very far. You can vertically scale by throwing more hardware at it, and you can horizontally scale, by just duplicating your monolith server behind a load-balancer. 2) That you actually know where…
Do you have actual examples of this?
Your solution is to basically do a re-write when scale becomes a problem. Which is the textbook example of something that sounds good but never works
On the other hand I can't think of a business that failed b/c it failed to scale :)
Re: What the hell have you built
#123It's sure a corny stance to hold if you're navigating an infrastructure nightmare daily, but in my opinion, much of the complexity addresses not technical, but organisational issues: You want straightforward, self-contained deployments for one, instead of uploading files onto your single server. If the process crashes or your harddisk dies, you want redundancy so even those twelve customers can still access the appli…
I'm not sure why your architecture needs to be complex to support CI pipelines and proper workflow for change management. And some of these guidelines have grown into satus quo common recipes. Take your starting database for example, the guideline is always "sqlite only for testing, but for production you want Postgres" - it's misleading and absolutely unnecessary. These defaults have also become embedded into PaaS s…
Years ago we had someone who wanted to make sure that two deployments were mutually exclusive. Can’t recall why now, but something with a test environment and bootstrapping so no redundancy.
I just set one build agent up with a tag that both plans required. The simplest thing that could possibly work.
Re: What the hell have you built
#124"Maybe Redis for caching". Really that's going way too far - you do NOT need Redis for caching. Just put it in Postgres. Why go to this much trouble to put people in their place for over engineering then concede "maybe Redis for caching" when this is absolutely something you can do in Postgres. The author clearly cannot stop their own inner desire for overengineering.
The sentiment here is right, but redis does make a difference at scale. I built a web app this year on AWS lambda that had up to 1000/requests/second and at that scale, you can have trouble with Postgres, but redis handles it like it’s nothing.
I think that redis is a reasonable exception to the rule of ”don’t complicate things” because it’s so simple. Even if you have never used it before, it takes a few minutes to setup and it’s very easy to reason about, unlike mongodb or Kafka or k8s.
Re: What the hell have you built
#125I built a small simple page that I send to people when they start proposing crazy db architectures that people might like if they like this page: https://nocommasql.com/
just a nit. it pollutes back button history when I expand content. took 9 presses of back button to return to HN.
Re: What the hell have you built
#126It's sure a corny stance to hold if you're navigating an infrastructure nightmare daily, but in my opinion, much of the complexity addresses not technical, but organisational issues: You want straightforward, self-contained deployments for one, instead of uploading files onto your single server. If the process crashes or your harddisk dies, you want redundancy so even those twelve customers can still access the appli…
I think that's a slightly different set of things to what OP is complaining about though. They're much more reasonable, but also "outside" of the application. Having secret management or CI (pretty much mandatory!) does not dictate the architecture of the application at all. (except the caching layer. Remember the three hard problems of computer science, of which cache invalidation is one.) Still hoping for a good "s…
Cache invalidation is replacing one logical thing with a new version of the same logical thing. So technically that’s also naming things. Doubly so when you put them in a kv store.
Re: What the hell have you built
#127It's sure a corny stance to hold if you're navigating an infrastructure nightmare daily, but in my opinion, much of the complexity addresses not technical, but organisational issues: You want straightforward, self-contained deployments for one, instead of uploading files onto your single server. If the process crashes or your harddisk dies, you want redundancy so even those twelve customers can still access the appli…
>It's sure a corny stance to hold if you're navigating an infrastructure nightmare daily, but in my opinion, much of the complexity addresses not technical, but organisational issues: You want straightforward, self-contained deployments for one, instead of uploading files onto your single server ...
You can get all that with a monolith server and a Postgres backend.
Re: What the hell have you built
#128It's sure a corny stance to hold if you're navigating an infrastructure nightmare daily, but in my opinion, much of the complexity addresses not technical, but organisational issues: You want straightforward, self-contained deployments for one, instead of uploading files onto your single server. If the process crashes or your harddisk dies, you want redundancy so even those twelve customers can still access the appli…
Conway's Law: > Organizations which design systems... are constrained to produce designs which are copies of the communication structures of these organizations.
Lesser known trick: reorganize your teams so the code isn’t batshit.
Re: What the hell have you built
#129Earlier quoted context omitted.
This is also how you can identify decent places to work at: look for job postings that emphasize you aren't expected to already know the language. For example, in the recent "who's hiring" thread, I saw at least two places where they did that: Duckduckgo (they mention only algorithms and data structures and say "in case you're curious, we use Perl") and Stream (they offer a 10-week intro course to Go if you're not al…
To do that you need a mixture of elements: work in a somehow "exotic" language [1] and the company can afford to pay top-talent salary [2] [1] all those examples check that box, but please let's not start a language war over this statement. [2] for Jane Street I hear they do, DDG pays pretty well especially because it pay the same rate regardless where you are in the world, so it's a top-talent salary for many places…
Sounds like the best type of place to work for me. Instead of being a replaceable cog in a meat grinder that doesn't even pay well, working with boring tech, you get to work with talented people in an actually interesting language and get decently paid.
And best of all, you don't feel the need to keep chasing after the latest hype just to keep your CV relevant.
Re: What the hell have you built
#130I feel like sometimes it’s a form of procrastination. There are things we don’t want to do (talk to costumers, investors, legal, etc.), so instead we do the fun things (fun for engineers). It’s a convenient arrangement because we can easily convince ourselves and others that we’re actually being productive (we’re not, we’re just spinning wheels).
My first 5 years or so of solo bootstrapping were this. Then you learn that if you want to make money you have to prioritise the right things and not the fun things.
I'm at this stage. We have a good product with a solid architecture but only a few paying clients due to a complete lack of marketing. So I'm now doing the unfun things!