Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

In defense of simple architectures

danluu.com

Post reply on HN
Print view View original

61–70 of 196 posts

Re: In defense of simple architectures

#61

Post by lifefeed »

I was interviewing for software jobs recently, and while I was studying up on the "system design" portion I kept circling around the same insight that Dan Luu writes about so well here.

I would sit down at an interview and try to create these "proper" system designs with boxes and arrows and failovers and caches and well tuned databases. But in the back of my mind I kept thinking, "didn't Facebook scale to a billion users with PHP, MySQL, and Memcache?"

It reminds me of "Command-line Tools can be 235x Faster than your Hadoop Cluster" at https://adamdrake.com/command-line-tools-can-be-235x-faster-... , and the occasional post by https://rachelbythebay.com/w/ where she builds a box that's just fast and with very basic tooling (and a lot of know-how).

Re: In defense of simple architectures

#62

Post by Beltalowda »

dmw_ng wrote: post #41
pavlov wrote: post #34

There are some web apps still in production that I wrote almost a decade ago in Node+Express in the simplest, dumbest style imaginable. The only dependencies are Express and some third-party API connectors. The database is an append-only file of JSON objects separated by newlines. When the app restarts, it reads the file and rebuilds its memory image. All data is in RAM. I figured these toys would be replaced pretty…

> database is an append-only file of JSON objects separated by newlines. When the app restarts, it reads the file and rebuilds its memory image. All data is in RAM Apps like this tend to perform like an absolute whippet too (or if they dont, getting them to perform well is often a 5 line change). It's really freeing to be able to write scans and filters with simple loops that still return results faster than a networ…

Another issue with "just a JSON file" as a database is that you need to be a bit careful to avoid race conditions and the like, e.g. if two web pages try to write the same database at the same time. It's not an issue for all applications, and not that hard to get right, but does require some effort. This is a huge reason I prefer SQLite for simple file storage needs.

Re: In defense of simple architectures

#63

Post by pavlov »

dmw_ng wrote: post #41
pavlov wrote: post #34

There are some web apps still in production that I wrote almost a decade ago in Node+Express in the simplest, dumbest style imaginable. The only dependencies are Express and some third-party API connectors. The database is an append-only file of JSON objects separated by newlines. When the app restarts, it reads the file and rebuilds its memory image. All data is in RAM. I figured these toys would be replaced pretty…

> database is an append-only file of JSON objects separated by newlines. When the app restarts, it reads the file and rebuilds its memory image. All data is in RAM Apps like this tend to perform like an absolute whippet too (or if they dont, getting them to perform well is often a 5 line change). It's really freeing to be able to write scans and filters with simple loops that still return results faster than a networ…

Yes, you need to be sure that you understand the growth pattern if you want to YOLO in RAM. If your product aims to be the next Instagram, this is clearly not the architecture.

But a lot of small businesses are genuinely small. They may not sign up new customers that often. When they do, the impact to the service is often very predictable ("Amy at customer X uses this every other day, she's very happy, it generates 100 requests / week"). If growth picks up, there would be signs well in advance of the toy service becoming an actual problem.

Re: In defense of simple architectures

#64

Post by throw0101a »

adra wrote: post #20
bob1029 wrote: post #6

Earlier quoted context omitted.

Trap or integrated win-win? We use one of these "aggressively simple" architectures too. At this point, I would quit my job instantaneously if I had to even look at k8s or whatever the cool kids are using these days.

Man, kubernetes is so much easier than the smattering of crap that you have to jungle together before it. Puppet and co? No thanks. Terraform? It's fine, but only a part of a CI/CD picture. If you think the alternatives are better, I really have to wonder how much of the trenches crap that people in your org deal with regularly that you're insulated from. That, or you're a release-quarterly kinda company?

> Puppet and co? No thanks.

Puppet? Luxury. I started my configuration management journey with cfengine. And the folks that I first heard CM about started with Makefiles:

* http://www.infrastructures.org/papers/bootstrap/bootstrap.ht...

* https://www.usenix.org/legacy/publications/library/proceedin...

Re: In defense of simple architectures

#65

Post by aidos »

In terms of the choices they're unsure about; I'd say it's best to stay away from Celery / RabbitMQ if you don't really need it. For us just using RQ (Redis backed queue) has been a lot less hassle. Obviously it's all going to depend on your scale, but it's a lot simpler.

RE the sqlalchemy concern; you do need to decide on where your transactions are going to be managed from and have a strict rule about not allow functions to commit / rollback themselves. Personally I think that sqla is a great tool, it saves a lot of boilerplate code (and data modelling and migrations are a breeze).

But overall the sentiments in this article resonate with my experience.

Re: In defense of simple architectures

#66

Post by treis »

woah wrote: post #46
ryanbrunner wrote: post #37

Earlier quoted context omitted.

In my experience SPAs bring a lot of headaches that you just don't really need to think about with traditional HTML. Browser navigation, form handling, a lot of accessibility stuff comes out of the box for free, and there's one source of truth about what makes a particular object valid or how business logic works (which is solvable in the SPA world but brings a lot of complexity when you need to share logic between t…

Probably depends on the requirements. If the product should basically feel like a static web page, and you are OK making design and product decisions that work easily in that paradigm, then a server side framework built to make static web pages is going to be simpler. If you have product or design requirements that it should feel more dynamic like a native app, then trying to patch that on top of a static webpage mig…

IMHO the important thing is where your data is. If can all be client side then write a SPA. If it's on the server then the more you do on the server the better.

Returning HTML and doing a simple element replace with the new content is 99.9% indistinguishable from a SPA.

Re: In defense of simple architectures

#67

Post by cameronh90 »

bob1029 wrote: post #9

I think the biggest problem for most developers is not understanding what one computer can actually do and how reliable they are in practice. Additionally, understanding of how tolerant 99% of businesses are to real-world problems that could hypothetically arise can help one not frustrate over insane edge case circumstances. I suspect a non-zero number of us have spent time thinking about how we could provide determi…

These requirements don't come out of nowhere. Normally they come from:

1. CEOs/whoever that don't listen to how much additional complexity it is to build a system with extremely high uptime and demand it anyway.

2. Developers with past experience that systems going down means they get called in the middle of the night.

3. Industry expectations. Even if you're a small finance company where all your clients are 9-5 and you could go down for hours without any adverse impacts, regulators will still want to see your triple redundant, automated monitoring, high uptime, geographically distributed, tested fault tolerant systems. Clients will want to see it. Investors will check for it when they do due diligence.

Look at how developers build things for their own personal projects and you'll see that quite often they're just held together with duct tape running on a single DO instance. The difference is, if something goes wrong, nobody is going to be breathing down their neck about it and nobody is getting fired.

Re: In defense of simple architectures

#68

Post by dan-robertson »

guelo wrote: post #38
WJW wrote: post #22

Earlier quoted context omitted.

Wtf are you doing with it? My slack instance (on linux) is resting around 300 MB resident set size and 0% cpu. 300 MB is still a lot for a chat app, but it is definitely not gigabytes.

Make sure you're counting all the sub-processes it spawns (at least on Mac, don't know about linux)

If you just add up memory usage for subprocesses you are likely to over count due to shared memory. The number you typically want to add up in Linux is ‘proportional set size’ which is, I think, the sum over every page of the process’s memory of page_size / number of processes which can access the page. I don’t know what happens if you mmap some physical memory twice (I think some newish Java GC does this).

Re: In defense of simple architectures

#69

Post by zie »

a9h74j wrote: post #54
zrail wrote: post #19

Earlier quoted context omitted.

Realistically almost every web app can start as a one-tier web app that uses SQLite as a data store and serves mostly HTML.

I have a dumb question ... In almost all performance areas -- gaming, PCs, autos, etc -- there are usually whole publications dedicated to performing benchmarks and publishing those results. Are there any publications or sites which implement a few basic applications against various new-this-season "full stacks" or whatnot, and document performance numbers and limit-thresholds on different hardware? Likewise, there m…

Fossil SCM is a great example of a sqlite application that has stood the test of time. I don't know what sqlite.org's traffic is like, but it's not tiny and it runs on a tiny VPS without issue(and has for years now).

Re: In defense of simple architectures

#70

Post by mftb »

pavlov wrote: post #34

There are some web apps still in production that I wrote almost a decade ago in Node+Express in the simplest, dumbest style imaginable. The only dependencies are Express and some third-party API connectors. The database is an append-only file of JSON objects separated by newlines. When the app restarts, it reads the file and rebuilds its memory image. All data is in RAM. I figured these toys would be replaced pretty…

I do almost this exact thing for all my personal stuff. I have 5 or 6 going in a vm for simple things like my bookmarks, etc... works great. I could definitely see it solving many small business use-cases.
Post reply on HN