Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

In defense of simple architectures

danluu.com

Post reply on HN
Print view View original

141–150 of 196 posts

Re: In defense of simple architectures

#141

Post by rendall »

scrubs wrote: post #27

Nah, I don't much like the tone of this article. Not at all. The engineering message should be: keep your architecture as simple as possible. And here are some ways (to follow) on how to find that minimal and complete size 2 outfit foundation in your size 10 hoarder-track-suite-eye-sore. Do we really need to be preached at with a warmed over redo of `X' cut it for me as a kid so I really don't know why all the kids t…

I'm not sure why this particular author is so popular on HN, but he hits the front page regularly.

Re: In defense of simple architectures

#142

Post by mapme »

lifefeed wrote: post #61

I was interviewing for software jobs recently, and while I was studying up on the "system design" portion I kept circling around the same insight that Dan Luu writes about so well here. I would sit down at an interview and try to create these "proper" system designs with boxes and arrows and failovers and caches and well tuned databases. But in the back of my mind I kept thinking, "didn't Facebook scale to a billion…

I think that’s a large oversimplification of Facebook. While it’s true a lot of FB storage is MySQL backed they also created many complex systems such as:

- Cassandra (based on dynamo/big table)

- wrote a custom KV store named RocksDb that is open source/now a company

- wrote a custom photos storage system that replaced an NFS based design

- wrote another custom binary object store

- wrote a custom geo distributed graph db (Tao)

- wrote an in house distributed FS replacement for HDFS

https://www.cs.cornell.edu/projects/ladis2009/papers/lakshma...

https://www.usenix.org/legacy/event/osdi10/tech/full_papers/...

https://www.usenix.org/system/files/conference/osdi14/osdi14...

https://www.usenix.org/system/files/conference/atc13/atc13-b...

https://www.cs.princeton.edu/~wlloyd/papers/tectonic-fast21....

https://m.facebook.com/nt/screen/?params=%7B%22note_id%22%3A...

Re: In defense of simple architectures

#143

Post by stackbutterflow »

lifefeed wrote: post #61

I was interviewing for software jobs recently, and while I was studying up on the "system design" portion I kept circling around the same insight that Dan Luu writes about so well here. I would sit down at an interview and try to create these "proper" system designs with boxes and arrows and failovers and caches and well tuned databases. But in the back of my mind I kept thinking, "didn't Facebook scale to a billion…

Some people say that Leetcode is nothing more than rote learning. Some disagree and I disagree. There's a minimal amount of rote learning that helps but you need more than that. On the other hand "system design" interviews are...strange? You, someone who never built something at the scale of these giant tech companies, are being asked to come up on the spot with a design for a system that'd scale to billions of users. There's a 100% chance that if you were to attempt building such systems and you'd never done it before, you'd discover holes in your original design left and right. Leetcode tests your logic, how you think, your IQ maybe. But system design interviews consists in regurgitating knowledge that you've crammed in your head by studying books/articles/videos on system design. It's closer to reciting poetry than problem solving. In my experience it could be replaced with multiple choice questions and you'd get the same result.

What am I missing?

Re: In defense of simple architectures

#144

Post by hw »

aadvark69 wrote: post #17

Simple architectures work well, until they don't. A good example is ye olde ruby on rails monolith. Dead simple to set up and iterate quickly, but once you reach a certain organization and/or codebase size, velocity starts to degrade exponentially

Tell that to Github, Shopify, Gitlab, Zendesk etc that have been doing fine.

How often do you hit ‘that certain size’ when velocity starts to degrade anyway?

X works well until it doesnt… is not exactly a compelling argument. That can be said of simple and complex architectures, or just anything at all

Re: In defense of simple architectures

#145

Post by nunez »

rendall wrote: post #140
wackget wrote: post #110

I don't know how the author can claim that they run a "simple" architecture. From their job pages: Our stack : backend: Python 3 (+ mypy) API layer: GraphQL android frontend: Kotlin/Jetpack iOS frontend: Swift/SwiftUI web frontend: TypeScript/React database: Postgres infrastructure: GCP / Terraform orchestration: Kubernetes That is not simple by any stretch of the imagination.

How would you simplify this?

for starters i wouldnt use kubernetes. love the system, but boy is it complicated. i'd use a few cloud function or stick them in VMs behind a load balancer and call it good.

Re: In defense of simple architectures

#146

Post by aprdm »

bob1029 wrote: post #59
dmw_ng wrote: post #41

Earlier quoted context omitted.

> database is an append-only file of JSON objects separated by newlines. When the app restarts, it reads the file and rebuilds its memory image. All data is in RAM Apps like this tend to perform like an absolute whippet too (or if they dont, getting them to perform well is often a 5 line change). It's really freeing to be able to write scans and filters with simple loops that still return results faster than a networ…

> The problem is always growth, either GC jank from a massive heap, running out of RAM, or those loops eventually catching up with you Absolutely. The challenge is having enough faith that it will take long enough to catch up to you. Statistically speaking, it won't catch up to you and if it does, it will take so long you should have seen it coming from miles away and had time to prepare. In my systems that use an in…

Could you expand what you mean by keeping pointers and basic index in memory?

Re: In defense of simple architectures

#147

Post by conradfr »

danenania wrote: post #72
pavlov wrote: post #34

There are some web apps still in production that I wrote almost a decade ago in Node+Express in the simplest, dumbest style imaginable. The only dependencies are Express and some third-party API connectors. The database is an append-only file of JSON objects separated by newlines. When the app restarts, it reads the file and rebuilds its memory image. All data is in RAM. I figured these toys would be replaced pretty…

Built-in first-class concurrency (ala node, golang, rust, etc.) is a huge win for simple architectures, since it lets you avoid adding a background queue, or at least delay it for a very long time. I think people are also too quick to add secondary data stores and caches. If you can do everything with a transactional SQL database + app process memory instead, that is generally going to save you tons of trouble on ops…

> Built-in first-class concurrency (ala node, golang, rust, etc.) is a huge win for simple architectures, since it lets you avoid adding a background queue, or at least delay it for a very long time.

>For example: instead of memcache/redis, set aside ~100 MB of memory in your app process for an LRU cache.

Erlang/Elixir for the win with (almost transparent multi-core) concurrency and ETS ;)

Re: In defense of simple architectures

#148

Post by [deleted] »

rendall wrote: post #140
wackget wrote: post #110

I don't know how the author can claim that they run a "simple" architecture. From their job pages: Our stack : backend: Python 3 (+ mypy) API layer: GraphQL android frontend: Kotlin/Jetpack iOS frontend: Swift/SwiftUI web frontend: TypeScript/React database: Postgres infrastructure: GCP / Terraform orchestration: Kubernetes That is not simple by any stretch of the imagination.

How would you simplify this?

[deleted]

Re: In defense of simple architectures

#149

Post by lelanthran »

Beltalowda wrote: post #95
endorphine wrote: post #86

Earlier quoted context omitted.

Doesn't the fact that its opened in append only mode (Linux) mitigate data races with regards to writes?

Your write will be fine; that is, it's not as if data from one write will be interspersed with the data from another write. It's just that the order might be wrong, or opening the file multiple times (possibly from multiple processes) could be fun too. The program or computer crashing mid-write can also cause problems. Things like that. Again, may not be an issue at all for loads of applications. But I used a lot of…

> Your write will be fine; that is, it's not as if data from one write will be interspersed with the data from another write.

Are you sure? I thought it could be if the first write had more data than the size of the kernel/fs-driver buffer, not all of it would be written, and then it could be interrupted when another thread calls write() with a small buffer that gets written in one go.

Re: In defense of simple architectures

#150

Post by kragen »

endorphine wrote: post #86
Beltalowda wrote: post #62

Earlier quoted context omitted.

Another issue with "just a JSON file" as a database is that you need to be a bit careful to avoid race conditions and the like, e.g. if two web pages try to write the same database at the same time. It's not an issue for all applications, and not that hard to get right, but does require some effort. This is a huge reason I prefer SQLite for simple file storage needs.

Doesn't the fact that its opened in append only mode (Linux) mitigate data races with regards to writes?

It eliminates them if they're smaller than PIPE_BUF (IIRC, Beltalowda, dmoy, and stevenhuang are wrong about this), but the thing that prevents data races with regard to writes is running the application in Node, which is completely single-threaded.
Post reply on HN