Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

In defense of simple architectures

danluu.com

Post reply on HN
Print view View original

151–160 of 196 posts

Re: In defense of simple architectures

#151

Post by kragen »

mbrodersen wrote: post #132
pavlov wrote: post #34

There are some web apps still in production that I wrote almost a decade ago in Node+Express in the simplest, dumbest style imaginable. The only dependencies are Express and some third-party API connectors. The database is an append-only file of JSON objects separated by newlines. When the app restarts, it reads the file and rebuilds its memory image. All data is in RAM. I figured these toys would be replaced pretty…

I suspect that 95% of business applications could be implemented just fine with that architecture. However I would use SQLite instead of a plain file. Just for added commit safety.

For simple write patterns like this, I've had better robustness experiences with dumb filesystem access than with SQLite.

Re: In defense of simple architectures

#152

Post by kragen »

mapme wrote: post #142
lifefeed wrote: post #61

I was interviewing for software jobs recently, and while I was studying up on the "system design" portion I kept circling around the same insight that Dan Luu writes about so well here. I would sit down at an interview and try to create these "proper" system designs with boxes and arrows and failovers and caches and well tuned databases. But in the back of my mind I kept thinking, "didn't Facebook scale to a billion…

I think that’s a large oversimplification of Facebook. While it’s true a lot of FB storage is MySQL backed they also created many complex systems such as: - Cassandra (based on dynamo/big table) - wrote a custom KV store named RocksDb that is open source/now a company - wrote a custom photos storage system that replaced an NFS based design - wrote another custom binary object store - wrote a custom geo distributed gr…

I think all of this was after their first billion users?

Re: In defense of simple architectures

#153

Post by danenania »

conradfr wrote: post #147
danenania wrote: post #72

Earlier quoted context omitted.

Built-in first-class concurrency (ala node, golang, rust, etc.) is a huge win for simple architectures, since it lets you avoid adding a background queue, or at least delay it for a very long time. I think people are also too quick to add secondary data stores and caches. If you can do everything with a transactional SQL database + app process memory instead, that is generally going to save you tons of trouble on ops…

> Built-in first-class concurrency (ala node, golang, rust, etc.) is a huge win for simple architectures, since it lets you avoid adding a background queue, or at least delay it for a very long time. >For example: instead of memcache/redis, set aside ~100 MB of memory in your app process for an LRU cache. Erlang/Elixir for the win with (almost transparent multi-core) concurrency and ETS ;)

Oh yeah, Erlang/Elixir certainly belong in that list (probably at the front of it).

Re: In defense of simple architectures

#154

Post by aidos »

hu3 wrote: post #119
chubot wrote: post #97

Earlier quoted context omitted.

Reader mode in your browser goes a long way, I think they all have it now

Thanks. In Chrome I had to enable reader mode in chrome://flags

Ah thanks. I use it on mobile and did a cursory look for it on Chrome last night without success.

Re: In defense of simple architectures

#155

Post by dismantlethesun »

kragen wrote: post #152
mapme wrote: post #142

Earlier quoted context omitted.

I think that’s a large oversimplification of Facebook. While it’s true a lot of FB storage is MySQL backed they also created many complex systems such as: - Cassandra (based on dynamo/big table) - wrote a custom KV store named RocksDb that is open source/now a company - wrote a custom photos storage system that replaced an NFS based design - wrote another custom binary object store - wrote a custom geo distributed gr…

I think all of this was after their first billion users?

Cassandra was released in 2008. Facebook hit 1 billion users ~ 2012, and 2 billion ~ 2017. Back when Cassandra was released, they had a 'mere' 100 million users.

Facebook stats:

https://www.statista.com/statistics/264810/number-of-monthly...

Re: In defense of simple architectures

#156

Post by lliamander »

So, I definitely agree with this. Most of us don't have to do any thing at FAANG scale. But what counts as simple?

It's quite easy these days to deploy an app using AWS Lambda, DynamoDB, SNS, etc., all with a single Cloud Formation template. Is that simple? In one sense I've abstracted away a lot of the operational work that comes with self-hosted, but now I've intertwined (Rich Hickey might say complected) myself into Amazon's ecosystem.

Also, is a document store like DynamoDB, MongoDB, etc., simpler than a relational database like Postgres? On the one hand, a document database's interface is very simple compared to the complexity like SQL. On the other, that simplicity is generally considered a necessary sacrifice to scale. If you don't need to scale, why make the sacrifice?

Also, there can be simple things that are better at scaling. Elixir is a very nice scripting language like Ruby or Python, but it also has much better performance scaling (comparable with NodeJS or Go).

Re: In defense of simple architectures

#157

Post by bmn__ »

zie wrote: post #60
spicybright wrote: post #28

Earlier quoted context omitted.

I love irc, but is is just silly. Slack has much better history because you don't need to have been online when messages are sent to log them. Slack is absolutely more reliable in this regard. IRC is easy to script because the protocol is so simple. But you leave so much on the table for that cost. Obviously if your use case is text only that you don't care about being persistent and you lean heavily on scripting to…

Slack is not instantly "better" than IRC, it's just a different approach to the chat problem and it's arguably more approachable for people that don't want to learn about the chat space. Logging is just different between the two. For IRC, logging is outside the scope of the IRC protocol. Anyone can log anything anytime anywhere with whatever policies and procedures they want. This usually leads to each channel/projec…

> For IRC, logging is outside the scope of the IRC protocol.

Nope, the community has understood that server-side logging (and making it available to clients who missed stuff happening) is a useful thing. https://ircv3.net/specs/extensions/chathistory

Re: In defense of simple architectures

#158

Post by bmn__ »

dang wrote: post #75
Jtsummers wrote: post #51

Earlier quoted context omitted.

The "previous" article at the bottom is the most recent article in his archive, which was apparently published in March 2022. So I'm guessing this year, and either this month or last month. But the archive doesn't seem to have been updated yet with this article.

Ah ok - it's new then. Thanks to both of you!

FYI, GP wrote "the archive" and meant the index page: https://danluu.com

Article's Last-Modified header says 2022-04-06.

Re: In defense of simple architectures

#159

Post by bob1029 »

aprdm wrote: post #146
bob1029 wrote: post #59

Earlier quoted context omitted.

> The problem is always growth, either GC jank from a massive heap, running out of RAM, or those loops eventually catching up with you Absolutely. The challenge is having enough faith that it will take long enough to catch up to you. Statistically speaking, it won't catch up to you and if it does, it will take so long you should have seen it coming from miles away and had time to prepare. In my systems that use an in…

Could you expand what you mean by keeping pointers and basic index in memory?

Pointers are tuples of (Id, LogOffset) and are used to map logical identities to positions of those objects in the append-only log.

Indexes are usually a tuple of (Some64BitKey, Id) and are used to map physical business keys to logical object identities. These entries are only candidates in the case where the key material needs to be hashed and inspected for actual equivalence.

One big advantage with this approach is that you can stream big blobs directly out of the log to a caller-supplied buffer or stream. No intermediate allocations required aside from some small buffers.

Re: In defense of simple architectures

#160

Post by Beltalowda »

mpyne wrote: post #118
Beltalowda wrote: post #78

Earlier quoted context omitted.

I set a user style in Stylus for danluu.com: body { font: 16px/1.6em sans-serif; max-width: 50em; margin: auto; } Can even add it manually in the inspector if you want.

I do something similar, except it's the CSS from whatever of the 'Better Fucking Website' hits come up for me on search each time. I do wish I didn't have to do that though.

I find it's a lot easier to add a simple CSS to an understyled website, than remove huge fixed banners, weird low-contrast thin fonts, etc. from overstyled websites.
Post reply on HN