Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

In defense of simple architectures

danluu.com

Post reply on HN
Print view View original

91–100 of 196 posts

Re: In defense of simple architectures

#91

Post by danenania »

gregmac wrote: post #83
danenania wrote: post #72

Earlier quoted context omitted.

Built-in first-class concurrency (ala node, golang, rust, etc.) is a huge win for simple architectures, since it lets you avoid adding a background queue, or at least delay it for a very long time. I think people are also too quick to add secondary data stores and caches. If you can do everything with a transactional SQL database + app process memory instead, that is generally going to save you tons of trouble on ops…

> For example: instead of memcache/redis, set aside a ~100 MB of memory in your app process for an LRU cache. When an object is requested, hit the DB with an indexed query for just the 'updatedAt' timestamp (should be a sub-10ms query). If it hasn't been modified, return the cached object from memory, otherwise fetch the full object from the DB and update the local cache. I've never built something with this type of…

It definitely does depend on various factors, but if your query is indexed, both the SQL DB request and the Redis/Memcache lookup of the full object are likely to be dominated by internal network latency. If your object is large, the DB single-field lookup could easily be faster since you're sending less back over the wire.

In other words, a single-field indexed DB lookup can be treated more like a cache request. Though for heavier/un-indexed queries, your "avoid round trips to the database" advice certainly applies.

With this architecture, the internal invalidation request is just an optimization. It isn't necessary and it doesn't matter if it fails, since you always check the timestamp with a strongly consistent DB read before returning a cached object.

Re: In defense of simple architectures

#92

Post by powersurge360 »

uuyi wrote: post #88
pavlov wrote: post #34

There are some web apps still in production that I wrote almost a decade ago in Node+Express in the simplest, dumbest style imaginable. The only dependencies are Express and some third-party API connectors. The database is an append-only file of JSON objects separated by newlines. When the app restarts, it reads the file and rebuilds its memory image. All data is in RAM. I figured these toys would be replaced pretty…

I would love to see more stuff like that. An application I have written recently for personal use is a double entry accounting system after GNUcash hosed itself and gave me a headache. This is based on Go and SQLite. The entire thing is one file (go embed rocks) and serves a simple http interface with a few JS functions like it is 2002 again. The back end is a proper relational model that is stored in one .db file. I…

Check out alpinejs or stimulusjs and combine it with htmx to get to a SPA like experience with very little additional complexity! Htmx let’s you serve partials over the wire instead of a page load so you can update the page incrementally and alpine and stimulus are both tools to add JS sprinkles like you’ve described in a way that is unobtrusive.

Re: In defense of simple architectures

#93

Post by yakshaving_jgt »

ChrisMarshallNY wrote: post #21
snvzz wrote: post #15

Earlier quoted context omitted.

Slack comically uses gigabytes of RAM and plenty of CPU time in the client side.

Obligatory I Am Developer toon: https://twitter.com/iamdevloper/status/1072503943790497798/p...

I wonder who he stole that joke from.

Re: In defense of simple architectures

#94

Post by ChrisMarshallNY »

yakshaving_jgt wrote: post #93
ChrisMarshallNY wrote: post #21

Earlier quoted context omitted.

Obligatory I Am Developer toon: https://twitter.com/iamdevloper/status/1072503943790497798/p...

I wonder who he stole that joke from.

I would assume bruised_blood, but I can't [easily] find the original, so I posted that.

Re: In defense of simple architectures

#95

Post by Beltalowda »

endorphine wrote: post #86
Beltalowda wrote: post #62

Earlier quoted context omitted.

Another issue with "just a JSON file" as a database is that you need to be a bit careful to avoid race conditions and the like, e.g. if two web pages try to write the same database at the same time. It's not an issue for all applications, and not that hard to get right, but does require some effort. This is a huge reason I prefer SQLite for simple file storage needs.

Doesn't the fact that its opened in append only mode (Linux) mitigate data races with regards to writes?

Your write will be fine; that is, it's not as if data from one write will be interspersed with the data from another write. It's just that the order might be wrong, or opening the file multiple times (possibly from multiple processes) could be fun too. The program or computer crashing mid-write can also cause problems. Things like that.

Again, may not be an issue at all for loads of applications. But I used a lot of "flat file databases" in the past, and found it's not an issue right up to the point that it is. Overall, I found SQLite simple, fast, and ubiquitous enough to serve as a good fopen() replacement. In some cases it can even be faster!

Re: In defense of simple architectures

#96

Post by uuyi »

powersurge360 wrote: post #92
uuyi wrote: post #88

Earlier quoted context omitted.

I would love to see more stuff like that. An application I have written recently for personal use is a double entry accounting system after GNUcash hosed itself and gave me a headache. This is based on Go and SQLite. The entire thing is one file (go embed rocks) and serves a simple http interface with a few JS functions like it is 2002 again. The back end is a proper relational model that is stored in one .db file. I…

Check out alpinejs or stimulusjs and combine it with htmx to get to a SPA like experience with very little additional complexity! Htmx let’s you serve partials over the wire instead of a page load so you can update the page incrementally and alpine and stimulus are both tools to add JS sprinkles like you’ve described in a way that is unobtrusive.

I appreciate the notion but my objective was to do the exact opposite of this and keep away from external dependencies and scripts where possible, apart from the solitary go-sqlite3.

The result is about 30K of source (including Go, CSS, HTML templates) which is less than minified alpinejs!

Re: In defense of simple architectures

#97

Post by chubot »

gherkinnn wrote: post #47

At the risk of making an ad-hominem attack, I found this website unreadable. Minimalism is fine. But there comes a point when there's so little, it is nothing. danluu.com is a bucket of sand facing an overbuilt cathedral.

Reader mode in your browser goes a long way, I think they all have it now

Re: In defense of simple architectures

#98

Post by jkaptur »

pavlov wrote: post #63
dmw_ng wrote: post #41

Earlier quoted context omitted.

> database is an append-only file of JSON objects separated by newlines. When the app restarts, it reads the file and rebuilds its memory image. All data is in RAM Apps like this tend to perform like an absolute whippet too (or if they dont, getting them to perform well is often a 5 line change). It's really freeing to be able to write scans and filters with simple loops that still return results faster than a networ…

Yes, you need to be sure that you understand the growth pattern if you want to YOLO in RAM. If your product aims to be the next Instagram, this is clearly not the architecture. But a lot of small businesses are genuinely small. They may not sign up new customers that often. When they do, the impact to the service is often very predictable ("Amy at customer X uses this every other day, she's very happy, it generates 1…

> If your product aims to be the next Instagram, this is clearly not the architecture.

But maybe! https://instagram-engineering.com/dismissing-python-garbage-...

Re: In defense of simple architectures

#99

Post by CraigJPerry »

uuyi wrote: post #88
pavlov wrote: post #34

There are some web apps still in production that I wrote almost a decade ago in Node+Express in the simplest, dumbest style imaginable. The only dependencies are Express and some third-party API connectors. The database is an append-only file of JSON objects separated by newlines. When the app restarts, it reads the file and rebuilds its memory image. All data is in RAM. I figured these toys would be replaced pretty…

I would love to see more stuff like that. An application I have written recently for personal use is a double entry accounting system after GNUcash hosed itself and gave me a headache. This is based on Go and SQLite. The entire thing is one file (go embed rocks) and serves a simple http interface with a few JS functions like it is 2002 again. The back end is a proper relational model that is stored in one .db file. I…

>> This whole concept could scale to tens of users

I strongly suspect this approach scales to tens of thousands of users. Maybe 30-40k users would be my guess on a garden variety intel i5 desktop from the past 3 years or so.

I say this because that hardware (assuming NVMe storage) will do north of 100k connect + select per second (connect is super cheap in sqlite, you're just opening a local file), assuming 2-3 selects per page serve gets me to the 30-40k number. The http server side won't be the bottleneck unless there's some seriously intensive logic being run.

Re: In defense of simple architectures

#100

Post by uuyi »

CraigJPerry wrote: post #99
uuyi wrote: post #88

Earlier quoted context omitted.

I would love to see more stuff like that. An application I have written recently for personal use is a double entry accounting system after GNUcash hosed itself and gave me a headache. This is based on Go and SQLite. The entire thing is one file (go embed rocks) and serves a simple http interface with a few JS functions like it is 2002 again. The back end is a proper relational model that is stored in one .db file. I…

>> This whole concept could scale to tens of users I strongly suspect this approach scales to tens of thousands of users. Maybe 30-40k users would be my guess on a garden variety intel i5 desktop from the past 3 years or so. I say this because that hardware (assuming NVMe storage) will do north of 100k connect + select per second (connect is super cheap in sqlite, you're just opening a local file), assuming 2-3 selec…

Interesting point. I may have to write a performance test suite for it now and test this.
Post reply on HN