Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

In defense of simple architectures

danluu.com

Post reply on HN
Print view View original

171–180 of 196 posts

Re: In defense of simple architectures

#171

Post by mapme »

kragen wrote: post #170
dismantlethesun wrote: post #155

Earlier quoted context omitted.

Cassandra was released in 2008. Facebook hit 1 billion users ~ 2012, and 2 billion ~ 2017. Back when Cassandra was released, they had a 'mere' 100 million users. Facebook stats: https://www.statista.com/statistics/264810/number-of-monthly...

Thanks for the correction! So the more correct statement would be that Facebook scaled to 100 million users with PHP, MySQL, and Memcache, and then to a billion users with Cassandra and Haystack, and then built all the other stuff after their first billion?

Yeah, there’s also a lag sometimes ie. They write the paper 1-2 years after building the system. They’ve published some really interesting papers over time. If you filter by year you can see them all!

https://research.facebook.com/publications/

Re: In defense of simple architectures

#172

Post by treis »

woah wrote: post #168
treis wrote: post #66

Earlier quoted context omitted.

IMHO the important thing is where your data is. If can all be client side then write a SPA. If it's on the server then the more you do on the server the better. Returning HTML and doing a simple element replace with the new content is 99.9% indistinguishable from a SPA.

Every app will have a bunch of serverside data so IMO that's not really a consideration. The great thing about SPAs is that your server now just returns a simple JSON API. It greatly simplifies the server side. It also tends to make things a lot easier to unit test on the server side. Also, it cleanly segments your staffing requirements. If your app is in some way non-trivial, you can have domain experts working on t…

>Every app will have a bunch of serverside data so IMO that's not really a consideration.

Every app will not have so much data that they can't send it all to the client. 99% of them will which is why 99% of web apps should be server side. That 1% is when you want a heavy client side app.

>Also, it cleanly segments your staffing requirements. If your app is in some way non-trivial, you can have domain experts working on the backend, and just dumping json into http responses, and have a frontend engineer who doesn't understand any of the magic work on the frontend.

You can separate frontend and backend code without introducing a client server call in the middle.

>Once you're doing this, your backend engineers are dealing with html in addition to whatever their real job is.

Why? You can still have front end engineers. They just write server side HTML templates which is hardly any different than writing JS based HTML templates.

Re: In defense of simple architectures

#173

Post by kragen »

rendall wrote: post #141
scrubs wrote: post #27

Nah, I don't much like the tone of this article. Not at all. The engineering message should be: keep your architecture as simple as possible. And here are some ways (to follow) on how to find that minimal and complete size 2 outfit foundation in your size 10 hoarder-track-suite-eye-sore. Do we really need to be preached at with a warmed over redo of `X' cut it for me as a kid so I really don't know why all the kids t…

I'm not sure why this particular author is so popular on HN, but he hits the front page regularly.

Because he says things that are true and less well known than they should be, and then gives a clearly written argument that shows you that they are true using logic and empirical evidence.

So far that just describes any good science paper or math textbook; the difference is that he's writing about questions of great interest to HN, like in this case "how to build a web service", "how to do software version tracking" (https://danluu.com/monorepo/), "how to do statistics" (https://danluu.com/linear-hammer/), "why hardware development is hard" (https://danluu.com/why-hardware-development-is-hard/), "why everything is broken" (https://danluu.com/nothing-works/), and "how to hire talented people" (https://danluu.com/talent/). These are topics where there is an enormous amount of hot air out there on the web, but very little that is epistemologically justifiable.

Re: In defense of simple architectures

#174

Post by musicale »

sitkack wrote: post #40
musicale wrote: post #16

Earlier quoted context omitted.

It's a nice demonstration of the efficiency of web apps vs. native apps.

It really has nothing to do with that. The slack client is just written poorly.

> The slack client is just written poorly.

Why yes, Slack does seem to be written poorly - just as poorly as every other web app that I use including VS Code, Discord, MS Teams, etc..

Maybe the Slack developers are just stupid, uneducated, malicious, poorly managed, or ambivalent (or all of the above) but the platform does to be conducive to creating clunky and bloated software.

Re: In defense of simple architectures

#175

Post by derefr »

eternityforest wrote: post #128
derefr wrote: post #116

Earlier quoted context omitted.

How do you feel about SQLite? Because when I read this architecture description, it mostly vibes with me, until I think about what happens to the data in the event of a power cut. Is there logic in your app to potentially throw the last line away (incl. truncating it from the file) if it's invalid due to being the result of a non-atomic write? If so, seems a bit Not-Invented-Here compared to just using a (runtime-emb…

SQLite doesn't do "lines" like a text file, or truncate anything(Or, it probably does internally, but that's not how users think of it). It's a real SQL DB with real records and transactions, and it is one of the most trusted and reliable pieces of software ever made. Like, check out the change log to get a sense of how they do stuff.

I think you misread what I said, or maybe didn't read the post I was replying to. I was pointing out that flat files have a problem with truncation; and that SQLite, despite being a very different thing than an append-only text file, can be effectively used as "an append only text file, but ACID." My question on how the GP poster feels about SQLite, is down to SQLite being the obvious solution to a problem I wasn't sure they realized they had — but also potentially still being "too many dependencies" for them.

Re: In defense of simple architectures

#176

Post by derefr »

pavlov wrote: post #122
derefr wrote: post #116

Earlier quoted context omitted.

How do you feel about SQLite? Because when I read this architecture description, it mostly vibes with me, until I think about what happens to the data in the event of a power cut. Is there logic in your app to potentially throw the last line away (incl. truncating it from the file) if it's invalid due to being the result of a non-atomic write? If so, seems a bit Not-Invented-Here compared to just using a (runtime-emb…

The AOF reader will discard and emit a warning about lines that can’t be parsed, but that’s the extent of it. These apps are on Digital Ocean, and I don’t remember ever having unplanned downtime with them. They do sometimes migrate instances with advance notice, but that’s a clean shutdown. I’m sure SQLite is a better choice for almost any app. My reason for not using it was to try avoiding dependencies out of curios…

> My reason for not using it was to try avoiding dependencies out of curiosity, and also that I honestly really don’t like writing SQL — it just feels boring and error-prone.

Well, sure, but you can just read and write JSON blobs to a single-column table in SQLite. See also, "SQLite makes a better BLOB store than the filesystem does" (https://www.sqlite.org/fasterthanfs.html)

Re: In defense of simple architectures

#177

Post by spiffytech »

brundolf wrote: post #117
Beltalowda wrote: post #62

Earlier quoted context omitted.

Another issue with "just a JSON file" as a database is that you need to be a bit careful to avoid race conditions and the like, e.g. if two web pages try to write the same database at the same time. It's not an issue for all applications, and not that hard to get right, but does require some effort. This is a huge reason I prefer SQLite for simple file storage needs.

A normal Express app (assuming it's one process per JSON file) shouldn't have that problem, because JavaScript is single-threaded

It can definitely be a problem in Node.js. Assuming the workflow is read from disk -> modify -> write to disk, and that you're using the async fs functions, two async code paths running at the same time will have last-write-wins semantics and will lose data.

That's the naive scenario. If all code paths write out a global data structure, then it'd be fine. Or if the file is written append-only instead of as a single, atomic data structure, then it could be fine.

Re: In defense of simple architectures

#178

Post by kirab »

weaksauce wrote: post #121
uuyi wrote: post #88

Earlier quoted context omitted.

I would love to see more stuff like that. An application I have written recently for personal use is a double entry accounting system after GNUcash hosed itself and gave me a headache. This is based on Go and SQLite. The entire thing is one file (go embed rocks) and serves a simple http interface with a few JS functions like it is 2002 again. The back end is a proper relational model that is stored in one .db file. I…

sqlite really does very well with reads but not as much for locking writes. not saying it couldn't scale to many users but I think the other person is a bit optimistic on a double entry accounting app being only reads. I would imagine it could certainly easily serve a few hundred though if not a few thousand.

SQLite does really well for locking writes as well as soon as you activate the WAL.

Re: In defense of simple architectures

#179

Post by deterministic »

kragen wrote: post #151
mbrodersen wrote: post #132

Earlier quoted context omitted.

I suspect that 95% of business applications could be implemented just fine with that architecture. However I would use SQLite instead of a plain file. Just for added commit safety.

For simple write patterns like this, I've had better robustness experiences with dumb filesystem access than with SQLite.

That makes sense.

Re: In defense of simple architectures

#180

Post by rendall »

kragen wrote: post #173
rendall wrote: post #141

Earlier quoted context omitted.

I'm not sure why this particular author is so popular on HN, but he hits the front page regularly.

Because he says things that are true and less well known than they should be, and then gives a clearly written argument that shows you that they are true using logic and empirical evidence. So far that just describes any good science paper or math textbook; the difference is that he's writing about questions of great interest to HN, like in this case "how to build a web service", "how to do software version tracking"…

I don't disagree with you, but other people write on these topics more compellingly, and do not include off-putting Wolfram/Doctorow-style self regard. I chalk it up to his being astoundingly prolific.

https://news.ycombinator.com/from?site=danluu.com

Post reply on HN