Live data from Hacker News

Ask HN: Is it possible to have forum type community running without a database?

news.ycombinator.com

11–20 of 66 posts

Re: Ask HN: Is it possible to have forum type community running without a database?

#11
post #3

You can run a community of sorts, but the feature list will be sparse. The only viable system sans-database that comes to mind is plain-text. I'm a member of a forum that had an old system in the beginning that used plain text "database" for a very long time. Performance was fairly reasonable at 5000 visitors per day, if the daily hit counter was to be trusted. They since moved it to SQLite, which is very reliable in…

Thanks, I appreciate the nice answer but it's amazing how often newbies get laughed at for being curious! Glad it hasn't happened today :-)

So on to your answer, I'm going to be running this community at way less than 5000 visitors per day! So from what I understand, the site was run using PHP and every time somebody made a new entry a new file was created on disk with a reference to that file in the main index page. With the name of the file was based on the time it was written to disk, with some checking to make sure to avoid files with duplicate names. I assume the main index file was the front page of the site?

Have I got that right? And if so, where do you think I should take my studying to set something like this up. I mean, are there any frameworks that you know of like Jekyll for instance or would I need to learn PHP before I could set it up?

Re: Ask HN: Is it possible to have forum type community running without a database?

#12

If the only requirement is to run it on a raspberry pi, and you find that a RDBMS will be too resource intensive, you can always use one of the cloud data storage services out there. Amazon has an RDS offering as part of AWS, MySQL, Postgres, etc. Then your app only needs to talk to the database over the wire with zero local resource consumption by the database.

Wait a minute, not sure I follow. Do you mean that I can run something like one of the forum communities I mentioned on the Raspberry Pi, but then offload all of the grunt work to Amazon?

Obviously that would mean that the Raspberry Pi would always need to be connected to the Internet which wouldn't believe much of a problem, but aren't there fairly significant privacy concerns with doing something like that? Or do you think it's possible to mitigate those concerns by encrypting the link to Amazon, I am only speculating wildly here while I furiously Google all of the acronyms you mentioned. :-)

Thanks!

Re: Ask HN: Is it possible to have forum type community running without a database?

#13

One of the main points of static sites is that they don't update terribly often. Forums, on the other hand, update constantly. You are wanting something with good caching (server side and client side), and thats about it. Otherwise, you are going to be regenerating the pages constantly on every post/update.

"good caching" needs to be "stores in cache forever, unless an update is required, at which point, only the content that actually changes is updated in cache" to be as good as static files. In my experience, caching is never that good. What's the problem with writing to the fs on every POST? The number of GETs will still outweigh the number of POSTs significantly.

Re: Ask HN: Is it possible to have forum type community running without a database?

#14

One of the main points of static sites is that they don't update terribly often. Forums, on the other hand, update constantly. You are wanting something with good caching (server side and client side), and thats about it. Otherwise, you are going to be regenerating the pages constantly on every post/update.

A static forum website seems like a contradiction in terms. Static means unchanging and forums must change by definition. I guess the author means the site will serve .html files?

The site could not be completely static, because something must be live to handle POST requests.

Re: Ask HN: Is it possible to have forum type community running without a database?

#15

If the only requirement is to run it on a raspberry pi, and you find that a RDBMS will be too resource intensive, you can always use one of the cloud data storage services out there. Amazon has an RDS offering as part of AWS, MySQL, Postgres, etc. Then your app only needs to talk to the database over the wire with zero local resource consumption by the database.

Wait a minute, not sure I follow. Do you mean that I can run something like one of the forum communities I mentioned on the Raspberry Pi, but then offload all of the grunt work to Amazon? Obviously that would mean that the Raspberry Pi would always need to be connected to the Internet which wouldn't believe much of a problem, but aren't there fairly significant privacy concerns with doing something like that? Or do y…

You can indeed run a forum community but offload the work to Amazon. Speaking about that, you can easily run everything on Amazon, because the latency between your webserver and the database might become a problem.

Privacy concerns wise, Amazon and Azure and the sorts are probably way better protected than your own installation. Of course the privacy with Amazon and Azure are all clearly written in their terms of usage and privacy statements. And NSA wise, if they get into an Amazon (either politically or technically) they will easily be able to hack your own installation.

Re: Ask HN: Is it possible to have forum type community running without a database?

#16
post #3

You can run a community of sorts, but the feature list will be sparse. The only viable system sans-database that comes to mind is plain-text. I'm a member of a forum that had an old system in the beginning that used plain text "database" for a very long time. Performance was fairly reasonable at 5000 visitors per day, if the daily hit counter was to be trusted. They since moved it to SQLite, which is very reliable in…

Thanks, I appreciate the nice answer but it's amazing how often newbies get laughed at for being curious! Glad it hasn't happened today :-) So on to your answer, I'm going to be running this community at way less than 5000 visitors per day! So from what I understand, the site was run using PHP and every time somebody made a new entry a new file was created on disk with a reference to that file in the main index page.…

There are a lot of open source products you can use, I would advise start at http://en.wikipedia.org/wiki/List_of_content_management_syst... to find a Wiki/CMS in the language you like most, with a flat-file database. Then look for plugins/extensions that provide the community features you need.

Re: Ask HN: Is it possible to have forum type community running without a database?

#17
Hi there, you ask interesting questions. I have two things for you that aren't exactly what you're looking for, but they are definitely relevant.

First of all, a while ago I made a serverless chat for in your browser. It's just a proof of concept, it does work however. There will be nobody there, but you can test it by opening two browser tabs or using a second device. It works in the same way that Skype works, peer to peer. It's cool, because it is in the browser. http://codepen.io/Azeirah/pen/BHnbz

Second of all, you know bitcoins, right? It has this blockchain thing. People are starting to realize the potential of the blockchain. I've seen multiple initiatives and examples of applications using the blockhain to store the data without a static database. Note however, the data is still stored somewhere, on the users' pc's!

This is not a database-less application, it is decentralized. It's still very cool however, and may have a huge influence on the internet in the future if it stabilizes.

https://eris.projectdouglas.org/ (and their github: https://github.com/project-douglas/eris)

Re: Ask HN: Is it possible to have forum type community running without a database?

#18
This is a fun exercise :)All right. Let us first analyze what the requirements should be. Obviously I am assuming a lot of things and thinking about this in a way I would go about building a (minimal)forum on raspberry pi:-

1. There should be a couple of predefined categories.

2. There should be a way to authenticate/authorize users.

3. There should be a way to search the previous posts.

4. There should be a facility to moderate.

Static forums

Very neat idea. All we need is a simple webserver, I am thinking, nginx that transforms and renders our posts to a user. Our first requirement that of predefined categories can be easily met by creating sub directories inside the directory that will be served. And we can avoid duplicating the pages between the sub directory (ie. a tag) and the main directory by using symbolic links[1] and configuring nginx to serve the symlinks [2]. Moderation of posts can be done by migrating a post in and our of a `moderate` folder that is not served by the application. But that obviously limits the moderation only to the webmaster. The trouble arises in two cases

a) When we need a facility to manage sessions. There is no way we can do that in static pages.

b) It will be quite difficult to create a search index for our application.

Now nginx is a good choice as it is pretty light weight and we will keep it. But what if we could have another application that can easily talk with nginx, manage user authentication for us, allow us to create search indexes, takes care of our data and still be light weight. Other have already suggested sqllite.How about couchdb?[3]

First couchdb has built in authentication and authorzation support. It will take you no more than 3 http requests to implement your own register-login-logout-scenario[4]. Second it is pretty easy creating simple search indexes with couchdb[5]. You can serve your forum directly from couchdb. It does not statisfy your orignal requirement of no database but it certainly gives you a way to work without any application layer. But perhaps most importantly it is very low on memory consumption. Not as low as sql lite but still quite low.

[1](http://stackoverflow.com/questions/1951742/how-to-symlink-a-...)

[2](http://nginx.org/en/docs/http/ngx_http_core_module.html#disa...)

[3](http://couchdb.apache.org/)

[4](http://www.staticshin.com/programming/easy-user-accounts-man...)

[5](http://wiki.apache.org/couchdb/View_collation#String_Ranges)

Re: Ask HN: Is it possible to have forum type community running without a database?

#19
post #8

Sure. HN itself has no database, only the filesystem (or rather the filesystem is the DB).

Really, Wow. Would you mind expanding a little on how that works? I mean, is HN just stored as a series of files on a server somewhere?

Ask PG: Database, flat files or other for YC News? https://news.ycombinator.com/item?id=99092

Re: Ask HN: Is it possible to have forum type community running without a database?

#20
You're probably overthinking this. While the Pi is limited in processing power and I/O, if it's really a small community it will run a forum just fine.

Here's what to look out for:

1) Make sure you run NginX, it's fast and memory-efficient.

2) Install PHP-FPM and have NginX connect to it using a socket file. Keep the number of FPM workers small.

2a) Zend Opcache comes standard with PHP now, but there are scenarios where you might want to look into using APC instead, especially if you want to hand-tweak some caching into the forum app. But see if it's necessary first.

3) I had good experiences with MariaDB, a MySQL fork, which comes with the Aria table type. If possible, change the forum tables to that type, it's very fast. The worst table type on a Pi is InnoDB, it's pretty is much unusable.

4) Beware of flash medium write wear, buy a larger-than-needed SD card or place files that update frequently on an external USB hard drive.

Post reply on HN