...and on a mildly related subject: more people should consider LDAP.
You mean using data stores that support access through LDAP as general purpose databases?
Now that people are considering NOSQL will more people consider no-DB
11–20 of 147 posts
Re: Now that people are considering NOSQL will more people consider no-DB
#12Wouldn't this system have a bunch of drawbacks: - Long startup times as the entire image needs to be loaded and prepared. - It would be hard to distribute the state across multiple nodes - What happens in case of a crash? How fault tolerant would this be? - Does this architecture essentially amount to building in a sort-of-kind-of datastore into your already complex application? Without a well-defined well-tested exi…
- The startup times can be a problem if you have a lot of data. Modern disks are pretty fast for streaming reads, though, and you can split the deserialization load across multiple processors.
- Mirroring state is easy; you just pipe the serialized commands to multiple boxes.
- It's very fault tolerant. Because every change is logged before being applied, you just load the last snapshot and replay the log.
- It didn't seem that way to me.
- In code. In the system I built, each mutation was packaged as a command, and the commands enforced integrity.
- Each command is a transaction. As with DB transactions, you do have to be careful about where you draw your transaction boundaries.
- Via API. Which I like better, as it allows you to enforce more integrity than you can with DB constraints.
Re: Now that people are considering NOSQL will more people consider no-DB
#13answering directly to the subject: i do hope so. SQL too often introduces only a layer of complexity between the server-side application and the storage, while most of times an application could be designed to just use the filesystem, which is a database on its own by the way: it's a big, usually efficient, lookup table that maps keys (file paths) to values (file contents).
why store passwords through SQL when a server application could just use a specific directory containing one file for each user, each file named with the username and containing his password (without any file format, just the password, possibly hashed or encrypted)? the operating system I/O cache should be able to handle that efficiently and the advantage would be the elimination of the dependence on another software, the DBMS.
Re: Now that people are considering NOSQL will more people consider no-DB
#14Wouldn't this system have a bunch of drawbacks: - Long startup times as the entire image needs to be loaded and prepared. - It would be hard to distribute the state across multiple nodes - What happens in case of a crash? How fault tolerant would this be? - Does this architecture essentially amount to building in a sort-of-kind-of datastore into your already complex application? Without a well-defined well-tested exi…
Mostly agree, but to this point:
> - Isn't this going to lead to you writing code that almost always has side-effects, causing it to be really hard to test? How would you implement this system in Haskell?
You'd have to figure out how to isolate the IO monad as much as possible, but this is no different than interacting with a database in Haskell. And Haskell would give you nice features like STM to address other concerns as well.
Re: Now that people are considering NOSQL will more people consider no-DB
#15I am 100% agreeing with the article, with one caveat. Database engines are not just for storing - each is basically a "utility knife" of data retrieval - indexing, sorting and filtering are available via (relatively) simple SQL constructs. If your app uses an index right now, ditching the DB will mean re-implementing it manually. It's not hard, but it's extra code. So basically, the DB engine might still be a necessa…
When you're 3-4 orders of magnitude ahead of the game because you've ditched network round trips and disk accesses, indices become less ( not completely un- ) necessary.
The impedance mismatch between database and application is a lot of code too.
Re: Now that people are considering NOSQL will more people consider no-DB
#16I am 100% agreeing with the article, with one caveat. Database engines are not just for storing - each is basically a "utility knife" of data retrieval - indexing, sorting and filtering are available via (relatively) simple SQL constructs. If your app uses an index right now, ditching the DB will mean re-implementing it manually. It's not hard, but it's extra code. So basically, the DB engine might still be a necessa…
Having built a couple of systems like this, I didn't find this to be a big problem. You have to organize your data in memory somehow, and that tended to be along lines that made for fast access. Occasionally I'd have to index something, which meant adding a hashtable here and there. It also allowed me to index and organize in ways that SQL doesn't make available.
The area where I still most needed SQL was for ad-hoc querying and reporting. I dreamed of building a relational-to-object mapper, but settled for a) XPath queries against our snapshot files, and b) dumping to an SQL database for reports.
Re: Now that people are considering NOSQL will more people consider no-DB
#17No matter how skilled I become as a developer, there is always something lurking around the corner to make me feel more naive than ever. As I was reading this article, I realized that my whole career and knowledge about the way applications work is based around the one core idea that when non-binary data needs to be persisted, you use a database. The idea that you can reliably use event sourcing in memory to persist…
It seems like email maps onto this model fairly well. An event (mail) comes in and gets written to the log (mbox). Starting the server (session) entails reading the events from file. It's not a perfect fit, of course.
The big players are backing their services with custom data stores, though.
Re: Now that people are considering NOSQL will more people consider no-DB
#18answering directly to the subject: i do hope so. SQL too often introduces only a layer of complexity between the server-side application and the storage, while most of times an application could be designed to just use the filesystem, which is a database on its own by the way: it's a big, usually efficient, lookup table that maps keys (file paths) to values (file contents). why store passwords through SQL when a serv…
One reason would be that you expect more than a few thousand users.
More generally, for all but the very simplest apps I'm afraid can't agree with you (though, to clarify, this isn't really addressing the topic of the original post). Years ago I had a summer job maintaing a set of Perl scripts that persisted online store inventories to flat files. It was horrible. I've never found Postgres to add much complexity to a project, but on the storage side it provides me with the reassurance that smart people have thought hard about issues of consistency, stability and performance -- which millions have tested -- and on the query side, I can do various clever things at amazing speed if ever that's required. Why would I not go for that?
Re: Now that people are considering NOSQL will more people consider no-DB
#19Using the no-DB approach is particularly tempting with a language like Clojure. Clojure can slice & dice collections easily and efficiently. It has built-in constructs for managing concurrency safely.
I actually have a couple Clojure apps that rely on a hefty amount of in-memory data to do some computations. Even the cost of pulling the data from Redis would be too expensive. The in-memory data grows very slowly, so it's easy to maintain. Moving faster-growing data in-process would be trickier, but this article makes me want to try.
Re: Now that people are considering NOSQL will more people consider no-DB
#20answering directly to the subject: i do hope so. SQL too often introduces only a layer of complexity between the server-side application and the storage, while most of times an application could be designed to just use the filesystem, which is a database on its own by the way: it's a big, usually efficient, lookup table that maps keys (file paths) to values (file contents). why store passwords through SQL when a serv…
Most file systems don't gracefully handle 100k files in a single directory. It's not what they are designed for. Disk block sizes also mean that each password may be consuming several kB. The space waste on a modern system might not matter, but inode exhaustion might be a real issue on Unix-type filesystems.