Now that people are considering NOSQL will more people consider no-DB

#1

Post by fogus » Wed, Aug 31, 2011, 1:52 PM UTC

Now that people are considering NOSQL will more people consider no-DB
martinfowler.com

Re: Now that people are considering NOSQL will more people consider no-DB

#2

Post by [deleted] » Wed, Aug 31, 2011, 2:05 PM UTC

[deleted]

Re: Now that people are considering NOSQL will more people consider no-DB

#3

Post by adelevie » Wed, Aug 31, 2011, 2:33 PM UTC

This seems like an excellent idea. Is it possible to preserve most of the APIs/design patterns we're used to working with for (no)SQL when using Event Sourcing?

Re: Now that people are considering NOSQL will more people consider no-DB

#4

Post by kahawe » Wed, Aug 31, 2011, 2:36 PM UTC

...and on a mildly related subject: more people should consider LDAP.

Re: Now that people are considering NOSQL will more people consider no-DB

#5

Post by mmatants » Wed, Aug 31, 2011, 2:37 PM UTC

I am 100% agreeing with the article, with one caveat.

Database engines are not just for storing - each is basically a "utility knife" of data retrieval - indexing, sorting and filtering are available via (relatively) simple SQL constructs. If your app uses an index right now, ditching the DB will mean re-implementing it manually. It's not hard, but it's extra code.

So basically, the DB engine might still be a necessary "library", at least for data retrieval. A middle-of-the-road take on this is e.g. using an in-memory Sqlite instance to perform indexing, etc - seeding it at run-time to help with data searches, but then still not using it for storing persistent information and discarding the data at the end.

Re: Now that people are considering NOSQL will more people consider no-DB

#6

Post by pointyhat » Wed, Aug 31, 2011, 2:39 PM UTC

I've always wanted "no-DB" to the level of it being part of the platform/language.

I've always thought that software-transactional memory and persistent distributed heaps would get us there. Unfortunately the nearest things have been Redis and Terracotta plugged into Clojure. It should be:

Insert? new an object. Delete? dispose an object. Look up? Hash table.

Solved problems that just require persistence.

Re: Now that people are considering NOSQL will more people consider no-DB

#7

Post by courtewing » Wed, Aug 31, 2011, 2:43 PM UTC

No matter how skilled I become as a developer, there is always something lurking around the corner to make me feel more naive than ever. As I was reading this article, I realized that my whole career and knowledge about the way applications work is based around the one core idea that when non-binary data needs to be persisted, you use a database.

The idea that you can reliably use event sourcing in memory to persist your data is as foreign to me as it is impressive. Is anyone familiar with major applications (web apps, ideally) that use this method for their data persistence?

Re: Now that people are considering NOSQL will more people consider no-DB

#8

Post by IgorPartola » Wed, Aug 31, 2011, 2:44 PM UTC

Wouldn't this system have a bunch of drawbacks:

- Long startup times as the entire image needs to be loaded and prepared.

- It would be hard to distribute the state across multiple nodes

- What happens in case of a crash? How fault tolerant would this be?

- Does this architecture essentially amount to building in a sort-of-kind-of datastore into your already complex application? Without a well-defined well-tested existing code base, is this just re-inventing the wheel for each new project?

- How do you enforce constraints on the data?

- How do transactions work (debit one account, [crash], credit another account?

- How do you allow different components (say web user interface, admin system, reporting system, external data sources) to share this state?

Just curious.

EDIT:

- Isn't this going to lead to you writing code that almost always has side-effects, causing it to be really hard to test? How would you implement this system in Haskell?

Re: Now that people are considering NOSQL will more people consider no-DB

#9

Post by wpietri » Wed, Aug 31, 2011, 2:45 PM UTC

I'd encourage everybody to try this out; building an app like this really broadened my way of thinking about system design.

Compared with a database-backed system, many operations are thousands of times faster. Some things that I was used to thinking of as impossible became easy, and vice versa. Coming to grips with why was very helpful.

Re: Now that people are considering NOSQL will more people consider no-DB

#10

Post by yesimahuman » Wed, Aug 31, 2011, 2:50 PM UTC

I think it's interesting you can distribute more of your "persistent" state to in-memory storage and then distribute snapshots throughout the day. Online game servers often rely on state being in memory rather than being queried on demand. Achieving high performance otherwise is difficult.

However, I wouldn't call this "no-DB." Rather, it's "less-db." Ultimately, historical and statistical data needs to be stored and databases are great for that (and for a stats team).