Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Now that people are considering NOSQL will more people consider no-DB

martinfowler.com

Post reply on HN
Print view View original

21–30 of 147 posts

Re: Now that people are considering NOSQL will more people consider no-DB

#21

Post by wpietri »

courtewing wrote: post #7

No matter how skilled I become as a developer, there is always something lurking around the corner to make me feel more naive than ever. As I was reading this article, I realized that my whole career and knowledge about the way applications work is based around the one core idea that when non-binary data needs to be persisted, you use a database. The idea that you can reliably use event sourcing in memory to persist…

You're already familiar with a couple of things that can be built this way: word processors and multiplayer game servers. In both cases SQL databases are too slow and too awkward.

Financial trading is another area where databases are too slow. I know of one place that uses this approach to keep pricing data hot in RAM for their financial models. And Fowler previously documented using this for a financial exchange:

http://martinfowler.com/articles/lmax.html

Re: Now that people are considering NOSQL will more people consider no-DB

#22

Post by jcromartie »

pointyhat wrote: post #6

I've always wanted "no-DB" to the level of it being part of the platform/language. I've always thought that software-transactional memory and persistent distributed heaps would get us there. Unfortunately the nearest things have been Redis and Terracotta plugged into Clojure. It should be: Insert? new an object. Delete? dispose an object. Look up? Hash table. Solved problems that just require persistence.

There are some neat little libraries that help with this in Clojure. The basic idea is that when you introduce changes through transactions, the actual transaction (code) is appended to disk (which is very fast), and this becomes your "database" file. So, what is persisted is just a list of state changes that are "replayed" to restore state.

The individual transactions could also easily be distributed to multiple nodes via a message queue.

Re: Now that people are considering NOSQL will more people consider no-DB

#23

Post by dwolfson20 »

courtewing wrote: post #7

No matter how skilled I become as a developer, there is always something lurking around the corner to make me feel more naive than ever. As I was reading this article, I realized that my whole career and knowledge about the way applications work is based around the one core idea that when non-binary data needs to be persisted, you use a database. The idea that you can reliably use event sourcing in memory to persist…

This is the field we choose.

Expand your thinking in the abstract about what a database is. As 71104 mentions, a file system is also a database. What you are thinking of as "a database" is really a specific type of key-value store that is located on disk. But the fact most DBs are on disk has nothing to do with the concept itself.

Re: Now that people are considering NOSQL will more people consider no-DB

#24

Post by guygurari »

In many applications, data outlives code. This is certainly the case in enterprise applications, where data can sometimes migrate across several generations of an application. Data may also be more valuable to the organization than the code that processes it.

While I'm no fan of databases, one obvious advantage is that they provide direct access to the data in a standard way that is decoupled from the specific application code. This makes it easy to perform migrations, backups etc. It also increases one's confidence in the data integrity. Any solution that aims to replace databases altogether must address these concerns. I think that intimately coupling data with the application state, as suggested in the article, does not achieve this.

Re: Now that people are considering NOSQL will more people consider no-DB

#26

Post by Duff »

mmatants wrote: post #5

I am 100% agreeing with the article, with one caveat. Database engines are not just for storing - each is basically a "utility knife" of data retrieval - indexing, sorting and filtering are available via (relatively) simple SQL constructs. If your app uses an index right now, ditching the DB will mean re-implementing it manually. It's not hard, but it's extra code. So basically, the DB engine might still be a necessa…

The SQL constructs are great, but the biggest advantage to relational databases is that the engine handles your data consistency issues for you. Consistency isn't just about rolling the datastore back to a specific moment in time -- you have to handle locking, concurrent reads/writes, etc.

If you're building a trading platform that handles 6M transactions/second, you have the money to handle this in the application layer and the load to justify the expense. But for many other tasks, you may be wasting money or putting data at risk.

Re: Now that people are considering NOSQL will more people consider no-DB

#27

Post by andrewcooke »

wpietri wrote: post #9

I'd encourage everybody to try this out; building an app like this really broadened my way of thinking about system design. Compared with a database-backed system, many operations are thousands of times faster. Some things that I was used to thinking of as impossible became easy, and vice versa. Coming to grips with why was very helpful.

what implementation did you use? what others exist? thanks.

Re: Now that people are considering NOSQL will more people consider no-DB

#28

Post by gte525u »

andrewcooke wrote: post #27
wpietri wrote: post #9

I'd encourage everybody to try this out; building an app like this really broadened my way of thinking about system design. Compared with a database-backed system, many operations are thousands of times faster. Some things that I was used to thinking of as impossible became easy, and vice versa. Coming to grips with why was very helpful.

what implementation did you use? what others exist? thanks.

Isn't this essentially what prevayler http://en.wikipedia.org/wiki/Prevayler is?

Re: Now that people are considering NOSQL will more people consider no-DB

#29

Post by jkkramer »

courtewing wrote: post #7

No matter how skilled I become as a developer, there is always something lurking around the corner to make me feel more naive than ever. As I was reading this article, I realized that my whole career and knowledge about the way applications work is based around the one core idea that when non-binary data needs to be persisted, you use a database. The idea that you can reliably use event sourcing in memory to persist…

I don't know if it uses "event sourcing" per se, but doesn't HN use in-memory & serialized Lisp data structures instead of a DB?

Re: Now that people are considering NOSQL will more people consider no-DB

#30

Post by yk_42 »

EventSourcing/CQRS (Command/Query Responsibility Segregation) is gaining a bit of traction in the .NET community. There are some great presentations[1], blogs[2][3] and projects[4] related to this architecture.

[1]: http://www.infoq.com/presentations/Command-Query-Responsibil...

[2]: http://www.udidahan.com/?blog=true

[3]: http://blog.jonathanoliver.com/

[4]: https://github.com/joliver/EventStore/

Post reply on HN