Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Now that people are considering NOSQL will more people consider no-DB

martinfowler.com

Post reply on HN
Print view View original

121–130 of 147 posts

Re: Now that people are considering NOSQL will more people consider no-DB

#121

Post by silverbax88 »

joshu wrote: post #81
dgreensp wrote: post #63

After reading the article and all the comments here, and from my own experience, I just don't think it's possible to not have a DB. At best, you write your own basic DB, because you don't need anything fancy. For example, you write S-expressions to files like Hacker News does. This is clever, because the file system has some of the features of a database system, and files and S-expressions are abstractions that alrea…

Of course, filesystems are a kind of database as well.

Yes, agreed, which is why I find Fowler's (and others) documents a little hard to take seriously. They ARE writing to disk, and doing a lot of other work to try and keep the data intact in case of failure...which is what a DB does. I'm all for the idea of keeping some data in memory for speed, but moving it all to resident memory is just moving the same components around.

Re: Now that people are considering NOSQL will more people consider no-DB

#122

Post by wpietri »

colanderman wrote: post #117
wpietri wrote: post #93

Earlier quoted context omitted.

> So a single writer would block all readers, right? Correct. For the fraction of a millisecond the transaction is executing, anyhow. Since transactions only deal with data hot in RAM, transactions are very fast. > No, I mean like "I already wrote some data, but now a constraint has been violated so I need to undo it". That shouldn't happen, and I've used two approaches to make sure. One is do all your checking befor…

> Correct. For the fraction of a millisecond the transaction is executing, anyhow. Since transactions only deal with data hot in RAM, transactions are very fast. Transactions don't just read and write. They sometimes compute things, like joins, which can take several milliseconds. These computations often must run within the transaction and would thus need to acquire the lock for several milliseconds.

Do you have a real-world example?

Joins haven't been a problem for me, mainly because this approach doesn't constrain you to a tables-and-joins model of the world. With Prevayler, for example, you treat things as a big object graph, so there are no splits to join back up.

Of course, it could be that some problem is just computationally intense, but I can think of a number of approaches to lessen the impact of that in a NoDB system.

Re: Now that people are considering NOSQL will more people consider no-DB

#123

Post by wpietri »

colanderman wrote: post #116
wpietri wrote: post #80

Earlier quoted context omitted.

One box is the master; the others are slaves. And yes, the easy way to do this is system having single write lock. That seems ridiculous if you are thinking like databases do, in terms of taking away the pain of all those disk seeks needed to write something. But if everything is hot in RAM, executing a command is extremely fast. Much faster than a database. If that still isn't fast enough, you can split your data gr…

The lock doesn't need to just cover one write. It needs to cover the whole transaction . The canonical example is that of incrementing a counter. Replica synchronization aside, without a transaction lock (or some other guarantee of transaction consistency), at some point you will read a counter value which another client is in the middle of updating. The first "fix" to this that comes to mind is timestamping data, ro…

Yes, these approaches provide proper transactions, but the approach is much simpler than you imagine.

Imagine you have a graph of objects in RAM. Say, a Set of Counters. Imagine you want to change the graph by incrementing a counter. To do this create an object, the UpdateCounterCommand, with one method: execute. You hand the command to the system, and it puts it in a queue. When it gets to the top, it serializes the command to disk and then executes it. Exactly one write command runs at a time.

For a real-world example, check out Prevayler. It provides all the ACID guarantees that a database does, but in a very small amount of code.

Re: Now that people are considering NOSQL will more people consider no-DB

#124

Post by kfool »

dgreensp wrote: post #63

After reading the article and all the comments here, and from my own experience, I just don't think it's possible to not have a DB. At best, you write your own basic DB, because you don't need anything fancy. For example, you write S-expressions to files like Hacker News does. This is clever, because the file system has some of the features of a database system, and files and S-expressions are abstractions that alrea…

Was there something specific it the data model that made the versioning hard to write? Or was it that, for this to work across the board, the entire model had to be versioned?

It sounds that SQL itself wasn't the problem. Were you looking for versioning alternatives in SQL that weren't up to par?

Re: Now that people are considering NOSQL will more people consider no-DB

#125

Post by ivanhoe »

It looks more like a different DB engine implementation than no-DB system... you still need a structured data persisted somewhere, difference here is in the way you store it and how you buffer it, but IMHO that all can be seen simply as in-memory "DB engine"

Re: Now that people are considering NOSQL will more people consider no-DB

#126

Post by artsrc »

Having a database model and an application model in separate processes, and mapping between the two is expensive in many ways.

There are many ways get rid of the problem with getting rid of the database. For example putting all knowledge of the domain in the database via stored procedures, couchapps, and object stores.

Re: Now that people are considering NOSQL will more people consider no-DB

#127

Post by artsrc »

matwood wrote: post #60
jcromartie wrote: post #50

Earlier quoted context omitted.

When people set out to design a SQL database, they usually end up updating and deleting records. This is bad because it destroys history, and nothing that you can add to your SQL architecture will fix it at a fundamental level. By basing your system on a journaled event stream, you start with a foundation of complete history retention, and you can build exactly the sort of reporting views you need at any time (say, b…

When people set out to design a data driven application, they usually end up updating and deleting records. FTFY... It's not hard to build history into a SQL table design. You can even store events in a...wait for it... SQL database. I have built numerous systems backed by SQL databases that have complete history retention. Answering questions like 'who had id 'X' on this date 3 years ago' are easily solvable with ba…

You loose some features of SQL when you have historical values.

For example if you only want customers to create orders you can set a foreign key between customer and order. If you don't delete orders and customers, then the database will allow you to insert new orders against old customers, unless you apply a more complex constraint.

Re: Now that people are considering NOSQL will more people consider no-DB

#128

Post by kragen »

ynd wrote: post #115
zwischenzug wrote: post #96

Perhaps I am an old dinosaur, but this article merely annoyed me. "The key element to a memory image is using event sourcing, which essentially means that every change to the application's state is captured in an event which is logged into a persistent store." That is a key element of a database. It's called a logical log. "Furthermore it means that you can rebuild the full application state by replaying these events…

When you say old-school DB do you mean something like mysql?

Yes, MySQL is the kind of database that canonically uses approaches like physical logs and logical logs to provide ACID transactions (and, in MySQL in particular, replication.)

The interesting thing that Prevayler and such things did was that they expanded the use of logical logs beyond simple relational tables to much richer sets of state.

Re: Now that people are considering NOSQL will more people consider no-DB

#129

Post by kfool »

jcromartie wrote: post #104
matwood wrote: post #60

Earlier quoted context omitted.

When people set out to design a data driven application, they usually end up updating and deleting records. FTFY... It's not hard to build history into a SQL table design. You can even store events in a...wait for it... SQL database. I have built numerous systems backed by SQL databases that have complete history retention. Answering questions like 'who had id 'X' on this date 3 years ago' are easily solvable with ba…

It's not hard, no, but it usually doesn't happen in the average application. That's the issue: it's not built in, it's not standardized, and every SQL database is fully mutabile by default. If your system operates in this journaled/event-sourcing way at the most basic level then you have the ultimate future-proof storage layer. You could decide to completely change the way the data is stored and represented (in-memor…

That's one of the things ChronicDB does. It makes historical values available in the average application.

Re: Now that people are considering NOSQL will more people consider no-DB

#130

Post by kfool »

artsrc wrote: post #127
matwood wrote: post #60

Earlier quoted context omitted.

When people set out to design a data driven application, they usually end up updating and deleting records. FTFY... It's not hard to build history into a SQL table design. You can even store events in a...wait for it... SQL database. I have built numerous systems backed by SQL databases that have complete history retention. Answering questions like 'who had id 'X' on this date 3 years ago' are easily solvable with ba…

You loose some features of SQL when you have historical values. For example if you only want customers to create orders you can set a foreign key between customer and order. If you don't delete orders and customers, then the database will allow you to insert new orders against old customers, unless you apply a more complex constraint.

This can be handled by deleting orders from the logical representation, yet preserving them in the historical layer.
Post reply on HN