Running Redis in journaling mode is essentially this, too. Mongo can run like that too.
Re: Redis, only if you never compact the append-only file.
Now that people are considering NOSQL will more people consider no-DB
111–120 of 147 posts
Re: Now that people are considering NOSQL will more people consider no-DB
#112Earlier quoted context omitted.
When people set out to design a SQL database, they usually end up updating and deleting records. This is bad because it destroys history, and nothing that you can add to your SQL architecture will fix it at a fundamental level. By basing your system on a journaled event stream, you start with a foundation of complete history retention, and you can build exactly the sort of reporting views you need at any time (say, b…
You can have a design where your previous version of a record gets automatically copied into another table along with the timestamp of the operation. Then you can slice this history however you want. All with no additional app code. But I wouldn't write off the noDB approach for various transitional data, or data that isn't mean to live long anyway, like tweets.
Why would the lifetime of the data be at all relevant? Most noDB (a term I don't like much) stores are built to be highly durable.
There are a lot of questions that make the choice of data store a difficult one, but I'm not sure that plays into it at all.
Re: Now that people are considering NOSQL will more people consider no-DB
#113Earlier quoted context omitted.
"With multiple code bases touching the same data, schema improvements become nearly impossible." Few companies have procedures in place that allow this; but it is possible if you have the right procedures.
Sure, but the problem becomes harder the larger you get. Look at almost any Internet-wide deployment, though, and you see the alternative: isolate database schemas behind APIs, and rev APIs and schemas separately, as the situation demands.
Isolating databases behind APIs and rev-ing API+schema separately is not enough. When the schema changes, data must be transformed to match the new schema version. As you point out this takes too long with a large database, and it doesn't account for data consistency.
We have been working on building what we hope are the procedures for this with ChronicDB (http://chronincdb.com). But it turned out harder than it seems, and we are not sure it will quite work out. We'd welcome feedback.
Re: Now that people are considering NOSQL will more people consider no-DB
#114Wouldn't this system have a bunch of drawbacks: - Long startup times as the entire image needs to be loaded and prepared. - It would be hard to distribute the state across multiple nodes - What happens in case of a crash? How fault tolerant would this be? - Does this architecture essentially amount to building in a sort-of-kind-of datastore into your already complex application? Without a well-defined well-tested exi…
Re: Haskell it would probably look a lot like (exactly like?) Happs-State: http://happs.org/
Re: Now that people are considering NOSQL will more people consider no-DB
#115Perhaps I am an old dinosaur, but this article merely annoyed me. "The key element to a memory image is using event sourcing, which essentially means that every change to the application's state is captured in an event which is logged into a persistent store." That is a key element of a database. It's called a logical log. "Furthermore it means that you can rebuild the full application state by replaying these events…
When you say old-school DB do you mean something like mysql?
Re: Now that people are considering NOSQL will more people consider no-DB
#116Earlier quoted context omitted.
- Mirroring state is easy; you just pipe the serialized commands to multiple boxes. What? No, that's ridiculous. That's how inconsistencies crop up. Unless you plan on locking the entire system during each command.
One box is the master; the others are slaves. And yes, the easy way to do this is system having single write lock. That seems ridiculous if you are thinking like databases do, in terms of taking away the pain of all those disk seeks needed to write something. But if everything is hot in RAM, executing a command is extremely fast. Much faster than a database. If that still isn't fast enough, you can split your data gr…
The lock doesn't need to just cover one write. It needs to cover the whole transaction. The canonical example is that of incrementing a counter. Replica synchronization aside, without a transaction lock (or some other guarantee of transaction consistency), at some point you will read a counter value which another client is in the middle of updating.
The first "fix" to this that comes to mind is timestamping data, rolling back transactions which try to write having read outdated data. Do extant NoDB systems do this?
Re: Now that people are considering NOSQL will more people consider no-DB
#117Earlier quoted context omitted.
"All cores can read simultaneously." Unless someone is writing, of course, in which case you have to worry about isolation. So a single writer would block all readers, right? "You mean a bug in our code that causes a problem?" No, I mean like "I already wrote some data, but now a constraint has been violated so I need to undo it".
> So a single writer would block all readers, right? Correct. For the fraction of a millisecond the transaction is executing, anyhow. Since transactions only deal with data hot in RAM, transactions are very fast. > No, I mean like "I already wrote some data, but now a constraint has been violated so I need to undo it". That shouldn't happen, and I've used two approaches to make sure. One is do all your checking befor…
> Correct. For the fraction of a millisecond the transaction is executing, anyhow. Since transactions only deal with data hot in RAM, transactions are very fast.
Transactions don't just read and write. They sometimes compute things, like joins, which can take several milliseconds. These computations often must run within the transaction and would thus need to acquire the lock for several milliseconds.
Re: Now that people are considering NOSQL will more people consider no-DB
#118In many applications, data outlives code. This is certainly the case in enterprise applications, where data can sometimes migrate across several generations of an application. Data may also be more valuable to the organization than the code that processes it. While I'm no fan of databases, one obvious advantage is that they provide direct access to the data in a standard way that is decoupled from the specific applic…
When people set out to design a SQL database, they usually end up updating and deleting records. This is bad because it destroys history, and nothing that you can add to your SQL architecture will fix it at a fundamental level. By basing your system on a journaled event stream, you start with a foundation of complete history retention, and you can build exactly the sort of reporting views you need at any time (say, b…
I used to work on an application that did all the typical insert/update/delete operations on the core data tables and retained a journaled event stream in a separate audit database sufficient to regenerate the entire database from scratch (which was done at least a couple of times).
I suppose it would be equally possible to think of the main data tables as a "reporting view" in the sense you use here, except that the application was 1000:1 or more in update frequency to read frequency, and all the reads were performed on the main data tables, so that's kind of a "tail wagging the dog" view of the app.
For an application with different requirements, your view of things might be quite useful, of course.
Re: Now that people are considering NOSQL will more people consider no-DB
#119EventSourcing/CQRS (Command/Query Responsibility Segregation) is gaining a bit of traction in the .NET community. There are some great presentations[1], blogs[2][3] and projects[4] related to this architecture. [1]: http://www.infoq.com/presentations/Command-Query-Responsibil... [2]: http://www.udidahan.com/?blog=true [3]: http://blog.jonathanoliver.com/ [4]: https://github.com/joliver/EventStore/
Greg Young, himself, asserts that CQRS is NOT an architecture. And also asserts that CQRS itself has nothing to do with Event Sourcing.
http://codebetter.com/gregyoung/2010/02/16/cqrs-task-based-u...
"CQRS is not eventual consistency, it is not eventing, it is not messaging, it is not having separated models for reading and writing, nor is it using event sourcing."
It is gaining a lot of traction simply because it seems like a complicated and cool way to solve an uncommon problem. I fear that Event Sourcing will soon become a over-engineered hammer for the wrong nail.
If it were me, I would use CQRS principles, with a relational backend as my source model. Then when the need for scale arises, use ETL to either non-relational db or no-db for queries.
Re: Now that people are considering NOSQL will more people consider no-DB
#120After reading the article and all the comments here, and from my own experience, I just don't think it's possible to not have a DB. At best, you write your own basic DB, because you don't need anything fancy. For example, you write S-expressions to files like Hacker News does. This is clever, because the file system has some of the features of a database system, and files and S-expressions are abstractions that alrea…
[deleted]