Earlier quoted context omitted.
Check out Rethink. It seems to be what you're after.
It's probably better to go with Cockroach or TiDB. RethinkDB has problems of its own.
What problems (outside I guess the current status of the project as a whole)?
MongoDB gets support for multi-document ACID transactions
221–230 of 245 posts
Re: MongoDB gets support for multi-document ACID transactions
#222Earlier quoted context omitted.
This is something you heard from people who never really used it I used it at a previous job. Project to move a multi-tera dataset from an Oracle box (24 CPUs, 24G RAM, SAN) to a MongoDB cluster (10 boxes, each with 48 cores, 96G RAM and internal SSD). MongoDB couldn't perform for shit, and it couldn't stay up in a usable state for more than a few hours at a time. This is with 20x the processors and 40x the memory of…
See this is the sort of crazy things I used to see people do and wonder why they had problems. MongoDB is a document database. You can't just take relational database tables, move them across and expect it to behave the same. And frankly I don't feel sympathy for bad engineering practice. You don't do system migrations without fully testing and understanding all of the systems. But for those of us that had document o…
See this is the sort of crazy things I used to see people do and wonder why they had problems.
Financial time series data is exactly one of the use cases Mongo claimed to be for. Seems you’re the one who can’t tell good engineering practice from bad. And yes, they also pitched themselves as a direct replacement for Oracle. That was highly disingenuous.
Re: MongoDB gets support for multi-document ACID transactions
#223Earlier quoted context omitted.
On the other hand, PostgreSQL is a very good example of a successful implementation of the opposite strategy, that is, "correctness first". And since PostgreSQL fills that niche very well (correctness + real ACID + extensibility + decent performance), maybe it was really PostgreSQL who killed RethinkDB?
If you're playing the long game and not looking to make a profit that's fine, but PostgreSQL as a company would have been doomed a long time ago. You have to keep in mind the timelines of the business and what they need to do to keep the lights on. MongoDB has identified a real pain point: many developers don't like to use SQL to interface with a transactional database. I'm not going into the merits of SQL vs. NoSQL,…
> MongoDB has identified a real pain point: many developers don't like to use SQL to interface with a transactional database.
The pain point relates less to SQL but more to the RDBMS and the rigid schema. SQL is spreading and may become a ubiquitous query language.
Re: MongoDB gets support for multi-document ACID transactions
#224Earlier quoted context omitted.
> otherwise you wouldn't be capitalizing on the advantages of JSON Shouldn't this be "otherwise you wouldn't be capitalizing on the advantages of a schemaless, document-oriented database"? The fact that it's JSON (vs. some other format) seems immaterial
As an aside, I hate the term 'schemaless' because there is always a schema. https://blog.jooq.org/2014/10/20/stop-claiming-that-youre-us...
Yep. Schema-on-read (E.g. MongoDB) or schema-on-write (E.g. Oracle)
Re: MongoDB gets support for multi-document ACID transactions
#225Earlier quoted context omitted.
> Having to resort to doing things like map/reduce for a simple group by / order is not the way to go IMO. Perhaps you are unaware, but that is a straw-man. That’s not how you’re supposed to with MongoDB. You’d use the aggregation framework: db.sales.aggregate( [ { $group : { _id : null, totalPrice: { $sum: { $multiply: [ "$price", "$quantity" ] } }, averageQuantity: { $avg: "$quantity" }, count: { $sum: 1 } } } ] )…
> db.sales.aggregate( [ { $group : { _id : null, totalPrice: { $sum: { $multiply: [ "$price", "$quantity" ] } }, averageQuantity: { $avg: "$quantity" }, count: { $sum: 1 } } } ] ) Now look at SQL version of it and tell me which one is easier: SELECT SUM(price * quantity), AVG(quantity), COUNT(*) FROM sales;
Am I the only one that finds it ridiculous that the way to query for data in SQL is by passing a string?
Re: MongoDB gets support for multi-document ACID transactions
#226Earlier quoted context omitted.
I wasn't responding to the distributed transaction argument, though nothing in the MongoDB blog says it won't support it. I was specifically responding to "one table one service" which is the foundation of domain driven micro service architecture. To have a single domain silo'd and tested within its own boundaries. Whether you're firing events between services or just calling other services is a design decision, but…
Just because something can be done, including the fashionable microservices "architecture", it does not mean that it should be done, or that it is a good idea.
Application architecture is inherently malleable, yes.
But when properly implemented, micro service architecture is actually a great idea.
I can’t help if your experience has not been the same as mine.
Re: MongoDB gets support for multi-document ACID transactions
#227Earlier quoted context omitted.
I am curious why a municipality needs custom software. I mean, the scandinavian countries had standardised paper forms for most municipal tasks (population register, ledgers etc) already in the 17-18th century, and those were used nationwide, or at least throughout a single province. Why can't the same be done with software?
Well there are 98 municipalities and 98 ways to operate in a thousand different ways. I’ve worked on quite a few multi-municipalitiy open source projects, like handling employee refunds on driving. Basically I drive x kilometers for a meeting, I get paid x and the taxman gets the report. Simple stuff. Well in the 6 parties involved there were 6 ways to interpret tax laws, 4 different agreements with unions on what ra…
Would it be fair to say that the political entity one step above the municipalities (whatever that is in Denmark) are not doing their job? I mean not doing their job on standardising things that can be in common between the municipalities. Some things will of course have to differ, but a lot of stuff likely differ just because not-invented-here. It sounds like the legislative environment is too complex, and that you have to work around it with a ton of software. Could it even be the case that computer systems have somewhat removed the incentive for the administration to rationalise the various systems? With just manual labor and typewriters all of that would have been very expensive, but with a server hall and a medium-size IT-team it kind of works out.
Perhaps digitalisation only having come half-way is a factor - you mention scanning, but by now the so called "paper free office" that was a buzzword in the 1990s should be here already. Or is it perhaps just another sign that the IT industry overall is still very immature and this will sort itself out with time?
Re: MongoDB gets support for multi-document ACID transactions
#228Earlier quoted context omitted.
Well there are 98 municipalities and 98 ways to operate in a thousand different ways. I’ve worked on quite a few multi-municipalitiy open source projects, like handling employee refunds on driving. Basically I drive x kilometers for a meeting, I get paid x and the taxman gets the report. Simple stuff. Well in the 6 parties involved there were 6 ways to interpret tax laws, 4 different agreements with unions on what ra…
Would it be fair to say that the political entity one step above the municipalities (whatever that is in Denmark) are not doing their job? I mean not doing their job on standardising things that can be in common between the municipalities. Some things will of course have to differ, but a lot of stuff likely differ just because not-invented-here. It sounds like the legislative environment is too complex, and that you…
I think it's too complicated to blame anyone really. I mean, we working on standardising as much as possible, but it's often impossible because business practices are just so different. Often big standard products fall extremely short, or end up in complete failures because you can't jam people into boxes on an enterprise scale especially not when the people who build the systems have next to no domain knowledge and the people who write contracts have no technical knowledge. :)
I guess our government should work on writing laws that are more friendly to digitisation and stop expecting IT to fix business practices that don't really make sense in the first place. There has been a genuine movement toward that, but it's slow because none of our top politicians or bureaucrats are from technical fields, and they operate on such a high strategic level that they're often rather far from the daily challenges in a daycare institution.
Local political leadership and bureaucracy could certainly do more to focus on corporation, standardisation and digital transformation, and they actually do, but political views differ and they change every 4 years, and the truth is that there just isn't any voter interest in IT unless it goes wrong.
We're trying to build national standards, we've had a set of architectural standards called Rammearkitekturen for a few yers now, but getting them implemented is slow. For one they're made by muniplicities and our structure of government is split in three. Muniplicities, Counties and the State and each branch has it's own ideas, leading to bureaucracy and political differences. Some want us to use EU standards, others want us to build our own, and even if we decided, there are different sets of EU standards as well as different sets of Danish standards.
I personally think the best we can do is try to use whatever national standards are in favour, and build smaller applications on them, with open API's, and run everything as SaaS in infrastructures such as AWS or Azure. I also think we should do a lot more work on business development, modifying business practices before we throw IT at something.
But it's complicated and it's on a giant scale where even minor changes take years to implement
Re: MongoDB gets support for multi-document ACID transactions
#229Earlier quoted context omitted.
That's one thing I keep "hearing". The default configuration now is a WriteConcern of 1 - meaning at least the primary has to have written it successfully. You can choose 2 or majority among others.
That won't protect you, you can read the analysis for MongoDB from here: http://jepsen.io/analyses The last analysis looks good, but you need to note that this is only true with the strongest settings (this means that Mongo will be configured with slowest settings) also this analysis did not include node crashes or restarts.
I read it. From his analysis it seems like the loss of data will rarely happen in the real world if you have a WriteConcern of Majority and you don't have a large latency between nodes.
One of the comments said that theoretically, that could also happen with any system if something happened between the time that it wrote to disk and the time it acknowledged it, you may have extra data.
There are times though that you care more about speed than reliability - ie capturing string data for an IOT device, logging etc.
There are even times that eventual consistentency is good enough. But definitely choose the right tool for the job. I wouldn't trust Mongo for financial data where transactions are a must.
And in my preferred language - C#, if you write your code correctly, you can switch out your Linq provider from Entity Framework to the Mongo driver for instance without any major code rewrite so you aren't stuck with your choice.
Re: MongoDB gets support for multi-document ACID transactions
#230Earlier quoted context omitted.
Just because something can be done, including the fashionable microservices "architecture", it does not mean that it should be done, or that it is a good idea.
Application architecture is inherently malleable, yes. But when properly implemented, micro service architecture is actually a great idea. I can’t help if your experience has not been the same as mine.
Certainly, one can architect anything, use NoSQL for critical data storage, or use Kafka for synchronizing two SQL databases with the same schema and living on the same netowrk(an actual proposal from a fan of microservices, btw). But whether it should be done is another question, and I have yet to see a single example of microservices being a good idea. Not "it works" -- obviously, you can get it to work, but being better than alternatives. Come to think of it, Mongo is the same...