One of the biggest challenges in databases is handling concurrency and sharding, wish this would have talked a bit more about how that changes between a graph database and a relational database.
Graph Databases 101
101–107 of 107 posts
Re: Graph Databases 101
#102Earlier quoted context omitted.
Had the state of art changed from the time of their last attempt to move to relational DB? Had the hardware changed during that time? You tell us about some old project, written in hard to maintain languages, which had many failures to adapt to new tech. This is exactly what to expect. I am talking about relatively modern language (C#) using good DB tech (lagging about seven, maybe five years from the state of art).…
The last attempt was made in around 2008, AFAIR. Fundamentally nothing changed in the relational storage. Follow-a-graph-edge operation is as expensive as it used to be (involves an index lookup, it cannot be cheap). If you know a relational arrangement suitable for a cheap O(1) edge traversal - please share. But I am very skeptical. And I cannot see how the host language is relevant at all. C#, Haskell, whatever - n…
Use covering indices for edges, it is vastly cheaper than regular indices. Choose write-optimized storage layer (fractal indices (circa ~2007, BTW!), LSM) for DB. Don't do singular accesses, use bulk accesses (and now we hit the wall of PL/1 and Fortran you mentioned). I can go on.
I cannot help but feel that what you describe is a classic example of technical inertia due to massive technical debt. It cannot prove that relational DBs are bad for CADs.
Re: Graph Databases 101
#103Earlier quoted context omitted.
The last attempt was made in around 2008, AFAIR. Fundamentally nothing changed in the relational storage. Follow-a-graph-edge operation is as expensive as it used to be (involves an index lookup, it cannot be cheap). If you know a relational arrangement suitable for a cheap O(1) edge traversal - please share. But I am very skeptical. And I cannot see how the host language is relevant at all. C#, Haskell, whatever - n…
Use covering indices for edges, it is vastly cheaper than regular indices. Choose write-optimized storage layer (fractal indices (circa ~2007, BTW!), LSM) for DB. Don't do singular accesses, use bulk accesses (and now we hit the wall of PL/1 and Fortran you mentioned). I can go on. I cannot help but feel that what you describe is a classic example of technical inertia due to massive technical debt. It cannot prove th…
Is any of the indices you're describing O(1)? Very, very unlikely.
As for the bulk operations, they're in most cases totally useless. Rendering - maybe, but most of the other CAD operations require precise edge following.
And why even going into all the troubles with using this totally unsuitable relational representation when a proper graph dbms is so much easier? Relational religion is so funny, almost as funny as OOP.
Re: Graph Databases 101
#104Anybody know dgraph.io?
it's a Scalable, Distributed, Low Latency, High Throughput Graph Database over terabytes of structured data.
DGraph supports facebook GraphQL as query language, and responds in JSON and the storage engine is facebook rocksdb a very fast database.
see more in https://github.com/dgraph-io/dgraph
Re: Graph Databases 101
#105Earlier quoted context omitted.
Use covering indices for edges, it is vastly cheaper than regular indices. Choose write-optimized storage layer (fractal indices (circa ~2007, BTW!), LSM) for DB. Don't do singular accesses, use bulk accesses (and now we hit the wall of PL/1 and Fortran you mentioned). I can go on. I cannot help but feel that what you describe is a classic example of technical inertia due to massive technical debt. It cannot prove th…
Is any of the indices you're describing O(1)? Very, very unlikely. As for the bulk operations, they're in most cases totally useless. Rendering - maybe, but most of the other CAD operations require precise edge following. And why even going into all the troubles with using this totally unsuitable relational representation when a proper graph dbms is so much easier? Relational religion is so funny, almost as funny as…
What exactly are these "most CAD operations"? I bet they follow many links from nodes (note the plural!) in most of them. Single node operations are seldom and even then there are many links to follow. This is true for scheme/PCB editor, I bet it is even more true for arch design.
You can look at these indices as O(1) operations. Btrees are just like that.
(if you think that memory access is O(1), you are wrong)
Graph DBs, more often than not, are ad hoc bug ridden slow poor implementation of one tenth of relatively complete implementation of relational DB.
Re: Graph Databases 101
#106Earlier quoted context omitted.
Is any of the indices you're describing O(1)? Very, very unlikely. As for the bulk operations, they're in most cases totally useless. Rendering - maybe, but most of the other CAD operations require precise edge following. And why even going into all the troubles with using this totally unsuitable relational representation when a proper graph dbms is so much easier? Relational religion is so funny, almost as funny as…
What exactly are these "most CAD operations"? I bet they follow many links from nodes (note the plural!) in most of them. Single node operations are seldom and even then there are many links to follow. This is true for scheme/PCB editor, I bet it is even more true for arch design. You can look at these indices as O(1) operations. Btrees are just like that. (if you think that memory access is O(1), you are wrong) Grap…
> What exactly are these "most CAD operations"?
CSG, place and route (pipes, cables, etc.), design constraint checks, all that stuff.
> I bet they follow many links from nodes (note the plural!) in most of them.
Not that many, mostly single-digit numbers.
> This is true for scheme/PCB editor
Which is very, very different from an oil refinery or an aircraft carrier. Both in a scale and typical operations.
> You can look at these indices as O(1) operations. Btrees are just like that.
WAT?!? Not even close. O(log n) at best. And a multiplier there is huge.
> Graph DBs, more often than not, are ad hoc bug ridden slow poor implementation of one tenth of relatively complete implementation of relational DB.
What?
Graph DBs are orders of magnitude simpler than any relational pile of a mess. It's really hard to screw them up. Everything is trivial there, including transactions, logging, referential transparency and all that.
Re: Graph Databases 101
#107Earlier quoted context omitted.
What exactly are these "most CAD operations"? I bet they follow many links from nodes (note the plural!) in most of them. Single node operations are seldom and even then there are many links to follow. This is true for scheme/PCB editor, I bet it is even more true for arch design. You can look at these indices as O(1) operations. Btrees are just like that. (if you think that memory access is O(1), you are wrong) Grap…
> What exactly are these "most CAD operations"? CSG, place and route (pipes, cables, etc.), design constraint checks, all that stuff. > I bet they follow many links from nodes (note the plural!) in most of them. Not that many, mostly single-digit numbers. > This is true for scheme/PCB editor Which is very, very different from an oil refinery or an aircraft carrier. Both in a scale and typical operations. > You can lo…
Design constrain checks touch A LOT of stuff: usually things have several classes attached to them and check need to be performed on union or intersection of class-related data. Place and route, even for 2D PCB, also touches a lot of stuff - it needs to access constraints, at the very least. In our case, constraint system had to return data to PCB real-time router in under 100us, for several tens of millions of different classes of constraints. I cannot see how you can speak of singular accesses in this context. Or "single-digit as many" accesses.
Now I'll leave conversation. We clearly have different view on almost everything, including, but not limited to "huge multipliers".