How To Make An Infinitely Scalable RDBMS • Hacker News Board

Re: How To Make An Infinitely Scalable RDBMS

#71

Post by leif » Tue, Nov 26, 2013, 4:28 AM UTC

VladRussian2 wrote: post #69

leif wrote: post #67

Earlier quoted context omitted.

I think you're talking about an insert buffer, not a transaction log, and in that case, no matter how big your insert buffer is, it will eventually saturate and you'll end up hitting the performance cliff of the B-tree. You really need better data structures (like fractal trees or LSM trees) to get past it.

no, i'm talking about transaction log ("redo log" in Oracle parlance). Switching log files causes checkpoint (ie. flush - it is when the index datablocks changed by the inserts you mention will finally hit the disk ) http://asktom.oracle.com/pls/asktom/f?p=100:11:0::::P11_QUES... MS and DB2 have similar behavior.

Oh ok, now I see what you're saying, it's still similar to an insert buffer in that case. B-tree behavior is still to blame for this, and if you make your log file bigger it lets you soak up more writes before you need to checkpoint but you'll either have even longer checkpoints eventually, or you'll run out of memory before you have to checkpoint.

We also checkpoint before trimming the log, but our checkpoints are a lot smaller because of write optimization.

Re: How To Make An Infinitely Scalable RDBMS

#72

Post by leif » Tue, Nov 26, 2013, 4:32 AM UTC

mtravis wrote: post #47

jbellis wrote: post #37

Earlier quoted context omitted.

http://www.infinisql.org/docs/overview/#idp37097184 gave me that impression.

Oh, I thought you said 2PC (two-phased commit). Yes, InfiniSQL uses two phase locking. The Achilles' Heel is deadlock management. By necessity, the deadlock management will need to be single-threaded (at least as far as I can figure). No arguments there, so deadlock-prone usage patterns may definitely be problematic. I don't think there's a perfect concurrency management protocol. MVCC is limited by transactionid gen…

HAT (http://www.bailis.org/papers/hat-vldb2014.pdf) looks pretty promising in terms of multi-term transactions. It turns out that you can push the problem off to garbage collection in order to make transaction id generation easy, and garbage collection is easier to be sloppy and heuristic about. The only problem is it isn't clear yet that HATs are as rich as 2PL-based approaches, and that nobody's built an industrial strength implementation yet.

Re: How To Make An Infinitely Scalable RDBMS

#73

Post by army » Tue, Nov 26, 2013, 4:36 AM UTC

mbesto wrote: post #34

glogla wrote: post #15

There was very interesting presentation by one professor. I'm not sure about what university, but he seemed to know his work. He talked about how databse world is about to change. ACID is really expensive in terms of resources, and so are the more difficult things about relational schema (foreign keys, checks, etc). And architecture of classic RDBMSes is pretty wasteful -- they use on-disk format but cache it in memo…

> But this seems to be first actual Main memory database. Eh, not really... This is exactly what SAP has been doing for several years via Hasso Plattner and the Potsdam Institute: https://epic.hpi.uni-potsdam.de/Home/HassoPlattner If you've ever worked with large scale "enterprise" database warehouses, they tend to be slow and clunky. Back in 2006ish SAP took the whole Data Warehouse (well mainly just the data cubes)…

InnoDB isn't anything like that - it's a transactional database engine that's been around since the 90's and has since become the standard storage engine for MySQL - it competes directly with Postgres' storage layer.

Re: How To Make An Infinitely Scalable RDBMS

#74

Post by army » Tue, Nov 26, 2013, 4:39 AM UTC

nl wrote: post #33

JeffDClark wrote: post #19

Earlier quoted context omitted.

Is this the talk that you are referring to? http://slideshot.epfl.ch/play/suri_stonebraker

Note that Stonebraker makes some good points, but there are many ways to build scalability and Stonebraker is too fast to dismiss many. In particular, his criticism of traditional databases seems based more on philosophy rather than evidence. I'd advise reading both sides of the story: http://lemire.me/blog/archives/2009/09/16/relational-databas... http://lemire.me/blog/archives/2009/07/03/column-stores-and-... http:…

Stonebraker is a very smart person but he's also not shy about promoting his own companies/research. You generally get a well-informed but very opinionated take on things from him.

VoltDB, for example, is good for certain complex workloads over large but not-too-large data sets. For a lot of situations it isn't really an alternative to memcache+MySQL or a NoSQL solution.

Re: How To Make An Infinitely Scalable RDBMS

#75

Post by VladRussian2 » Tue, Nov 26, 2013, 4:44 AM UTC

leif wrote: post #71

VladRussian2 wrote: post #69

Earlier quoted context omitted.

no, i'm talking about transaction log ("redo log" in Oracle parlance). Switching log files causes checkpoint (ie. flush - it is when the index datablocks changed by the inserts you mention will finally hit the disk ) http://asktom.oracle.com/pls/asktom/f?p=100:11:0::::P11_QUES... MS and DB2 have similar behavior.

Oh ok, now I see what you're saying, it's still similar to an insert buffer in that case. B-tree behavior is still to blame for this, and if you make your log file bigger it lets you soak up more writes before you need to checkpoint but you'll either have even longer checkpoints eventually, or you'll run out of memory before you have to checkpoint. We also checkpoint before trimming the log, but our checkpoints are a…

>even longer checkpoints

yes, that is the point as big flush instead of many small ones would take either the same or, usually, less time than cumulative time of small flushes because of IO ordering and probability of some writes hitting the same data block.

Re: How To Make An Infinitely Scalable RDBMS

#76

Post by eksith » Tue, Nov 26, 2013, 5:03 AM UTC

mtravis wrote: post #8

alanctgardner2 wrote: post #6

I'm a little skeptical: - a bunch of the novel components (the UPS aware persistence layer, for example) aren't actually built yet - they're pushing for people to build businesses on it already. I would characterize it as "bleeding-edge with bits of glass glued on", so this doesn't seem entirely honest. - there's mostly a lot of breathless talk about how great and fast and scalable it is, but no mention of CAP theore…

Hi, Alan. Yes, many things are not built yet. Nowhere am I pushing anybody to build their business on it yet, but I am looking for hackers and early adopters/alpha testers. I've gone through pains on every doc that I've created that this is early, alpha, needs lots of work--including the 2nd paragraph of the linked-to article: "InfiniSQL is still in early stages of development--it already has many capabilities, but m…

>Nowhere am I pushing anybody to build their business on it yet...

"In fact, InfiniSQL ought to be the database that companies start with--to avoid migration costs down the road."

This is far from looking for hackers and early adopters. I understand that you're enthusiastic about something you've created, but let's be reasonable for a moment. I'm more than happy to try this out, but starting my business on something that isn't proven yet and has a fair number of durability features yet to be implemented is a no, no.

Edit: My posts on this thread may be coming off as negative and that's not what I intended. I'm cautious of new technology that purports to deliver the world to me on a silver platter. That said, I'd be happy to throw everything I've got at it to see what shakes loose so you can get this to production quality sooner.

Re: How To Make An Infinitely Scalable RDBMS

#77

Post by mtravis » Tue, Nov 26, 2013, 5:29 AM UTC

leif wrote: post #66

mtravis wrote: post #62

Earlier quoted context omitted.

Hi, Leif. It's not hard to get to the throughput limits of a single log device, even on a fast array. I've done it on Sybase, WebSphere MQ, Oracle, MySQL, basically on enough platforms that I assume it to be the general case. The log writes don't saturate the array itself--but the log file has a limit to how many blocks can be appended--even on fast arrays. But imagine getting rid of the transaction log entirely--the…

I'm not talking about the speed of the array, rather a battery backed controller. With one of those, as long as you're under the sequential write bandwidth of the underlying array, it pretty much doesn't matter how much you fsync the log, so that bottleneck (commit frequency) goes away. If you're planning to write data faster than disk bandwidth, then you have no hope of being durable and we're talking about problems…

Well, array=battery backed controller with a bunch of disks hanging off of it. Actually, there is latency associated with every sync. What I've seen on HDS arrays with Linux boxes and 4GB fibre channell adapters is about 200us per 4KB block write. That is very very good for disk. It's also slower than memory access by many orders of magnitude. This was about 3 years ago. Things are bound to be faster by now, but still not as fast as RAM.

I don't think it's unreasonable to want to write faster than an I/O subsystem can handle. Maybe it's not for every workload, but that doesn't mean it's for no workload.

The distinction I wasn't being clear about was that the storage array (the thing with the RAID controller, cache and disks hanging off of it) is not being saturated if a single transaction log file is being continuously appended to. But that headroom in the array does not translate to more throughput for the transaction log. I don't know if it's an important distinction.

Re: How To Make An Infinitely Scalable RDBMS

#78

Post by hatchoo » Tue, Nov 26, 2013, 6:21 AM UTC

glogla wrote: post #15

There was very interesting presentation by one professor. I'm not sure about what university, but he seemed to know his work. He talked about how databse world is about to change. ACID is really expensive in terms of resources, and so are the more difficult things about relational schema (foreign keys, checks, etc). And architecture of classic RDBMSes is pretty wasteful -- they use on-disk format but cache it in memo…

And just to add to the list of c-store databases, there's also Sybase IQ. (I believe Sybase is now owned by SAP so it may have been rebranded)

Re: How To Make An Infinitely Scalable RDBMS

#79

Post by cmccabe » Tue, Nov 26, 2013, 7:07 AM UTC

mtravis wrote: post #28

alanctgardner2 wrote: post #12

Earlier quoted context omitted.

CAP theorem can apply to any clustered system, it doesn't have to be multi-site. What happens if 6 of your 12 machines die? What if they get cut off from the other 6? edit: There's a bit of discussion further down about the SQL implementation. That's something I was very curious about as well. The projects linked below spend a lot of time working on supporting full ANSI SQL, and reducing latency by pushing down as ma…

Hi, Alan. Regarding CAP, I think that given redundant cluster interconnects, redundant managed power, odd # of cluster managers for quorum, all mean that split brain is just about out of the question, configured properly. The main reason that I have a FAQ about Hadoop is that I have been asked repeatedly by people, "what's the difference between InfiniSQL & Hadoop?" It seems to be the data project most on a lot of pe…

I work at Cloudera (although not on Impala). I work on HDFS.

Impala was about creating a low-latency SQL engine for Hadoop, so that queries could be done interactively by a human at a keyboard. This is something that you don't really get with Hive (despite all the recent hype) because it simply is too slow, and has high startup costs. That's unlikely to change in the future because of the overhead of spinning up JVMs, starting MapReduce jobs, etc.

It seems like you are trying to target the OLTP market. That's a difficult market to crack. A lot of the value of things like Oracle and Microsoft SQL server is not in the database itself, but in the surrounding software. Performance is nice, but unless you can get orders of magnitude, it's very difficult to compete.

Will anybody ever bridge OLTP and OLAP? The last people who claimed to be trying to do that were Drawn to Scale, and we all know how that turned out. I think it's better to focus on doing one thing well.

Good luck.

Re: How To Make An Infinitely Scalable RDBMS

#80

Post by DanWaterworth » Tue, Nov 26, 2013, 7:35 AM UTC

mtravis wrote: post #28

alanctgardner2 wrote: post #12

Earlier quoted context omitted.

CAP theorem can apply to any clustered system, it doesn't have to be multi-site. What happens if 6 of your 12 machines die? What if they get cut off from the other 6? edit: There's a bit of discussion further down about the SQL implementation. That's something I was very curious about as well. The projects linked below spend a lot of time working on supporting full ANSI SQL, and reducing latency by pushing down as ma…

Hi, Alan. Regarding CAP, I think that given redundant cluster interconnects, redundant managed power, odd # of cluster managers for quorum, all mean that split brain is just about out of the question, configured properly. The main reason that I have a FAQ about Hadoop is that I have been asked repeatedly by people, "what's the difference between InfiniSQL & Hadoop?" It seems to be the data project most on a lot of pe…

Congratulations, you have convinced me never to use your database. You can't ignore CAP, because machine failures, momentary high latency and bona fide network partitions happen.