Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

F1: A Distributed SQL Database That Scales

research.google.com

Post reply on HN
Print view View original

11–20 of 24 posts

Re: F1: A Distributed SQL Database That Scales

#11

Post by drunkpotato »

jacques_chester wrote: post #5

Impressive, as one might expect. Some reactions: -- Secret sauce A lot of the magic of F1 comes from Spanner, the distributed storage system. The name "F1" itself is an allusion to "inheriting" some of the properties of Spanner. -- Hierarchical tables What they call hierarchical tables, I would think best be viewed as one-to-many relationships. In guess they've privileged this model in their storage because that's wh…

How do hierarchical tables compare to Postgres' hstore?

I think recursive structures like lists and trees are one weakness of the relational model, and haven't found a fully satisfying relational answer to the limitations. It seems to me like the assumption of atomic column types is a major weakness inherent in the relational model itself when it comes to recursion. Any thoughts/comments?

Re: F1: A Distributed SQL Database That Scales

#12

Post by jacques_chester »

vosper wrote: post #9
jacques_chester wrote: post #5

Impressive, as one might expect. Some reactions: -- Secret sauce A lot of the magic of F1 comes from Spanner, the distributed storage system. The name "F1" itself is an allusion to "inheriting" some of the properties of Spanner. -- Hierarchical tables What they call hierarchical tables, I would think best be viewed as one-to-many relationships. In guess they've privileged this model in their storage because that's wh…

Your point about change history and its poor support in SQL is spot-on; at my employer we've gone through (1) and (2) and are trying to figure out (3) at the moment. From what I understand of Datomic temporal support is baked in, but it's about the only database I can think of that does that, and I haven't heard stories about it being used in production for large volumes of data.

Snodgrass wrote an entire book about dealing with the temporal blindness of SQL which might help you:

http://www.cs.arizona.edu/people/rts/tdbbook.pdf

More generally, you can usually build a model that handles (2), the changing state, by generalising your original model. I've done in both of classical ways, having either validity period fields (thus pushing complexity into every query), or by having audit tables (thus relying on triggers).

But building a changing model is hard, because at the schema level, SQL only supports (1). That is, it provides primitives to change the current state of the model.

So you wind up having to build a meta-model. The deficiencies of SQL (and I'm an RDBMS bigot) mean that we wind up building inner platforms. Or outer platforms -- witness migration toolkits.

Yet support for changing models is essential. No model is ever constant. Most of my work is for a public-sector client whose legal reporting requirements are constantly changing. You cannot throw away old reports which were made under old regulations; you must be able to recreate the world at any point in time. Which requires either a lot of bookkeeping code or taking periodic snapshots.

Datomic looks neat. I see that I'm not the only one who made the leap from transactional memory to transactional models.

Edit: I just found notes I wrote in 2009 on database languages while looking for something else. Spooky: http://chester.id.au/2013/08/28/notes-towards-a-set-objectiv...

Re: F1: A Distributed SQL Database That Scales

#13

Post by jacques_chester »

drunkpotato wrote: post #11
jacques_chester wrote: post #5

Impressive, as one might expect. Some reactions: -- Secret sauce A lot of the magic of F1 comes from Spanner, the distributed storage system. The name "F1" itself is an allusion to "inheriting" some of the properties of Spanner. -- Hierarchical tables What they call hierarchical tables, I would think best be viewed as one-to-many relationships. In guess they've privileged this model in their storage because that's wh…

How do hierarchical tables compare to Postgres' hstore? I think recursive structures like lists and trees are one weakness of the relational model, and haven't found a fully satisfying relational answer to the limitations. It seems to me like the assumption of atomic column types is a major weakness inherent in the relational model itself when it comes to recursion. Any thoughts/comments?

> How do hierarchical tables compare to Postgres' hstore?

If I am reading them correctly, it's a storage strategy, not a specific "feature" per se. The closest analogy to hstore is that they provide native support for storing and querying protobuf blobs.

> I think recursive structures like lists and trees are one weakness of the relational model, and haven't found a fully satisfying relational answer to the limitations.

It depends on why you're using trees or graphs.

If it's inherent in the data, then modern SQL has recursive queries that make it much easier than the old methods.

If it's inherent in the model, you will find it harder. You might need to pick a non-standard approach, such as PostgreSQL's inherited tables. I'd think long and hard before saying it's inherent in the model, by the way. Strictly speaking you can represent the same thing as sets of relations or as a graph; it's better to utilise the strengths of the tool in front of you.

Re: F1: A Distributed SQL Database That Scales

#14

Post by zapov »

drunkpotato wrote: post #11
jacques_chester wrote: post #5

Impressive, as one might expect. Some reactions: -- Secret sauce A lot of the magic of F1 comes from Spanner, the distributed storage system. The name "F1" itself is an allusion to "inheriting" some of the properties of Spanner. -- Hierarchical tables What they call hierarchical tables, I would think best be viewed as one-to-many relationships. In guess they've privileged this model in their storage because that's wh…

How do hierarchical tables compare to Postgres' hstore? I think recursive structures like lists and trees are one weakness of the relational model, and haven't found a fully satisfying relational answer to the limitations. It seems to me like the assumption of atomic column types is a major weakness inherent in the relational model itself when it comes to recursion. Any thoughts/comments?

Both Oracle and Postgres have objects and collections. This allows you to build hierarchical structures. Postgres unfortunately doesn't allow for recursive structures ;(

If you are interested in how to do it take a look here: https://blog.dsl-platform.com/postgres-bridge-between-worlds...

Re: F1: A Distributed SQL Database That Scales

#15

Post by jacques_chester »

zapov wrote: post #14
drunkpotato wrote: post #11

Earlier quoted context omitted.

How do hierarchical tables compare to Postgres' hstore? I think recursive structures like lists and trees are one weakness of the relational model, and haven't found a fully satisfying relational answer to the limitations. It seems to me like the assumption of atomic column types is a major weakness inherent in the relational model itself when it comes to recursion. Any thoughts/comments?

Both Oracle and Postgres have objects and collections. This allows you to build hierarchical structures. Postgres unfortunately doesn't allow for recursive structures ;( If you are interested in how to do it take a look here: https://blog.dsl-platform.com/postgres-bridge-between-worlds...

It should be noted that Oracle's object types are a feature of PL/SQL, their imperative programming language.

PostgreSQL's concept of object-relational databases is baked into the SQL. Into the logical model of the data itself.

Put another way: Oracle doesn't have table inheritance.

Re: F1: A Distributed SQL Database That Scales

#16

Post by zapov »

jacques_chester wrote: post #15
zapov wrote: post #14

Earlier quoted context omitted.

Both Oracle and Postgres have objects and collections. This allows you to build hierarchical structures. Postgres unfortunately doesn't allow for recursive structures ;( If you are interested in how to do it take a look here: https://blog.dsl-platform.com/postgres-bridge-between-worlds...

It should be noted that Oracle's object types are a feature of PL/SQL, their imperative programming language. PostgreSQL's concept of object-relational databases is baked into the SQL. Into the logical model of the data itself. Put another way: Oracle doesn't have table inheritance.

We are using Oracle object types in plain SQL, so no, they are not PL/SQL exclusive. But yeah, you can do a lot less with them outside of PL/SQL.

Re: F1: A Distributed SQL Database That Scales

#17

Post by jacques_chester »

zapov wrote: post #16
jacques_chester wrote: post #15

Earlier quoted context omitted.

It should be noted that Oracle's object types are a feature of PL/SQL, their imperative programming language. PostgreSQL's concept of object-relational databases is baked into the SQL. Into the logical model of the data itself. Put another way: Oracle doesn't have table inheritance.

We are using Oracle object types in plain SQL, so no, they are not PL/SQL exclusive. But yeah, you can do a lot less with them outside of PL/SQL.

Right, you're right, I overlooked CREATE TABLE OF.

Re: F1: A Distributed SQL Database That Scales

#18

Post by lucian1900 »

vosper wrote: post #9
jacques_chester wrote: post #5

Impressive, as one might expect. Some reactions: -- Secret sauce A lot of the magic of F1 comes from Spanner, the distributed storage system. The name "F1" itself is an allusion to "inheriting" some of the properties of Spanner. -- Hierarchical tables What they call hierarchical tables, I would think best be viewed as one-to-many relationships. In guess they've privileged this model in their storage because that's wh…

Your point about change history and its poor support in SQL is spot-on; at my employer we've gone through (1) and (2) and are trying to figure out (3) at the moment. From what I understand of Datomic temporal support is baked in, but it's about the only database I can think of that does that, and I haven't heard stories about it being used in production for large volumes of data.

Also, Datomic is not open source :(

Re: F1: A Distributed SQL Database That Scales

#19

Post by capkutay »

Can someone enlighten me as to why google doesn't want to make an enterprise play? Many companies tout that they are offering data infrastructures similar to those used at google..why wouldn't google commercialize that technology themselves? It should be pretty obvious as to how that could be of huge financial benefit to the company.

Re: F1: A Distributed SQL Database That Scales

#20

Post by dragonwriter »

capkutay wrote: post #19

Can someone enlighten me as to why google doesn't want to make an enterprise play? Many companies tout that they are offering data infrastructures similar to those used at google..why wouldn't google commercialize that technology themselves? It should be pretty obvious as to how that could be of huge financial benefit to the company.

> Can someone enlighten me as to why google doesn't want to make an enterprise play?

They do. They have a range of commercial, including enterprise, offerings (many of which are based on originally-internal technologies, including their MySQL-based distributed database that preceded F1.)

> why wouldn't google commercialize that technology themselves?

They probably will, just as they have commercialized their previous internal storage technologies.

Post reply on HN