Querying an RDF triplestore using SPARQL and Relational pipes

#1

Post by xkucf03 » Thu, Jul 30, 2020, 12:22 AM UTC

Querying an RDF triplestore using SPARQL and Relational pipes
relational-pipes.globalcode.info

Re: Querying an RDF triplestore using SPARQL and Relational pipes

#2

Post by f6v » Thu, Jul 30, 2020, 6:22 AM UTC

I was wondering if there’re use cases for this in the real world where relational database doesn’t cut it?

Re: Querying an RDF triplestore using SPARQL and Relational pipes

#3

Post by philjohn » Thu, Jul 30, 2020, 7:30 AM UTC

f6v wrote: post #2

I was wondering if there’re use cases for this in the real world where relational database doesn’t cut it?

I was tech lead for an RDF/Linked Data base Library Discovery Interface called Prism.

The dominant Libary Interchange Format is MARC which predates the relational database by a few years, and is a hierarchical document structure. It's possible, but not particularly helpful, to store it normalised in a relational database.

I gave a presentation almost a decade ago about our work. [1]

The takeaway is that Linked Data is just a more natural fit for modelling Library Data, especially when you use external sources like VIAF[2] to help anchor your identifier URI's.

Sadly, whilst RDF has seen some uptake in the library world (e.g. BIBFRAME[3]), the full potential was never tapped whilst I was working in the space. Quad Stores are very fun to tinker with though, and features such as inference and property paths in SPARQL 1.1[4] allow you to do some interesting things that are difficult, or non-idiomatic, in a relational system.

[1] - here: https://www.slideshare.net/philjohn/linked-library-data-in-t... [2] - http://viaf.org/ [3] - https://www.loc.gov/bibframe/ [4] - https://www.w3.org/TR/2013/REC-sparql11-query-20130321/#prop...

Re: Querying an RDF triplestore using SPARQL and Relational pipes

#4

Post by jerven » Thu, Jul 30, 2020, 7:38 AM UTC

f6v wrote: post #2

I was wondering if there’re use cases for this in the real world where relational database doesn’t cut it?

SPARQL is nicer than SQL in a few cases: * Federation is better than ETL+Datawarehousing * Data is to be integrated on demand by the user

A practical example of mine is the UniProt database in the life sciences and the European Patent Office SPARQL endpoints.

These two datasets have some intersection of data. Combining these two in a classical datawarehouse with ETL pipelines would cost a few million in start up costs (Full data fidelity, small team 1 year work, optimisitic). The same with non federated RDF/SPARQL is 3 days work. This with federated querying is 2 minutes work.

SQL has a richer ecosystem with many more people confident in it's usage. More deployment options. etc. Which is why often you will see tools like StarDog Virtual Graphs which will turn existing SQL DB's into SPARQL ones (via translating SPARQL->SQL) for in organization federated knowledge graphs. i.e. no to minimal ETL pipelines, direct querying on (standby, copies) of primary datasources.

In some domains the "business" analysts know SQL, even rarer but possible they know SPARQL. Letting them ask any query they can think of not bounded by what is one specific database can be extremely valuable. For organizations that can extract that value the lower market penetration of SPARQL is sad but not a real issue. This works in practice for SPARQL but not for SQL as what can be done with a user stated SERVICE clause in SPARQL requires a DBA to setup a foreign table connection in the SQL world.

Another example is when a few (N) hospitals need to exchange data. They have a few more relational databases (N+2) for this data in house for their patient groups. Upon project commencement they notice that all are SQL, but non are similar from vendor differences, but more problematic modelling issues. Transforming their data to RDF is as complicated as to standardize on one new schema. But the RDF gives immediate integration with SnoMedCT, ICD10 and LOINC which allows easy queries taking into account hierarchical knowledge in those medical standards. Those queries would be possible two write by hand in SQL but are easier in RDF/SPARQL when attaching a minimal OWL reasoner. Then integrating with GeoSpatial data is again easier because in this country that is provided in RDF as well.

NOTE: A full fidelity UniProt Schema (including all subdata sets e.g. UniParc) would be a 150-200 tables depending on some modelling choices. EPO I assume to be in the same order.

NOTE2: While federated querying is a standard SPARQL feature this can of course be limited/turned off depending on the security/legal context.

Re: Querying an RDF triplestore using SPARQL and Relational pipes

#5

Post by sawaruna » Thu, Jul 30, 2020, 8:18 AM UTC

f6v wrote: post #2

I was wondering if there’re use cases for this in the real world where relational database doesn’t cut it?

Most of my academic and current research career involves RDF and SPARQL and I wish it were a bit more used outside of academia or esoteric library / information science related fields. Currently taking a bunch of heterogenous data from different providers, most of which is in some kind of relational database, then transforming it into RDF and letting it be queryable using sparql. The creation of aggregated entities based on related resources from different providers, plus the possibility to do some advanced querying using sparql, is the main benefit here, though I don't know enough about relational databases to say whether or not it's just as doable there.

I'll say one disadvantage of RDF is the lack of any well implemented reification of triples.

Re: Querying an RDF triplestore using SPARQL and Relational pipes

#6

Post by matteuan » Thu, Jul 30, 2020, 8:34 AM UTC

For research, I created experimental RDF storage on top of Parquet and Apache Spark for querying big graphs[1]. It converts the RDF graph in a sort of property graph, where we have a row for each entity and where the columns are the all possible properties. The trick is to use a columnar format with the proper encoding (in our case Parquet), to solve the problem of having a lot of columns and a huge NULLs space. With this representation we can eliminate costly joins for most of the common queries, but also reduce the size of the necessary ones.

[1] PRoST https://github.com/tf-dbis-uni-freiburg/PRoST

Re: Querying an RDF triplestore using SPARQL and Relational pipes

#7

Post by ragebol » Thu, Jul 30, 2020, 9:01 AM UTC

f6v wrote: post #2

I was wondering if there’re use cases for this in the real world where relational database doesn’t cut it?

RDF is apparently heavily used in bio-informatics and life-sciences. I'm not well versed enough in any of the subjects involved whether a relation DB would not cur it though.

Being able to create cross links between data independent of a predefined schema seems useful for research though. With RDF, you can just define a predicate on your own and link two objects/subjects in completely different, independent triple stores, if I understand correctly.

Re: Querying an RDF triplestore using SPARQL and Relational pipes

#8

Post by melvinroest » Thu, Jul 30, 2020, 10:21 AM UTC

This is an awesome post! I'll be rereading this a couple of times ;-)

> Several ways of connecting to an RDF triplestore

There is another way to connect to an RDF triple store [1]. It's done by a company called Triply.

Triply made a product that is basically a GUI that allows you to host RDF data and use SPARQL to query it. You can also query other public datasets [2]. Currently, it's a B2B offering only, but there is a consumer version in the works. I wouldn't know when it comes out.

For now, you can try out the SPARQL querying feature on public datasets [2].

Disclaimer: I work there recently, this post is my own opinion, not of Triply.

[1] That uses Virtuoso or Jena under the hood.

[2] This example uses DBPedia: https://triplydb.com/wikimedia/dbpedia/sparql/dbpedia

Re: Querying an RDF triplestore using SPARQL and Relational pipes

#9

Post by abbe98 » Thu, Jul 30, 2020, 11:56 AM UTC

Wikidata has a great SPARQL editor with plenty of examples and built in visualizations:

https://query.wikidata.org/

Re: Querying an RDF triplestore using SPARQL and Relational pipes

#10

Post by etimberg » Thu, Jul 30, 2020, 12:17 PM UTC

f6v wrote: post #2

I was wondering if there’re use cases for this in the real world where relational database doesn’t cut it?

I work in the energy space. A relatively common format for information exchange is the Common Information Model ("CIM")[1] which is usually serialized as RDF XML documents.

In Europe, this format is used as the standard data model for transmission system operators (TSOs) to share network information needed to run the Europe wide electrical grid. ENTSO-E [2] publishes a set of RDFS profiles that more tightly scope the data model to their use cases. [3]

With that use case in mind, why would an RDF database be useful in this case: data size. It's quite easy to get into millions of objects in the data graph for a single distribution feeder. A large utility might have thousands of such feeders, plus associated sub-transmission & transmission infrastructure (the hierarchy is nicely shown in [4]).

This can be represented in a relational database, but a lot of the queries start to become recursive. If object A connects to object B which connects to object C, and you want to query everything connected to A, you don't know that C exists until you've found object B.

Refs:

1. https://en.wikipedia.org/wiki/Common_Information_Model_(elec...

2. https://en.wikipedia.org/wiki/European_Network_of_Transmissi...

3. https://cgmes.github.io/

4. https://en.wikipedia.org/wiki/Electric_power_distribution#/m...