11–20 of 108 posts
> ... RDF ... SPARQL ... OWL ... I want to believe. (really! I think that's hugely underestimated tech).
These tools were pretty cool and an enormous amount of work was put into them.
The ontologies were extremely extensible. There just wasn't enough of an ecosystem putting them into practice and demonstrating their utility.
Their examples are nice:
https://github.com/Netflix-Skunkworks/uda/blob/9627a97fcd972...
Why would Netflix engineering host on Medium? Very odd. And you just lose readers to the popups but you don't benefit from their discovery much either.
Sure you do.
And the types of engineers writing on Medium are the ones they want to recruit, too.
The business contract with a consolidated data definition is that everyone in the business, no matter which domain, can rely on it. But think about the red tape that introduces. Whenever you need to define or update a data definition, now you don't have to think just about your own use case, but about all of the potential use cases throughout the organization, and you likely need to get sign-off from a wide variety of stakeholders, because any change, however small, is by definition an org-wide change.
It's the data form of the classic big-org problem, "Why does it take two months to change the color of a button?"
Granted, in most cases, having data definitions duplicated, with the potential for drift, is going to be the more insidious problem. But sometimes you just want to get a small, isolated change out the door without having to go through several levels of cross-domain approval committees.
For all the benefits, there is a large problem with this approach that often goes unacknowledged. It is fundamentally a business problem, rather than a technical problem, but it has impact on development speed, so it's secondarily a technical problem. The business contract with a consolidated data definition is that everyone in the business, no matter which domain, can rely on it. But think about the red tape that in…
If your data/service/api is used by a lot of other people in the org, you have to work with them to make sure your change doesn't break them. That's true regardless of the architecture.
From a modeling perspective, there is certainly inherent complexity in representing data from domain models in different ways. One can argue though that this is a feature and not a big. Not the same level of nuance and complexity is needed in all of the use-cases. And representational models usually are optimized for particular read scenarios, this seems to mandate argue against that, favoring uniformity over contextual handling of information. It will most likely scale better in places where the level of understanding needed from the domain model is quite uniform, though I have seen most often that use-cases are often complicated when they do not simplify concepts that in their code domain model is very complex and nuanced.
For all the benefits, there is a large problem with this approach that often goes unacknowledged. It is fundamentally a business problem, rather than a technical problem, but it has impact on development speed, so it's secondarily a technical problem. The business contract with a consolidated data definition is that everyone in the business, no matter which domain, can rely on it. But think about the red tape that in…
For all the benefits, there is a large problem with this approach that often goes unacknowledged. It is fundamentally a business problem, rather than a technical problem, but it has impact on development speed, so it's secondarily a technical problem. The business contract with a consolidated data definition is that everyone in the business, no matter which domain, can rely on it. But think about the red tape that in…
For all the benefits, there is a large problem with this approach that often goes unacknowledged. It is fundamentally a business problem, rather than a technical problem, but it has impact on development speed, so it's secondarily a technical problem. The business contract with a consolidated data definition is that everyone in the business, no matter which domain, can rely on it. But think about the red tape that in…
At a place like Netflix where the product has been fundamentally the same for almost a decade, installing this kind of red tape is great for job security
It really doesn't, and that's not the point. This is for business entities that are larger than teams.
It's way worse to have a million different schemas with no way to share information. And then you have people everywhere banging on your door asking for your representation, you have to help them, you have to update it in their systems. God forbid you've got to migrate things...
If your entity type happens to be one that is core to the business, it's almost a neverending struggle. And when you find different teams took your definition and twisted it, when you're supposed to be the source of truth, and teams downstream of them consume it in the bastardized way...
This project sounds like a dream. I hope it goes well for Netflix and that they can evangelize it more.
High-level overview:
- https://www.w3.org/DesignIssues/LinkedData.html from TimBL
- https://www.w3.org/DesignIssues/ReadWriteLinkedData.html from TimBL
- https://www.w3.org/DesignIssues/Footprints.html from TimBL
Similar recent attempts:
- https://www.uber.com/en-SE/blog/dragon-schema-integration-at... an attempt in the similar direction at Uber
- https://www.slideshare.net/joshsh/transpilers-gone-wild-intr... continuation of the Uber Dragon effort at LinkedIn
- https://www.palantir.com/docs/foundry/ontology/overview/
Standards and specs in support of such architectures:
- http://www.lotico.com/index.php/Next_Generation_RDF_and_SPAR... (RDF is the only standard in the world for graph data that is widely used; combining graph API responses from N endpoints is a straightforward graph union vs N-way graph merge for JSON/XML/other tree-based formats). Also see https://w3id.org/jelly/jelly-jvm/ if you are looking for a binary RDF serialization.
- https://www.w3.org/TR/shacl/ (needs tooling, see above)
- https://www.odata.org/ (in theory has means to reuse definitions, does not seem to work in practice)
- https://www.w3.org/TR/ldp/ (great foundation, too few features - some specs like paging never reached Recommendation status)
- https://open-services.net/ (builds atop W3C LDP; full disclosure: I'm involved in this one)
- https://www.w3.org/ns/hydra/ (focus on describing arbitrary affordances; not related to LinkedIn Hydra in any way)
Upper models:
- https://basic-formal-ontology.org/ - the gold standard. See https://www.youtube.com/watch?v=GWkk5AfRCpM for the tutorial
- https://www.iso.org/standard/87560.html - Industrial Data Ontology. There is a lot of activity around this one, but I lean towards BFO. See https://rds.posccaesar.org/WD_IDO.pdf for the unpaywalled draft and https://www.youtube.com/watch?v=uyjnJLGa4zI&list=PLr0AcmG4Ol... for the videos