Live data from Hacker News

Model Once, Represent Everywhere: UDA (Unified Data Architecture) at Netflix

netflixtechblog.com

21–30 of 108 posts

Re: Model Once, Represent Everywhere: UDA (Unified Data Architecture) at Netflix

#21
I wonder how they deal with versioning or breaking changes to the model. One advantage of keeping things more segregated is that when you decide to change a model you can do it in much smaller pieces.

I guess in their world they’d add a new model for whatever they want to change and then phase out use of the old one before removing it.

Re: Model Once, Represent Everywhere: UDA (Unified Data Architecture) at Netflix

#22
How does this relate to domain-driven design? It seems to be at odds with it, because in DDD it's kind of expected that the same concept will be represented in a different way by each system? But to be honest, I didn't read the whole blog post because of the UML vibes.

Re: Model Once, Represent Everywhere: UDA (Unified Data Architecture) at Netflix

#23
post #13

For all the benefits, there is a large problem with this approach that often goes unacknowledged. It is fundamentally a business problem, rather than a technical problem, but it has impact on development speed, so it's secondarily a technical problem. The business contract with a consolidated data definition is that everyone in the business, no matter which domain, can rely on it. But think about the red tape that in…

You could store the info as a common definition and then just use transformations on retrieval or storing if there's an exception for that system/business group.

Re: Model Once, Represent Everywhere: UDA (Unified Data Architecture) at Netflix

#24
post #11
post #5

> ... RDF ... SPARQL ... OWL ... I want to believe. (really! I think that's hugely underestimated tech).

It's 2005 again! These tools were pretty cool and an enormous amount of work was put into them. The ontologies were extremely extensible. There just wasn't enough of an ecosystem putting them into practice and demonstrating their utility. Their examples are nice: https://github.com/Netflix-Skunkworks/uda/blob/9627a97fcd972...

That's Turtle, it's an awesome RDF serialization. https://www.w3.org/TR/turtle/

Imagine trying to convey this example in RDF/XML (that's more like 2005).

RDFa and microdata stuff for sharing got pretty far, but those are often simpler vocabularies (at least when seen from the outside, maybe folks who index that shit has something nicer going on, idk).

Honestly, I feel kind of relieved seen Netflix using this stuff. I suggested using this kind of tech to model knowledge in systems that had this problem of knowledge representation several times, but always had a hard time when people said "if it's so good, why no big player uses it?".

Re: Model Once, Represent Everywhere: UDA (Unified Data Architecture) at Netflix

#26
post #13

For all the benefits, there is a large problem with this approach that often goes unacknowledged. It is fundamentally a business problem, rather than a technical problem, but it has impact on development speed, so it's secondarily a technical problem. The business contract with a consolidated data definition is that everyone in the business, no matter which domain, can rely on it. But think about the red tape that in…

I tried, for some time, to develop a product designed to solve this. It would have made it easier to specialize models locally while complying with the corporate one. (Basically, beefing up the data definition language to something like prolog, and putting real thought into making the corporate model reality-based rather than just what suits your current requirements.)

Unfortunately it came about at exactly the same time as NoSQL and Big Data, which are basically the opposite. They let you be really loose with your model, and if some data gets lost or misunderstood, hey, no biggie. It's easier to patch it later than to develop a strong model to start with.

But am I bitter about it? No, why do you ask? Twitch, twitch.

Re: Model Once, Represent Everywhere: UDA (Unified Data Architecture) at Netflix

#27

I really believe a common vocabulary makes sense. But it is hard, very hard as you spread across organization (some to be bought and integrated), business processes and time. As soon as it comes to generating stuff things become hard. One may be able to generate interfaces between two systems but which enterprise has only two layers? Yes, if all knowledge is captured in the central catalog we may be able to do it but…

> Attempts to do this and survived either restricted themselves to being very abstract or limited their scope to specific use cases.

Wikidata? 1.65 billion graph nodes and counting under a common vocabulary.

Re: Model Once, Represent Everywhere: UDA (Unified Data Architecture) at Netflix

#28
No, unfortunately the activity is not modeling at all. It's software development. Pretending otherwise will not make our thinking (or data structures) more logically consistent.

I feel the dream. But we went to that place 25 years ago, and we saw that it was stupid.

Tell you what, I'll do a raffle. Leave a comment telling me that I just don't get it. One lucky winner will get my copy of this book https://www.amazon.com/Unified-Modeling-Language-Addison-Wes.... You pay shipping.

Re: Model Once, Represent Everywhere: UDA (Unified Data Architecture) at Netflix

#29
post #13

For all the benefits, there is a large problem with this approach that often goes unacknowledged. It is fundamentally a business problem, rather than a technical problem, but it has impact on development speed, so it's secondarily a technical problem. The business contract with a consolidated data definition is that everyone in the business, no matter which domain, can rely on it. But think about the red tape that in…

The alternative is the same barriers, except with a parallel phone a friend governance model when you have to share data between verticals or programs.

It’s a classic pattern in public sector applications, where it’s partially deliberate.

Re: Model Once, Represent Everywhere: UDA (Unified Data Architecture) at Netflix

#30

Doesn't Google achieve the same result with Protobuf?

I share the same perspective .. I was also wondering how UDA handles the problem of evolving schemas, "old clients" communicating with newer server or vice versa.
Post reply on HN