Data Mesh Architecture
datamesh-architecture.com
Data Mesh Architecture
1–10 of 46 posts
Re: Data Mesh Architecture
#2Is there an underlying assumption here that all of the datasets' domains are perfectly in sync with each other in the context of domain metadata?
As an example, a Team1 might define the manufacturer of a Sprocket as the company that assembled it, whereas a Team2 might define the manufacturer as the company that built the Sprocket's engine. Since the purpose of a datamesh is to enable other teams to perform cross-domain data analytics, there needs to be reconciliation regarding these definitions, or it'll become a datamess. Where does that get resolved?
Re: Data Mesh Architecture
#3Is there an underlying assumption here that all of the datasets' domains are perfectly in sync with each other in the context of domain metadata? As an example, a Team1 might define the manufacturer of a Sprocket as the company that assembled it, whereas a Team2 might define the manufacturer as the company that built the Sprocket's engine. Since the purpose of a datamesh is to enable other teams to perform cross-doma…
The chief data officer in close collaboration with the chief data engineering officer must elaborate automated normalization guidelines backed with implementations used across all data streams to insure any skew in the data model is limited to non production environments and all data entities are materialized consistently across the whole data model.
Re: Data Mesh Architecture
#4It really feels like data mesh is a fairly half baked concept born out of short term consulting gigs and a desire to become a technical thought leader.
Re: Data Mesh Architecture
#5This seems like mostly common sense. Infrastructure teams should always be building tools that the org consumes (and ideally the general public)
In a lot of orgs this goes sideways and the infrastructure teams end up owning everything and never have time to do anything else. Usually this happens due to upper management putting on the squeeze.
In order for teams to actually own their infrastructure and data we need better tooling to help them. This is coming along nowadays but isn’t fully there.
Re: Data Mesh Architecture
#6If you need so many "slides" to persuade your clients of something, I think you lost already.
Re: Data Mesh Architecture
#7If you need so many "slides" to persuade your clients of something, I think you lost already.
Considering how many big companies go about implementing this right now, I don’t agree. C line likes slides.
Re: Data Mesh Architecture
#8Is there an underlying assumption here that all of the datasets' domains are perfectly in sync with each other in the context of domain metadata? As an example, a Team1 might define the manufacturer of a Sprocket as the company that assembled it, whereas a Team2 might define the manufacturer as the company that built the Sprocket's engine. Since the purpose of a datamesh is to enable other teams to perform cross-doma…
The chief data officer in close collaboration with the chief data engineering officer must elaborate automated normalization guidelines backed with implementations used across all data streams to insure any skew in the data model is limited to non production environments and all data entities are materialized consistently across the whole data model.
what type of company you are working for?
Usually there is not even CIO, I haven't even heard about company with both CDO and CDEO (or even CDEO itself).
I thought big portion of need that data mesh fills is the organizations who are missing resources in their core BI team.
Re: Data Mesh Architecture
#9It really feels like data mesh is a fairly half baked concept born out of short term consulting gigs and a desire to become a technical thought leader.
Reminds me of first OLAP cubes a lot, something that consultant online praise as much as possible, just so then 3-4 years later they are contracted by the company to fix the mess it created.
Re: Data Mesh Architecture
#10So if I understand this correctly, data mesh is just data mart, that doesn't bring data in database as a table, but uses S3 storage instead (I assume because thats cheaper in the cloud?)