Talked to a CTO of a small bank. He has 4 disparate legacy databases which need to be updated when a mortgage rate changes, which is obviously a problem.
What solutions does the HN community have in dealing with this issue?
Ask HN: How do you update multiple disparate databases?
1–10 of 55 posts
Re: Ask HN: How do you update multiple disparate databases?
#2Banks use an ancient but powerful architecture called Source of Truth (SoT), or system of record. It's one of the first techniques developed to manage heterogeneous distributed databases.
One master is picked as the final authority, then changes flow to other databases, replicas, etc. either using database tools or applications.
Awareness of deadlines or how much latency (ie. when the updates are needed for each downstream pipeline) is helpful.
A practical example that solves a simple case is the recent release of a tool by Netflix.
Pro tip: when you interview people for a project like this, they should already know the above. :)
Source: DBA.
Re: Ask HN: How do you update multiple disparate databases?
#3Banks use an ancient but powerful architecture called Source of Truth (SoT), or system of record. It's one of the first techniques developed to manage heterogeneous distributed databases. One master is picked as the final authority, then changes flow to other databases, replicas, etc. either using database tools or applications. Awareness of deadlines or how much latency (ie. when the updates are needed for each down…
I get the feeling OP is the one being interviewed.
Mortgage rates change frequently.
The bank likely deals with this exact scenario on a regular basis and should have an established procedure in place.
Re: Ask HN: How do you update multiple disparate databases?
#4Send 4 API calls to update the 4 databases.
Name the bank so we know not patronize patronize it.
Re: Ask HN: How do you update multiple disparate databases?
#5Paxos sync protocol targets such use cases, and even accounts for intermittent outages of one or many servers
Re: Ask HN: How do you update multiple disparate databases?
#6Banks use an ancient but powerful architecture called Source of Truth (SoT), or system of record. It's one of the first techniques developed to manage heterogeneous distributed databases. One master is picked as the final authority, then changes flow to other databases, replicas, etc. either using database tools or applications. Awareness of deadlines or how much latency (ie. when the updates are needed for each down…
I get the feeling OP is the one being interviewed. Mortgage rates change frequently. The bank likely deals with this exact scenario on a regular basis and should have an established procedure in place.
No, no interview. He's a friend of mine but I kept his problem in my head to possibly provide some solution.
Re: Ask HN: How do you update multiple disparate databases?
#7Change Data Capture. - share the consulting loot when you get yours.
https://en.wikipedia.org/wiki/Change_data_capture
Re: Ask HN: How do you update multiple disparate databases?
#8Banks use an ancient but powerful architecture called Source of Truth (SoT), or system of record. It's one of the first techniques developed to manage heterogeneous distributed databases. One master is picked as the final authority, then changes flow to other databases, replicas, etc. either using database tools or applications. Awareness of deadlines or how much latency (ie. when the updates are needed for each down…
I get the feeling OP is the one being interviewed. Mortgage rates change frequently. The bank likely deals with this exact scenario on a regular basis and should have an established procedure in place.
Source of Truth should (in my opinion) be used in many sectors but even finance doesn't use it everywhere. Having each data point originate in only one system helps a lot with data quality but is complex to introduce in existing systems.
Re: Ask HN: How do you update multiple disparate databases?
#9Need to watch out for update failure, some sort of transaction management/atomic commit[2] such that if one update fails, other updated dbs should rollback.
Re: Ask HN: How do you update multiple disparate databases?
#10CDC via Kafka Connect is one very powerful option.