Lakshmi Narayanan buys a policy from an insurer through its website. Two years later she buys a second policy through a bank branch that sells the insurer’s products. She calls the contact centre once from her husband’s phone, and files a claim through an agent who types her name as “Laxmi Narayan”. The insurer now has five records for one person across four systems. Marketing sends her offers for a policy she already holds. The claims team cannot see that she is a long-standing, multi-policy customer. Risk analysis treats her as five separate small exposures. None of this is unusual. It is the problem master data management exists to solve.

Master data versus everything else

Data in an organisation falls roughly into three groups. Transactional data records events: orders, payments, claims, calls. Reference data is small, slowly changing lists of allowed values: country codes, currency codes, policy status codes. Master data describes the core business entities that transactions are about and that many systems share: customers, products, suppliers, employees, locations, assets.

Master data is where duplication and conflict hurt most, because every transaction inherits its quality. If the product master has two codes for the same item, sales split between them. If the supplier master has duplicates, the company may pay the same invoice twice or miss volume discounts.

The core MDM process

Whatever tool is used, MDM follows a recognisable sequence.

1. Standardise. Before records can be compared, they need a common shape. Names are split into parts and upper-cased, addresses are parsed into street, city and PIN code, phone numbers are normalised to a consistent format with country code, common abbreviations are expanded (“Rd” to “Road”).

2. Match. Records that likely represent the same real-world entity are identified. Deterministic matching uses exact rules: same PAN means same person. Probabilistic or fuzzy matching scores similarity across several fields: “Lakshmi Narayanan” and “Laxmi Narayan” score highly on phonetic and edit-distance similarity, and if date of birth and PIN code also match, the overall score passes a threshold. Indian names add real difficulty here: spelling variation across scripts and transliteration, initials used in place of a father’s or village name in parts of South India, and single-word names are all common.

3. Merge into a golden record. Matched records are combined into one best version, sometimes called the golden record. Survivorship rules decide which value wins for each attribute. The mobile number might come from whichever source updated it most recently; the date of birth from the system that verified KYC documents; the email from the source with the highest trust score. These rules are business decisions and should be written down.

4. Distribute and maintain. The master record gets a stable master ID, and source systems either receive the cleaned data back or at least receive a cross-reference so they know their local record 88231 corresponds to master ID M-004517. New records are checked against the master as they are created, so duplicates are prevented rather than repeatedly cleaned up.

Matching thresholds are a trade-off

Every matching rule makes two kinds of error. A false merge combines two different people, perhaps a father and son with the same name at the same address. A missed match leaves duplicates in place. Raising the threshold reduces false merges but leaves more duplicates; lowering it does the opposite.

Which error is worse depends on use. For marketing, a missed match means one extra email, which is mildly annoying. For a bank, a false merge can expose one customer’s account details to another, which is a serious privacy and regulatory issue. That is why most MDM setups have three bands: automatic merge above a high score, no match below a low score, and a manual review queue in between, worked by data stewards.

Architectural styles

Industry literature, including Gartner’s, usually describes four implementation styles:

  • Registry: the hub stores only match results and cross-references; data stays in source systems. Least intrusive, quickest to build, but sources stay messy.
  • Consolidation: the hub builds golden records for analytics and reporting, but does not push changes back. Common as a first step and very useful for the warehouse.
  • Coexistence: golden records are synchronised back to sources, and sources can still create and update records.
  • Centralised (transaction hub): master data is created and maintained only in the hub, and sources consume it. Cleanest outcome, hardest to implement because it changes how operational teams work.

Many organisations move through these over time, starting with consolidation for analytics and adding synchronisation once the value is proven.

Why MDM projects disappoint

MDM has a reputation for expensive programmes that underdeliver. The usual causes are not technical.

  • Starting with every domain at once. Customer, product, supplier and employee have different owners and different problems. Starting with one domain tied to a visible business outcome works far better.
  • No stewardship. The review queue fills up and nobody works it. Survivorship rules are never revisited. A golden record with no owner decays like any other.
  • Fixing only downstream. If the branch onboarding form still allows free-text city names and skips duplicate checks, the hub cleans the same mess every night. Lasting improvement needs search-before-create checks in source applications.
  • Treating it as an IT project. Deciding whether two records are the same customer, or which address is correct, is a business decision. Without business ownership, rules are guessed by developers.

Why analysts should care

Nearly every customer metric depends on master data: customer count, retention, lifetime value, share of wallet, cross-sell rate. If one person is counted as five customers, retention looks worse (four of them “churned” after one purchase), customer count looks better, and average value per customer looks smaller. Before trusting any customer-level analysis, it is worth asking how customers were identified and de-duplicated. If the answer is “by the source system’s ID”, the numbers are probably describing records rather than people.

Written by the RP Strativa Team. The customer, insurer and IDs in the examples are illustrative.

Leave a Reply

Trending