Data & Integrations
Design the contract before connecting the systems.
I define identity, authority, timing, failure and recovery before mapping fields—then shape operational data into signals the business can actually act on.
One commercial estate, read three ways: who owns each state, how it moves, and how both sides are reconciled when they disagree.
- CRMCommercial stateAccounts, opportunities, approvals—and the status sellers see
- Integration layerDelivery stateRequests, correlation, retries and delivery state—never business truth
- ERPLegal / transactional stateLegal customers, orders and invoices
- Data platformHistorical / derived stateConformed history, lineage and derived measures
- Analytics & activationPublished signalMetric definitions, segments and the signals written back
- CRM → to Integration layerEvent Creation command
- Integration layer → to CRMEvent Status
- Integration layer → to ERPAPI Idempotent create
- ERP → to Integration layerEvent Outcome
- CRM → to Data platformEvent Pipeline changes
- ERP → to Data platformBatch Orders & invoices
- Data platform → to Analytics & activationBatch Derive daily
- Analytics & activation → to CRMBatch Writeback: segment & KPI · reconciled
- CRM ↔ and ERPReconcile by request key · daily
Not API wiring. A business-state contract.
I do not treat integration as “map field A to field B and call the API”. The work is deciding what each fact means, who owns it, how it moves, and how everyone knows it arrived.
API wiringField mappingPoint-to-point automationCopying every ERP field into the CRMDashboards built on inconsistent definitionsRetrying failed calls until they “work”
- AuthorityAssign who may create, change and derive each fact.
- IdentityKeep one business entity recognisable in every system.
- ContractsDefine event, payload, meaning, version and validation.
- Interaction modeChoose sync, async or batch from business tolerance.
- Delivery stateExpose whether a transfer actually happened.
- Failure semanticsDistinguish failed, rejected and unknown.
- IdempotencyMake every retry and rerun safe by design.
- ReconciliationDetect silent divergence on a schedule, with owners.
- VersioningChange contracts without breaking their consumers.
- LineageTrace every derived value back to its sources.
- Metric contractsOne definition, grain, owner and freshness per metric.
- ActivationDeliver a signal to someone who acts on it.
- 01A timeout is not the same thing as a failure.
The caller may not know whether the receiving system committed the transaction.
Unknown outcomes - 02Retry without idempotency is duplication by design.
A second request for the same business intent must not create a second customer, order or task.
Unknown outcomes - 03“The source of truth” is often too coarse.
Authority exists per business concept—sometimes per attribute—and can move with the lifecycle.
Data authority - 04A metric without an owner and an action is reporting noise.
A metric is an operational contract: definition, grain, freshness, owner and the decision it should change.
Metric contracts
From business event to governed signal.
Ten moves from what happened in the business to what someone should do about it. Field mapping comes after the first four; dashboards come after all of them.
01Identify the business event
What actually happened in the business?
Integrations carry events, not tables.
- Prospect approved
- Customer created
- Order confirmed
- Invoice posted
- Sales target published
An integration without a named business event ends up syncing whatever changed—including noise
Identify the business event
What actually happened in the business?
Integrations carry events, not tables.
- Prospect approved
- Customer created
- Order confirmed
- Invoice posted
- Sales target published
Business event modelAn integration without a named business event ends up syncing whatever changed—including noise
Define authority
Which system is authoritative for each concept?
At concept or attribute level where needed—not per system.
- Create
- Update
- Read
- Derive
Data authority matrixAuthority can move with the lifecycle: a CRM proposal becomes an ERP fact at creation
Define identity
How do systems know they are talking about the same thing?
Identity is designed, not matched on names afterwards.
- Canonical identity
- System IDs
- Natural keys
- Correlation IDs
- Duplicate rules
- Merge and retirement
Identity modelMost integration defects are identity defects in disguise
Define the contract
What exactly is exchanged?
A contract is meaning plus shape, not an endpoint.
- Payload
- Semantic meaning
- Required fields
- Optional fields
- Version
- Validation
- Compatibility rules
Integration contractAn unversioned payload turns every change into a coordinated release
Choose the interaction mode
How quickly must the receiving system know?
Chosen from business tolerance, not fashion.
- Synchronous
- Asynchronous / event
- Batch
Interaction patternReal time is a cost; it needs a decision that loses value when delayed
Define delivery state
How do we know whether the transfer actually happened?
Delivery state is visible to the people who depend on it.
- Pending
- Sent
- Acknowledged
- Confirmed
- Failed
- Unknown
- Reconciled
Delivery-state modelAcknowledged is not confirmed: receipt is not a business outcome
Design failure and recovery
What happens when the outcome is uncertain?
Design the unknown outcome before the happy path.
- Retry
- Idempotency
- Replay
- Dead-letter handling
- Escalation
- Manual repair
- Ownership
Recovery designRetry without idempotency is duplication by design
Reconcile
How do we detect silent divergence?
Reconciliation is a permanent control, not cleanup.
- Expected population
- Comparison key
- Tolerance
- Frequency
- Discrepancy owner
- Repair action
Reconciliation modelAlerts see failed calls; only reconciliation sees records that never arrived
Shape analytical data
What should be stored operationally, historically and analytically?
Grain first; measures after.
- Facts
- Dimensions
- Grain
- Point-in-time history
- Derived measures
- Source lineage
Analytical data modelA CRM used as a warehouse gets slower and still cannot answer “as of when?”
Publish governed signals
Which metric should change which decision?
A metric is an operational contract, not a formula.
- Metric contract
- Owner
- Freshness
- Definition
- Threshold
- Action
- Activation destination
Signal-to-action modelA metric without an owner and an action is reporting noise
The symptoms arrive as “the sync is broken”.
Duplicates, stale statuses and three versions of the same KPI. Underneath, a contract, a key or an owner was never defined.
A timeout is logged as a failure—and someone presses “Create” again.
Usually missingAn unknown-outcome state and idempotent retryRetries create duplicate customers and orders downstream.
Usually missingIdempotency keys scoped to the business requestUsers cannot tell whether a record was synchronized.
Usually missingDelivery state exposed in business termsCRM and ERP disagree about the same customer status.
Usually missingAttribute-level authority and a daily reconciliationEach team computes the same KPI differently.
Usually missingOne metric contract with an ownerThe CRM has become a historical data warehouse.
Usually missingA boundary between operational and analytical dataA batch job succeeds while part of the population is silently missing.
Usually missingCompleteness checked against an expected populationPayloads change without warning and break their consumers.
Usually missingVersioned contracts with compatibility rules
Artefacts that make data trustworthy enough to act on.
Each one answers a question that field mapping leaves open.
- Business event catalogWhat actually happened in the business—and who needs to know?
- Integration contractWhich payload, meaning, version and validation cross the boundary?
- System & attribute authority matrixWho may create, update, read or derive each concept?
- Identity modelWhich keys prove two systems mean the same thing?
- Integration topologyWhich systems exchange what, and through which layer?
- Sync / async / batch modelHow quickly must the receiver know, and what may continue meanwhile?
- Delivery-state modelHow does everyone know whether a transfer happened?
- Idempotency strategyWhy can a retry or rerun never create twice?
- Recovery & replay modelWho repairs what, and how is replay made safe?
- Reconciliation designWhich population is compared, by which key, how often, owned by whom?
- Observability modelWhich business states are monitored—not just calls and jobs?
- Analytical grain modelWhat does one row mean, and at which grain do facts meet?
- Metric contractDefinition, owner, freshness—and the action it should change.
- Lineage modelWhich sources, rules and versions produced this value?
- Data activation flowWhere does a signal land, and who acts on it?
One flow, six layers of contract.
Customer creation, from the CRM approval to the status the seller sees. Read one layer across the flow, or open a step to see all six.
- Customer approved: CRM account ID, plus a request key minted once and scoped to the legal entity.
- Customer create requested: Request key for idempotency; correlation ID for tracing every hop.
- Customer validation: Natural keys—registration number, tax ID, country—are checked for an existing customer first.
- Customer created / rejected: The ERP customer number is linked to the request key and the CRM account ID.
- Visible status update: Cross-reference stored: account ↔ legal entity ↔ ERP customer.
Customer create requested
- Business event
- A command, not a fact: please create this legal customer.
- Payload
- Contract v2: legal name, registration and tax IDs, billing address, selling company, currency, request key. Required and optional fields declared; unknown fields rejected.
- Identity
- Request key for idempotency; correlation ID for tracing every hop.
- Authority
- The integration layer owns delivery—not the data it carries.
- Delivery state
- Pending → Sent → Acknowledged. Acknowledged means durably received, not created.
- Outcome
- Receipt confirmed. The business outcome is still open.
01Customer approvedCRM
- Business event
- Commercial approval is recorded: this prospect may become a legal customer.
- Payload
- Nothing leaves yet. The CRM freezes an immutable snapshot of the approved request.
- Identity
- CRM account ID, plus a request key minted once and scoped to the legal entity.
- Authority
- The CRM owns the approval and the commercial account.
- Delivery state
- Not started.
- Outcome
- The seller sees “Creation requested”.
02Customer create requestedIntegration layer
- Business event
- A command, not a fact: please create this legal customer.
- Payload
- Contract v2: legal name, registration and tax IDs, billing address, selling company, currency, request key. Required and optional fields declared; unknown fields rejected.
- Identity
- Request key for idempotency; correlation ID for tracing every hop.
- Authority
- The integration layer owns delivery—not the data it carries.
- Delivery state
- Pending → Sent → Acknowledged. Acknowledged means durably received, not created.
- Outcome
- Receipt confirmed. The business outcome is still open.
03Customer validationERP
- Business event
- The ERP validates legal and fiscal data before a customer exists.
- Payload
- The ERP works from the snapshot; it never calls back to ask the CRM for more.
- Identity
- Natural keys—registration number, tax ID, country—are checked for an existing customer first.
- Authority
- Only the ERP decides whether a legal customer exists, and its number.
- Delivery state
- Awaiting the outcome. A timeout here is Unknown, not Failed.
- Outcome
- Created, rejected with a stable reason—or unknown until reconciled.
04Customer created / rejectedERP → integration
- Business event
- An outcome event carrying the same request key.
- Payload
- Created: ERP customer number, company and validated legal data. Rejected: reason code, field and repair owner.
- Identity
- The ERP customer number is linked to the request key and the CRM account ID.
- Authority
- Legal name and billing address become ERP-authoritative from here on.
- Delivery state
- Confirmed—or Failed with a business reason.
- Outcome
- A business-readable result, not a stack trace.
05Visible status updateCRM
- Business event
- The CRM records the outcome the seller can act on.
- Payload
- Customer number and validated fields land read-only; a rejection shows its reason and owner.
- Identity
- Cross-reference stored: account ↔ legal entity ↔ ERP customer.
- Authority
- The CRM stops editing ERP-owned attributes and displays them instead.
- Delivery state
- Reconciled daily against the ERP by request key.
- Outcome
- The seller can order—or sees exactly who is fixing what.
An endpoint says where to send a request. The contract says what it means, who it is about, who may change what, how everyone knows it arrived—and what happens when it does not.
How quickly must the receiver know?
Synchronous, asynchronous and batch each fit a different business tolerance for waiting, inconsistency and loss. Start from a request, not from a technology.
Event Asynchronous / event
Validation can take minutes or need a person. The CRM continues in a visible pending state; the outcome arrives as an event.
What the business accepts: Eventual consistency, a delivery-state model and a daily reconciliation.
- API
Synchronous
Request–response- Best when
- An immediate answer is required
- The operation is short
- The caller can safely wait
- Watch for
- Coupling to the receiver’s availability
- Latency the user feels
- Timeouts with uncertain outcomes
- Needs
- A timeout budget
- Idempotent retry
- A degraded path when the receiver is down
- Event
Asynchronous / event
Command or event, outcome laterFits this request- Best when
- The process can continue in a pending state
- The operation is long or failure-prone
- Resilience matters more than immediacy
- Watch for
- Eventual consistency users must understand
- Out-of-order or duplicate delivery
- Silent loss without a check
- Needs
- Correlation
- Visible delivery state
- Retry and replay
- Reconciliation
- Batch
Batch
A population on a schedule- Best when
- Lower freshness is acceptable
- Processing the whole population matters
- Historical or analytical ingestion
- Watch for
- Partial loads that report success
- Late-arriving records
- Long recovery windows
- Needs
- Completeness checks
- A watermark or cursor
- Restartability
- Reconciliation
None of the three is more modern than the others. The architecture follows the business tolerance for waiting, for inconsistency and for loss.
A timeout is not the same thing as a failure.
The caller may not know whether the receiving system committed the transaction. Design that moment before the happy path.
- Request sentCreate customer · request key REQ-7F3A
- TimeoutNo response within the budget
FailedUnknownThe caller cannot know whether the ERP committed
Correct: Customer created once.
Wrong: A second legal customer is created. Orders and invoices now split across two.
Works only when the first attempt really failed—and the caller cannot know that.
- Idempotency key
One key per business intent, scoped to the legal entity. The receiver creates at most once per key.
- +Correlation
Every hop carries the same correlation ID, so an outcome is matched to its request without guessing.
- +Reconciliation
Whatever the resolution job cannot close is compared daily and owned by a named team.
Retry without idempotency is duplication by design. A second request for the same business intent must never create a second customer, order or task.
Delivery state, visible to the people who depend on it
Unknown
Timeout or lost response. The request may or may not have been committed.
- The seller sees
- “Outcome being confirmed”
- Can become
- Reconciled
- Owner
- Integration operations
Observe business state, not only calls
Technical status: HTTP 202 Accepted→Business status: Customer creation requestedTechnical status: Message delivered to the queue→Business status: Customer validation pendingTechnical status: HTTP 200 with an error body→Business status: Customer creation rejected: duplicate registration numberTechnical status: Socket timeout after 30 s→Business status: Customer creation unknown—reconcilingTechnical status: HTTP 200 OK→Business status: Customer created—number issuedTechnical status: Nightly job succeeded→Business status: Invoices for one company missing from the load
Recovery, owned
- Retry Transient faults only, with backoff and a budget—always with the same key.
- Replay Re-send a stored request or event; safe because the receiver is idempotent.
- Dead-letter Poison messages leave the flow with their reason, so one bad record cannot block the rest.
- Escalation Unknowns and dead letters older than their threshold page a named owner.
- Manual repair A form in the owning system—never a database edit—followed by a replay.
Reconciliation is part of the architecture.
Delivery guarantees fail silently. A scheduled comparison of the expected and the actual population—with an owner for every kind of difference—is how silent divergence becomes visible.
| Key | Expected · CRM | Actual · ERP | Outcome |
|---|---|---|---|
REQ-1041 | Created · ERP 30071 | Created · ERP 30071 | MatchBoth sides agree. |
REQ-1042 | Pending | Created · ERP 30072 | DifferenceThe outcome event was lost; the CRM never heard back. |
REQ-1043 | Billing address v3 | Billing address v2 | DifferenceAn address change after creation never reached the ERP. |
REQ-1044 | Requested | — | MissingThe request never arrived; it is stuck before the ERP. |
— | — | ERP 30090 · no request key | UnexpectedA customer was created directly in the ERP, outside the process. |
REQ-1046 | Created · ERP 30074 | Created · ERP 30074 | MatchBoth sides agree. |
- Match
Same identity, same state, within tolerance.
- Owner
- Nobody
- Action
- Record the check; nothing else.
- Difference
Same identity, different state or attribute.
- Owner
- Integration operations
- Action
- Apply the authoritative side’s value; replay the outcome.
- Missing
Expected on one side, absent on the other.
- Owner
- Integration operations
- Action
- Look up by request key; resubmit with the same key or close as rejected.
- Unexpected
Present on one side with no expected counterpart.
- Owner
- Master data governance
- Action
- Link it to an account, or flag an out-of-process creation.
The reconciliation contractPopulation, key, tolerance, frequency and owner—defined before the first run
- Expected population
- CRM accounts with a creation request in the last 30 days
- Comparison key
- Request key, then ERP customer number
- Compared
- Identity · business state · billing address · timestamp
- Tolerance
- Pending under 2 hours is not a difference
- Frequency
- Daily, and on demand after an outage
- Owner
- Integration operations; repairs by class
What is true now, what happened—and what to do about it.
Operational systems answer what is true now. The analytical platform answers what happened over time. Activation answers what someone should do because of it.
- CRM · ERP
- What is true now?
- What action should happen now?
- Data platform · warehouse
- What happened over time?
- How is performance derived?
- How do systems compare?
- CRM · campaigns · workflows
- What should someone do because of the signal?
- 01Transactions
Orders, invoices, opportunities—current state in the system that owns them.
- 02History
Every version kept, point in time, at a declared grain.
- 03Derivation
Measures computed once, from one definition.
- 04Signal
A published, versioned value with a freshness stamp.
- 05Action
A task, campaign or decision owned by a person.
Lineage: every step with a definition, an owner and a version
Performance to target %
- Definition
- (Invoiced + open orders) ÷ target, year to date.
- Owner
- Sales operations
- Version & time
- Metric definition v3
A metric without an owner and an action is reporting noise.
A metric is an operational contract, not a formula: one definition, a grain, a freshness promise, a place it is computed, a place it lands—and the decision it should change.
Pipeline coverage
Open eligible pipeline ÷ remaining target for the period
- Grain
- Account · segment · period
- Inputs
- OpportunityTarget
- Owner
- Sales operations
- Freshness
- Daily
- Computed in
- Data platform
- Activated in
- CRM
Below 3× remaining target
Generate pipeline where coverage falls below the threshold
Data is not value until it changes an action
- Signal: At Risk accountOwner: Account ownerAction: Recovery planOutcome: Performance back above threshold
- Signal: Low pipeline coverageOwner: Sales managerAction: Pipeline generation actionOutcome: Coverage restored for the period
- Signal: No SaleOwner: Commercial campaignAction: Reactivation taskOutcome: First order after reactivation
Five flows, five data dimensions.
Synthetic global B2B scenarios. The flagship CRM–ERP case is read here through its integration lens; four new cases cover grain, writeback, reconciliation and currency evolution.
- Case 01Selected workIdentity · async delivery · unknown outcome · reconciliation
Designing a CRM–ERP customer lifecycle
A create request can time out after the ERP has already committed—so a naïve retry creates a second legal customer.
What does a timeout mean—and how is the outcome settled without a duplicate?
CRMIntegration layerERPSystems & CRM ArchitectureProcess Architecture - Case 02Selected workGrain, identity & freshness
Turning ERP transactions and targets into commercial intelligence
Orders, invoices and targets arrive from different systems at different grains and times—so every report answers a slightly different question.
What is the grain of each fact, and where do they meet without double counting?
ERPPlanningCRMData platformRevenue OperationsSystems & CRM Architecture - Case 03Writeback, freshness & activation
Writing analytical signals back into operational CRM
Performance is computed outside the CRM, but sellers act inside it—so the numbers they see are stale, editable or unexplained.
Should the CRM hold a copy of the metric—and who owns it once it lands?
Data platformIntegration layerCRMRevenue OperationsAutomation - Case 04Population, removals & idempotent reruns
Designing reliable segmentation with reconciliation
Accounts end up with two segments or none, and campaigns keep targeting accounts that have already recovered.
How does every eligible account get exactly one current segment—and every rerun stay safe?
Data platformCRMCampaign platformRevenue OperationsAutomation - Case 05Evolution: V1 → V2 amount model
Evolving commercial analytics from single-currency to multi-currency
The first release reported in one currency. Subsidiaries that sell in others now threaten every total, threshold and historical report.
Which amount is the fact, which is a view—and can last year’s report still be reproduced?
ERPRate serviceData platformCRMRevenue OperationsSystems & CRM Architecture
Every data request has a second layer.
The visible request is where the design starts, not where it ends.
“Sync customers to ERP.”
Map the account fields to the ERP customer and call its API on save.
4 of 6 questions surfaced
The CRM owns the account, master data the legal entity, each ERP its customer number—linked, not merged.
Systems define responsibility. Data moves and reconciles it.
Data & Integrations
- Systems & CRM Architecture
Systems defines responsibility: which platform owns which concept. Data & Integrations moves and reconciles that state.
- Process Architecture
The process defines the business events and decisions that integrations carry.
- Revenue Operations
Revenue Operations turns governed signals into action—coverage, recovery, reactivation.
- Automation
Automation executes the downstream behaviour: tasks, enrolments, updates—once per signal.
- AI & Agentic Workflows
AI depends on reliable identity and context; an agent inherits every ambiguity in the data.
- Digital Operating Model
Owners for definitions, reconciliation and repair come from the operating model.
Architecture decisions
Related work
An integration is not a pipe. It is a contract that defines
- Identity
- Authority
- Event
- Payload
- Timing
- Delivery state
- Failure
- Recovery
- Reconciliation
And data architecture does not end when the data arrives.
It continues through history, derivation, a governed metric and activation—until someone takes an action.
Two systems that disagree—and nobody knows which one is right?
- What does a timeout mean in your integration today?
- Which KPI has three definitions—and no owner?
- What would a daily reconciliation find?