Carlos EscasainCommercial Systems & Process Architect
ENES

Data case / 04 · Population, removals & idempotent reruns

Designing reliable segmentation with reconciliation

A fictional architecture case: computing the segment is easy; guaranteeing exactly one per eligible account, explicit exits and safe reruns is the design.

Fictional scenario8 min readUpdated 28 Sep 2026
Fictional scenario

Independently created. Contains no employer or client implementation detail, internal names or figures.

01The outcome

Every eligible account has exactly one current segment, every exit is written explicitly, reruns are idempotent, and each run ends with a reconciliation whose differences have owners.

The reality

Accounts end up with two segments or none, and campaigns keep targeting accounts that have already recovered.

Architecture question

How does every eligible account get exactly one current segment—and every rerun stay safe?

Key decision

Publish the full population per period with explicit removals, apply it as an idempotent diff, and reconcile expected against actual membership after every run.

ContextCustomers are classified monthly into On Track, Underperforming, At Risk and No Sale from their performance to target. The segment drives recovery tasks in the CRM and campaign membership. The pipeline writes new segments, but never removes old ones, and a rerun after a failure writes some accounts twice.

Disciplines involvedData & IntegrationsRevenue OperationsAutomation
Systems and partiesData platformCRMCampaign platform

02The reality · current state

What the data does today.

  • Some accounts carry two segments after a rerun
  • Accounts with no sales and no target get no segment at all
  • Accounts that left the population keep their last segment for months
  • Campaigns keep enrolling accounts that already recovered
  • Nobody can say which accounts the last run missed

03Target flow

Where the data comes from, where it goes—and how.

Every hand-over is named with its mode. The mode follows the business tolerance for waiting, inconsistency and loss.

  1. 01 · Data platformEligible population

    Active accounts with a target or sales history.

    Hands over by Batch
  2. 02 · Data platformSegment computation

    Exactly one segment per eligible account and period.

    Hands over by Batch
  3. 03 · Data platformDiff vs current

    Entries, changes and removals—keyed by account and period.

    Hands over by Batch
  4. 04 · CRMCRM membership

    Current segment, read-only, with version.

    Hands over by Event
  5. 05 · Campaign platform · CRMCampaigns & tasks

    Enrol, exit and create tasks from membership changes.

04Contracts

What one row or message means.

Meaning, grain, key, freshness and owner—written down before anything is mapped.

Contract

Segment membership

The current segment of one eligible account for one period.

Grain
Account × period
Key
Account + period + run version
Freshness
Monthly, rerunnable
Owner
Sales operations
Contract

Membership change

An entry, change or removal to apply downstream.

Grain
Account × change
Key
Account + period + change type
Freshness
Within a day of the run
Owner
Data platform team

05Key dimension · Population, removals & idempotent reruns

Expected membership against actual membership

After every run, the published population is compared with what the CRM and campaigns actually hold. Each class of difference has an owner and a repair.

Expected populationEligible accounts for the period, as published

PublishedIn CRM & campaignsCompared byAccount identityCurrent segmentPeriodRun versionEligibility
Show
Published compared with In CRM & campaigns, one row per key
KeyPublishedIn CRM & campaignsOutcome
ACC-A12On TrackOn TrackMatchBoth sides agree.
ACC-B07At RiskUnderperformingDifferenceA rerun applied an older version after the newer one.
ACC-C33Underperforming—MissingThe writeback batch stopped half-way.
ACC-D19Not eligibleNo SaleUnexpectedThe account left the population; its removal was never written.
ACC-E02At RiskAt Risk + No SaleDifferenceTwo memberships: the old one was never closed.
ACC-F58No SaleNo SaleMatchBoth sides agree.
  1. Match

    Same account, same segment, same version.

    Owner
    Nobody
    Action
    Record the check.
  2. Difference

    Same account, different segment or version.

    Owner
    Data platform team
    Action
    Re-apply the current version; the upsert by key is idempotent.
  3. Missing

    Published, not present downstream.

    Owner
    Integration operations
    Action
    Replay the missing keys from the run.
  4. Unexpected

    Present downstream, not in the published population.

    Owner
    Data platform team
    Action
    Write the removal; exit campaigns and close tasks.

A zero-segment account and a stale membership are the same failure seen from two sides. Only a full-population comparison sees both.

06Data authority

Who may create, change, read or derive each concept.

Highlight
Data authority by business concept: which system may create, update, read or derive each concept
Business conceptData platformCRMCampaign platform
Eligible populationDefined once in the data platform; the CRM does not decide who is in scope.DeriveReadNo authority
Current segmentDerived by the data platform; written to the CRM read-only.DeriveReadRead
Segment historyKept by period in the data platform, never in CRM fields.CreateNo authorityNo authority
Campaign membershipEnrolled and exited by the campaign platform from membership changes.ReadNo authorityCreateUpdate
Recovery taskCreated once per entry into At Risk; closed on exit.No authorityCreateUpdateNo authority

No system owns a whole record here. Authority sits with each concept—and sometimes changes hands when the lifecycle does.

07Failure & recovery

Design the failure path before the happy path.

  • Activation partially succeedsReconciliation: missingReplay the missing keys; nothing else is touchedIntegration operations
  • A rerun after a failureSame period, new run versionApply by version; older versions are ignoredData platform team
  • An account leaves the populationReconciliation: unexpectedExplicit removal; campaigns exit and tasks closeData platform team
  • An eligible account gets no segmentPopulation count vs segmented countFail the run before publishingSales operations

08Candidate architectures

Credible options, judged against these premises.

Rejected

Recompute and overwrite in the CRM

Small populations that never shrink

Cost: Removals and zero-segment accounts are invisible
Rejected

Keep segments only in analytics

Segments that inform, not act

Cost: Cannot drive tasks or campaigns
Selected

Full population, explicit removals, idempotent diff, reconcile

Segments that drive actions

Cost: Needs a run version and a reconciliation step

09The second layer

Questions that change the design.

Population

  1. What is the eligible population?
  2. How is an account with no segment detected?
  3. How are removals represented?

Activation

  1. What happens if activation partially succeeds?
  2. How are downstream campaigns and tasks kept aligned?
  3. Which system is authoritative for the current segment?

Reruns & history

  1. How is rerun idempotency guaranteed?
  2. How is historical membership retained?
  3. Who owns each class of difference?

10Decisions & outputs

What the work produces.

Architecture decisionADR 008 — Run reconciliation as a permanent control
  1. 01Segmentation contract
  2. 02Eligibility model
  3. 03Removal semantics
  4. 04Run versioning & rerun rule
  5. 05Reconciliation design
  6. 06Activation state model
Disciplines involvedData & IntegrationsRevenue OperationsAutomation