When Integration Fails

First: which half failed?#

Integration has two halves that fail differently:

• Selection — the Integration Engine deciding which values need sending (watermark-driven)
• Delivery — the send itself (SQL statement, MQTT publish, HTTP post)

A selection problem sends nothing and logs little. A delivery problem logs errors per attempt. Working out which half you are in halves the search space immediately.

Where the evidence lives#

1. Deployment state — the Integration Engine and the consumer must both be Deployed. Different means the running config is stale; Faulted means delivery is failing. This one check resolves a large share of cases.
2. The Windows event log Flow on the Platform host — delivery errors land here, per consumer. Note the trap: an event-source registration problem can render entries as "the description for Event ID 0 … cannot be found"; the message content after that boilerplate is still the real error — read past the boilerplate rather than dismissing the entry.
3. The measure's Integration tab in Flow Config — per measure-consumer attachment, showing how far integration has progressed (its watermark) and the Set control (below).
4. The destination itself — the only place token residue, timezone offsets, and silently-wrong rows are visible. Infohub reporting success means sending succeeded, nothing more.

Decoding the common errors#

Timeouts (Execution Timeout Expired, aggregate timeouts against SQL targets): the statement ran longer than the consumer's timeout. Three causes in order of likelihood — a missing index at the destination for the statement's WHERE/key columns; a value-version pile-up (below); a destination under load. Raising the timeout treats the symptom; run the statement manually against the destination and look at its plan.

The version pile-up. Infohub keeps versions of a value; event-context integration in particular can accumulate ten or twenty versions of one value, and integration statements that scan versions get slower as history deepens. If integration timeouts grow over months with no other change, this is the likely shape: the fix is at the statement (key on the preferred version — see [Preferred] in the placeholder reference) and at the source (why are so many versions being produced?).

"Not moving" with no errors: the watermark says everything is sent. Either nothing changed (verify a value actually changed in range), the measure is not attached to the consumer, or a namespace filter excludes it. All three are selection-half causes — no amount of destination-side investigation finds them.

Payload wrong at the destination, no Infohub error: token residue, timezone, or (Event Hub, pre-7.2.0) the hard-coded-string payload defect. Capture one real delivered message and compare token by token.

Per-family quick table

Backfilling a consumer — pushing history out#

Two distinct mechanisms, often confused:

The Set control (gear icon on the measure's Integration tab) moves the integration watermark for that measure-consumer attachment: set it back to a date and the engine re-sends everything from there. This is the tool when the destination lost data or a new consumer needs history that already exists in Infohub. It is per-measure and manual by design — resending is expensive and should be deliberate.

Backfilling the measure re-derives the values themselves; integration then sends whatever changed. Use this when the values need rework, not just resending. See Backfilling and Recalculation.

For a new consumer that must carry history: deploy it, confirm live values flow, then Set the watermark back in bounded steps (a month at a time), letting each drain before the next. One giant resend competes with live sending and can trip destination rate limits.

Event-scheme integration specifics#

Integrating event attributes has two extra behaviours worth knowing:

• It sends on event change, and an event that keeps changing (attributes being completed over hours) legitimately re-sends. The destination must upsert on event identity.
• The version pile-up above is most acute here — an attribute pushed via a measure with many value versions is the documented slow path. Prefer keying statements on the preferred version, and question any scheme whose events accumulate double-digit version counts.

Cleaning up wrong integrations#

Deleting or repointing an integration attachment stops future sends; it does not delete what was already delivered. Cleanup at the destination is a destination-side operation — do it with the destination's owner, keyed on whatever identity the contract says (name or ID), and only after the sending side is stopped, or the rows return on the next pass.

The monitoring that prevents the 3am version of this#

• Alert on Faulted consumers (deployment-status check — scriptable, see CLI Recipes for CI).
• Have the destination's owner alert on staleness at their end — they see silence before you do.
• For critical feeds, an Infohub measure tracking last-successful-send age, with a limit on it.

See also#

• What a Data Consumer Is
• Consumer Placeholder Reference
• SQL and Snowflake Consumers
• Reading Infohub Logs