Connector Overview and Lifecycle

What a connector is#

A connector is a process you build and host. It owns a connection to some outside system (a historian, an OPC UA server, a SQL database, an MQTT broker) and it makes that system's points reachable from Atlas under names Atlas understands.

It reaches Atlas over exactly one surface: a single SignalR hub at /hubs/connector, serialized with MessagePack. There is no REST ingestion path, no file drop, and no message bus. If a value is in Atlas and came from outside, it came through this hub.

The names a connector makes reachable are binding paths: slash-delimited strings rooted at the connector's own datasource id, for example historian-juice-factory/The Juice Factory/131-TT-01.PV. An Atlas modeller writes one of those paths into an instance map to bind a property. Neither side validates the other's spelling, which is the single most common reason a correctly built connector produces no data.

The shape of the relationship#

The relationship is not request/response, and it is not symmetric. Three properties define it, and each is worth knowing before you start:

1. Atlas decides what is interesting, the connector decides when it arrives. Atlas opens a subscription for a path; from that moment the connector pushes on its own schedule, unprompted. Atlas never polls for a current value.
2. A connector advertises capability, not data. The handshake tells Atlas which paths this connector could serve. Nothing is exchanged until Atlas finds a compiled domain that binds one of them.
3. The connector is the client and Atlas is the server, in both directions. Your process dials out to Atlas. Atlas calls back into your process through that same connection, using handlers you registered on it. You never open a listening port.

The two halves of the protocol#

There is no point-in-time pull. Atlas answers a scalar read out of its own store, from the last row a live subscription pushed. A connector that implements only GetRange serves history and will never show a current value anywhere in the product.

The seven wire methods are Handshake, GetRange, Subscribe, Unsubscribe, PushSubscriptionRows, PushSubscriptionStatus and PushSubscriptionWatermark. This article describes what happens and in what order. The Connector Protocol Reference owns every record shape, key number, enum ordinal and code sample. Nothing on the wire is defined here.

Before the lifecycle can start#

Three settings have to be right before any of what follows happens, and each one fails in a way that does not point at itself. All three are specified in Connector Protocol Reference, "Mandatory client setup".

• ConnectorMessagePackOptions.Options is mandatory on both ends. Atlas binds it; your client must bind the same instance. A bare .AddMessagePackProtocol() negotiates successfully and then fails on every frame, because Atlas frames with LZ4 and writes Guids as 16-byte binary. This is stated once here and never repeated: see ch09 for the exact symptoms and the code.
• The port is 4551 (HTTP) or 4552 (HTTPS). settings.config, Dockerfile:112-113. Under Docker compose the host is the container name: http://atlas:4551/hubs/connector. Ports 5214 and 8080 appear in older articles and Atlas has never listened on either.
• The hub is authenticated by default. See "Authentication" below.

The lifecycle#

1. Connect#

Build a HubConnection against the hub URL, attach the MessagePack options and an access-token provider, and give it a retry policy that never gives up. SignalR's stock WithAutomaticReconnect() stops after four attempts, so a connector using the default dies permanently the first time Atlas restarts for longer than about a minute.

Raise ServerTimeout well above the 30-second default. A wide backfill response travelling back to Atlas on a long-polling transport - a supported live-data transport - can still be in flight when the default timeout fires.

2. Register every handler before starting the connection#

Register GetRange, Subscribe and Unsubscribe with connection.On(...) before StartAsync(). These three are not hub methods you invoke; they are client-side handlers Atlas reaches through IHubContext.Clients.Client(connectionId).

Timing matters more than it looks. Atlas can invoke Subscribe the instant your handshake returns, because the handshake fires a registry event that starts reconciliation immediately. A handler registered after the handshake can miss the first batch.

3. Start the connection, then handshake#

Call StartAsync(), then invoke Handshake with an advertisement carrying your connector id, a display name, and one descriptor per logical datasource. Each descriptor lists the exact binding paths it serves and any templates it matches.

The handshake returns a value. ConnectorHub.Handshake is Task<HandshakeResponse> and returns new HandshakeResponse(_sessionId.Value). Treating it as fire-and-forget compiles, works, and costs you Atlas-restart detection. Invoke it as InvokeAsync<HandshakeResponse>(...) and keep the AtlasSessionId (see "Detecting an Atlas restart").

Atlas logs one Information line on success:

Connector 'timebase-connector' (Timebase Connect Host) handshook on connection abc123 with 3 datasource(s)

Two things about the handshake are easy to get wrong:

• It is a full replacement, and it is expected more than once. Re-handshaking on the same connection sweeps that connection's prior binding paths and template matchers before applying the new advertisement. This is the supported way to change your advertised path set at runtime. The shipped client re-handshakes whenever its route set changes, not only after a reconnect.
• Overlapping templates in one advertisement abort the whole handshake. throws before any of the advertisement is applied, so your InvokeAsync sees the exception and the connector stays connected but unregistered. Only the structural shape is compared: {x:int} and {x:string} overlap. One bad template costs you the entire connector, not just that template.

Serialize concurrent handshakes behind a lock. Startup, Reconnected and a routes-changed re-advertise can all race.

4. Atlas reconciles your advertisement against the compiled worlds#

This step is Atlas's alone, and it is where most "I handshook and nothing happened" reports end.

Atlas does not subscribe to the paths you advertised. It takes the bound properties out of every compiled, cached world, groups them by binding path, and resolves each of those paths back through the registry. The advertisement is only the lookup table.

The consequence is blunt: if (descriptorsByPath.Count == 0) return;. A perfect handshake against a domain that has never been compiled, or whose compile predates the binding, opens zero subscriptions and raises no error anywhere. Nothing is logged above Debug.

Resolution per path is an exact dictionary hit on the advertised binding paths, then a linear scan of every compiled template in the fleet. Because template scanning is fleet-wide, a Subscribe or GetRange can arrive for a path this datasource never explicitly advertised and only matched by shape. Re-match locally to route it internally.

Reconciliation also re-runs whenever a world is stored or invalidated, not just on handshake. Recompiling a domain retargets existing subscriptions, unsubscribes orphaned paths and opens new ones without any connector involvement.

One owner per path. If two connections advertise the same concrete path, the second overwrites the first, with a Warning ending Last-writer-wins.. Resolution returns exactly one route; there is no fan-out and no round-robin anywhere in the registry. Running two instances of the same connector gives you replacement or failover, never load sharing. The displaced instance goes dark and its subscriptions are swept.

5. Subscribe and acknowledge#

For each route with pending work, Atlas invokes your Subscribe handler once with a batch of requests and awaits a list of acks. Each request carries an Atlas-minted subscription id, the binding path, a pushdown and a diagnostic label.

Three rules govern your response:

1. The ack list must mirror the request list one-to-one, in order. Atlas pairs them by index, not by id. A short or reordered list mis-assigns every subscription after the first divergence.
2. Partial failure is normal and must not throw. Report an unsupported path as a NotSupported ack on that item. If the invocation itself throws, Atlas logs SubscribeAsync threw for route ...; skipping N pending subscription(s). and abandons the whole batch.
3. Key your internal state by the subscription id, not by binding path. One path can carry several subscriptions.

Atlas retains only Active and Pending acks. Anything else is logged at Debug as ack'd {Status}, not retained. and dropped. A retained ack produces one Information line, which is the single most useful log line in the whole chain:

Subscribed 3f9a... on historian-juice-factory/The Juice Factory/131-TT-01.PV via
abc123/historian-ds fanning to 2 bound property/properties (Active)

AllowedLatency on the ack is the one latency knob that is actually consumed. Atlas records it per binding path (RecordAllowedLatency, ) and the live-projection gate reads it back per gating series, taking the max across an instance, so one slow path bounds how long an aligned row waits. A null value clears any prior declaration and falls back to the global LiveProjection:AllowedLatencyMs, shipped at 2000 ms.

The AllowedLatency and HeartbeatInterval fields on the datasource descriptor are a different pair and are not read. > own declaration and XML comment; no consumer exists in src or tests. Declare latency on the ack. Do not conflate the two.

6. Push: snapshot, then updates, with watermarks#

From the ack onward the direction reverses and stays reversed. Your connector pushes; Atlas never asks again.

The ordering contract per subscription is strict:

1. Zero or more PushSubscriptionRows emits marked Snapshot. This is the bootstrap. The first emit carries the current value and lands in the property's cell exactly as a later update does.
2. Exactly one PushSubscriptionStatus of SnapshotComplete. It always fires, which is what makes the empty-snapshot case unambiguous.
3. Zero or more emits marked Update, indefinitely.
4. PushSubscriptionWatermark interleaved throughout.

For a plain tick stream the Snapshot/Update distinction is cosmetic. For a rowset-shaped subscription it is load-bearing: Snapshot means "replace what you have", Update means "apply on top".

Watermarks are not optional. A watermark is your attestation that no value newer than T exists that you have not already pushed. Atlas fans it out to per-leaf watermark events and downstream calculated properties are watermark-gated. A connector that pushes values but never watermarks freezes every consumer that waits for aligned rows, while looking perfectly healthy. Polled connectors should emit after every poll, including polls that produced nothing. Realtime connectors should run a heartbeat timer and emit wall_clock_now - allowedLatency when nothing has fired since the last beat.

Two things to plan for in this step. Emits sent while the connection is not Connected are dropped rather than queued, so emit only while the connection is Connected. And rows carrying a subscription id this Atlas process never minted are logged at Debug and discarded (Received rows for unknown subscription {SubscriptionId}; ignoring.,), which is precisely the failure the session id exists to prevent.

7. Teardown while connected#

There are two ways a live subscription ends without the connection dropping.

Atlas ends it. Atlas sends Unsubscribe with a list of ids using SendAsync: fire and forget, no ack, no return value. The contract is best-effort no-op on unknown ids, because an id may already have faulted or never existed. After the call returns, emit nothing further for those ids. Atlas issues this when a recompile leaves a path no longer bound.

You end it. Push a terminal PushSubscriptionStatus: Faulted when you lost the point, Removed when the source deleted it. Both do the same teardown on the Atlas side: drop the fanout, release the live refcount, clear the recorded AllowedLatency, and reset every affected watermark to long.MinValue so the next subscription's snapshot rows are not rejected as late. Atlas logs it at Information with your reason text.

Mid-stream Pending and NotSupported are not part of the contract. They are logged as Unexpected mid-stream status and ignored. Active mid-stream is informational.

8. Disconnect#

When the connection drops, for any reason, OnDisconnectedAsync calls _registry.Unregister immediately. There is no grace period. Every binding path and every template matcher owned by that connection is swept, and the orchestrator sweeps every subscription that was routed through it:

Connector abc123 unregistered — swept 12 subscription(s). Affected SoA cells retain last value.

That last sentence is the part worth internalising, and it is the opposite of what earlier versions of this article said.

Bound properties do NOT go bad quality when a connector drops. They freeze. Atlas keeps serving the last value it received for every affected cell. There is no staleness flag, no quality downgrade and no error surfaced to any consumer. A chart driven by a dead connector looks like a chart driven by a very stable process. The only signal is the log line above.

Atlas also carries one piece of state across the gap: each swept leaf's watermark is reset to long.MinValue so that when you come back, your snapshot rows are accepted rather than discarded as late. Your snapshot replay on reconnect is therefore load-bearing for correctness, not a courtesy.

The sequence end to end#

Your connector                                Atlas
──────────────                                ────────────────────
connect (bearer + MessagePack options)  ─────▶ hub auth: HubAccess policy
register GetRange / Subscribe /
         Unsubscribe handlers
Handshake(advertisement)                ─────▶ registry: sweep prior, apply new
HandshakeResponse(AtlasSessionId)       ◀─────
                                               reconcile advertised paths against
                                               every COMPILED cached world
Subscribe(request batch)                ◀───── (only for paths a compiled world binds)
SubscriptionAck[] (in request order)    ─────▶ retain Active/Pending, record AllowedLatency
PushSubscriptionRows(Snapshot) x N      ─────▶ write cells
PushSubscriptionStatus(SnapshotComplete)─────▶
PushSubscriptionRows(Update) x N        ─────▶ write cells
PushSubscriptionWatermark(T)            ─────▶ advance per-leaf watermarks
   ...
PushSubscriptionStatus(Faulted|Removed) ─────▶ tear down; cells retain last value
   or
Unsubscribe(ids)                        ◀───── fire and forget, no ack
   or
[disconnect]                            ─────▶ unregister all paths; sweep all
                                               subscriptions; reset watermarks;
                                               cells retain last value

Reconnection and Atlas restarts#

Reconnecting#

WithAutomaticReconnect restores the transport. It does not replay hub invocations, and Atlas already unregistered every path you owned. A reconnected connector that does not re-handshake is connected, healthy by every external measure, and completely invisible to Atlas.

Hook Reconnected and re-handshake from it. The shipped client does exactly this and logs Reconnected to {HubUrl}; re-handshaking. Use an indefinite retry policy so an Atlas restart of any length is survivable.

Detecting an Atlas restart#

AtlasSessionId is a Guid minted once per Atlas process and returned on every handshake. It is constant across your reconnects and different after Atlas restarts. Only one Atlas can bind the hub endpoint, so a changed value can only mean the process restarted.

It matters because subscription ids are minted by Atlas and are process-scoped. After an Atlas restart your connector is still holding ids the new process has never heard of. It keeps pushing against them, Atlas drops every frame at Debug level, and nothing anywhere reports a fault. That is data loss until a human restarts the connector.

The required behaviour, per the reference client:

1. Keep the last-seen AtlasSessionId.
2. On each handshake, compare. On a change, drop every locally tracked subscription id and run your own upstream unsubscribe so the source system's watches are released. The shipped client logs Atlas session changed ({Prev} -> {Curr}); resetting {Count} stale subscription(s).
3. Do not attempt to reuse the old ids. The handshake you just completed already triggered a fresh reconcile, and Atlas will mint new ones.

What Atlas does on its own restart#

Atlas recompiles every recorded branch and domain pair at startup, so the compiled worlds are back before your connector reconnects. You do not need to prompt a recompile after an Atlas restart.

Authentication#

/hubs/connector is gated.:

app.MapHub<ConnectorHub>("/hubs/connector").RequireAtlasHubAuth(app.Configuration);

RequireAtlasHubAuth applies the HubAccess policy whenever Pulse:Auth:Enabled is true, and Pulse:Auth:Enabled: true is the shipped default. HubAccess pins both the SPA cookie scheme and the JWT bearer scheme, so an unauthenticated negotiate is refused with a 401, not redirected.

A machine connector has no browser session. It must present a Pulse client-credentials bearer through SignalR's HttpConnectionOptions.AccessTokenProvider, and its Pulse client registration must list Atlas's Pulse:Auth:ClientId (default Atlas, settings.config) as an audience. Without the audience the connector obtains a perfectly valid token that Atlas still refuses, and the symptom does not change at all.

This is the failure two freshly installed products produce. Atlas ships Pulse:Auth:Enabled=true and Timebase Connect ships Pulse:Auth:Enabled=false, and the connector decides whether to attach a credential by reading its own setting, so a connector can retry indefinitely with a generic warning that names no cause. Network isolation is no longer a substitute for a credential; it is a second layer on top of one. Full walkthrough in Connector Diagnostics, step 2.

Only when Pulse:Auth:Enabled=false is the hub open, and disabling it opens every hub and every API on that server, not just this one.

Connector state as Atlas sees it#

GET /api/v1/connectors

Returns one entry per currently registered connector: connectorId, displayName, datasourceCount, registeredAt and an optional transportId. The same list drives the Connectors panel on Atlas's Home page. Send a bearer on every call.

Read it with three facts in mind:

• Presence equals online. There is no state field and no grace period; the registry unregisters from OnDisconnectedAsync. Absence means offline. An empty list is a success response, not an error.
• connectorId is the identity; transportId is a diagnostic. The SignalR connection id is process-scoped and changes on every reconnect, so key any tooling on connectorId.
• datasourceCount is datasources, never binding paths. A connector serving four thousand tags through one plugin instance reports 1. Atlas does not publish advertised binding paths anywhere; get them from the connector's own configuration.

There is no Atlas-side listing of subscriptions. The Subscribed ... (Active) log line from step 5 is the closest thing, and it is the check to reach for.

Troubleshooting#

Not covered here. Working through this chain requires specific log lines, status surfaces and API calls in a fixed order.

See Connector Diagnostics: Data Not Flowing, which walks the whole chain in six steps: registration, authentication, connect and handshake, compile and binding-path match, subscription open, and where the data lands.

The two cases most likely to send a connector author looking in the wrong place:

• A missing binding path is not an error. Nothing rejects it at save time, at compile time or at run time. A GetRange for a path no connector owns returns an empty rowset carrying the whole request back as residual, logged at Debug. It never returns null and it never throws.
• A quiet connector is more often an uncompiled domain than a connector fault. Check GET /api/v1/branches/{branch}/domains/{domain}/world and its isStale flag before touching connector code.

Related reading#

• Connector Protocol Reference - the canonical owner of every wire shape: method signatures, record definitions, MessagePack key numbers, union tags, enum ordinals, and the compilable client sample. Everything this article does not state.
• Connector Diagnostics: Data Not Flowing - the owner of troubleshooting for this whole chain.
• Property Sourcing Strategies - binding-path format, placeholder syntax, and how a property declares that it is bound.

Note: with no address configured, Connect uses localhost on 4551 without TLS - the same address the shipped atlas.config states, so absence and the shipped default mean the same thing. Set the target's host, port and useTls whenever Atlas is not on the same machine. The older compiled default of 5214 is gone.