Connector Range Queries

Overview#

GetRange is the only pull method on the connector wire. The point-in-time GetValue path and its eight *ValueResponse union members were deleted in (2026-05-14). Every historical read Atlas performs, and every schema probe it issues against a binding path, arrives at your connector as a GetRange invocation. (Current values are a different mechanism entirely - see Range versus subscription at the end.)

This article covers the behaviour of that path: what Atlas asks for, how often, what you must send back, and which of those obligations are load-bearing enough to break charts when you get them wrong.

The wire shapes live in Connector Protocol Reference - ConnectorRangeRequest, ConnectorRangeResponse, PushdownRequest, PushdownResidual, CompoundRowSet, the seven-member Column union, every [Key(n)] and [Union(n)]. This article links to those tables rather than restating them.

If you read one thing here, read How often Atlas calls you. Atlas issues many GetRange calls per user-visible query, with windows that overlap. A connector that is not side-effect-free and overlap-tolerant will appear to work in a smoke test and then misbehave under a real trend.

The handler#

connection.On<ConnectorRangeRequest, ConnectorRangeResponse>(
    ConnectorMethods.GetRange,
    request => HandleGetRange(request));

In the shipped connector host you implement IDatasource.GetRange and the host does the registration for you.

IDatasource.GetRange is synchronous and returns ConnectorRangeResponse, not a Task. Two consequences:

• Bridge async upstream clients yourself. The shipped Historian plugin uses .GetAwaiter().GetResult() and documents the choice.
• Implementations must be thread-safe. The interface says so directly: the host's SignalR handler runs on the connection's message-loop thread and may be invoked concurrently with other plugin calls.

A GetRange can arrive for a path you never advertised#

The host resolves the concrete binding-path map first, and on a miss re-tests the path against your compiled templates before giving up. Atlas may itself have resolved the request through a fleet-wide template scan. So request.BindingPath may be a path only matched by shape. Re-match locally to route it internally and to extract your template parameters.

What Atlas asks for, and when#

Four distinct callers reach your GetRange handler, with four distinct request shapes. All four go through ConnectorDataStore, which resolves the binding path in the registry and invokes you through ConnectorBackfillDispatcher.GetRangeAsync.

Two of those are easy to overlook and are covered in Special request shapes.

A GetRange call never puts you on a live channel. From the value hub's own comment: the GetRange* hub methods are server-to-client streams and "Unlike Subscribe*, they NEVER register a live channel - a GetRange caller is fully isolated from the live reactor fan-out." Serving a range call creates no ongoing obligation on your side.

How often Atlas calls you#

One thing a user does produces many GetRange invocations against the same binding path, with adjacent and overlapping windows. There are two independent multipliers.

Multiplier 1: one call per strategy segment (split-and-stitch)#

An Atlas property's strategy is a timeline, not a constant. A property can be BoundStrategy before some changeover instant and CalculatedStrategy after it. The split-and-stitch engine (design docs under docs/06112026_atlas-split-and-stitch-range-queries/) resolves each part of the window with the strategy that was actually in force there.

PropertyRetrievalService.ResolveRangeAsync:

1. Reads the declared time axis of the output compound (CompoundTypeDescriptor.TimeAxisFieldName).
2. Extracts the window from, to) from the request predicate with RangeSegmentation.TryGetWindowBounds.
3. Asks TemporalEdgeStore.GetRange(entityId, typeId, from, to) for every strategy edge overlapping the window, plus the pre-range covering edge, and orders them by ValidFrom.
4. One edge is the fast path: a single ResolveSingleStrategyRangeAsync for the whole window.
5. Two or more edges goes to ResolveStitchedRangeAsync, which loops the edges and resolves one segment per edge, each with a request clipped to [segStart, segEnd) by RangeSegmentation.ClipRequestToSegment.

Every Bound segment bottoms out in its own _store.GetRangeAsync, which is its own GetRange invocation on your connector.

The clipped per-segment request is built like this:

var axis = new FieldRefExpr(new[] { axisField });
var clip = new List<PushdownPredicate>(3)
{
    new ComparePredicate(PushdownCompareOp.Gte, axis, new DateTimeLiteralExpr(segStart)),
    new ComparePredicate(PushdownCompareOp.Lt,  axis, new DateTimeLiteralExpr(segEnd)),
};
// ... non-axis predicate fragments are ANDed back in
return new PushdownRequest(predicate, request.SortKeys, Limit: null, Skip: null);

Note Limit: null, Skip: null. The stitch layer owns limit and skip globally and applies them once over the concatenation.

Multiplier 2: one call per one-day slice (streamed windows)#

The value hub time-slices every streamed range window so peak server memory stays at about one slice:

private const long SliceTicks = TimeSpan.TicksPerDay;

The loop (mirrored, and) resolves each slice independently:

for (long segStart = fromTicks; segStart <= toTicks; segStart += SliceTicks)
{
    bool finalSlice = segStart + SliceTicks > toTicks;
    long segUpper   = finalSlice ? toTicks : segStart + SliceTicks;
    var slice = await svc.ResolveRangeAsync(
        entityId, componentId, WindowRequest(axisField, segStart, segUpper, finalSlice), ct);
    ...
}

The rule that matters to you is in the comment directly above that loop:

Each slice independently re-fetches its leading boundary/LKG - duplicate boundary rows are no-ops under the egress's guarded upsert (D7).

So the slices are not merely adjacent. Each one is expected to return the last known value at or before its own window start, which means the row Atlas already received at the end of the previous slice comes back again.

Interior slice seams are Gte(lower) and Lt(upper). The final slice closes with Lte(upper) so a sample sitting exactly on toTicks is included.

What that adds up to#

• A 30-day trend on one bound property with no strategy changeover: about 30 GetRange calls on that binding path.
• The same 30 days with one strategy changeover inside day 12: 31 calls, because that slice splits into two segments.
• Type mode (GetRangeTypeProperty) multiplies by the entity count: slices on the outside, entities on the inside. 30 days across 50 pumps is about 1500 calls.
• Instance mode (GetRangeInstance) multiplies by the instance's time-series property count, because each property is resolved per slice before point-normalization (AlignEntitySliceAsync,).

The one obligation this creates#

GetRange must be side-effect-free and overlap-tolerant.

• Side-effect-free. The same call can be issued many times for one user action, and nothing on the Atlas side deduplicates or coordinates them. Do not advance a cursor, consume a queue, mark rows as read, or mutate upstream state inside GetRange.
• Overlap-tolerant. Windows overlap by design at every slice boundary. Returning the same boundary row twice across two calls is correct behaviour, expected.
• Idempotent and cheap. Identical input window means identical output rows. If a repeat costs a full upstream scan, a 30-day trend costs 30 of them.
• Stateless across calls. Never assume you saw the previous window, and never assume the window you were given is the window the user asked for. It is usually a fragment.

Why an identical repeat may not reach you at all#

Within a single Cypher query, UnwindPropertyOperator caches range results per (entityId, componentId, pushdown):

var cacheKey = (entityId, _componentId, _pushdown);
if (Context.TryGetRangeFromCache(cacheKey, out var cached))
    return cached;

This is why PushdownRequest, PushdownResidual and PushdownPredicate all hand-roll deep Equals and GetHashCode over the whole predicate tree: the shipped record equality compares list references only, which would defeat the cache. Two structurally identical requests collapse to one connector call; two different entities always produce two.

The cache is per query, so it does not apply to the slice loop, which is a separate caller.

Finding the time window#

There is no TimeFrom or TimeTo field. PushdownRequest carries exactly Predicate, SortKeys, Limit, Skip (see [Connector Protocol Reference). The window is encoded inside the predicate tree, as ComparePredicates comparing a FieldRefExpr on the axis field against a DateTimeLiteralExpr carrying UTC ticks.

Use the shipped helper#

TimestampBounds is the extractor every shipped plugin uses.

var (minTicks, maxTicks) = TimestampBounds.Extract(request.Pushdown.Predicate);
bool hasMin = minTicks != long.MinValue;   // long.MinValue means unbounded below
bool hasMax = maxTicks != long.MaxValue;   // long.MaxValue means unbounded above

Extract walks the tree and tightens the two sentinels to the most restrictive bounds it can prove. The bounds it returns are inclusive ticks: the helper has already converted the operators for you, turning Gt(t) into min = t + 1 and Lt(t) into max = t - 1 (ApplyBound,). Eq pins both bounds to the same instant.

Operand order does not matter. TryNormalize flips a literal-on-the-left comparison.

What it will not do#

The same constraint applies to the split-and-stitch plan: per-segment clip predicates must use a datetime literal so the bound is recognised, and a bound axis that is not a Timestamp degrades to the default lookback.

Deciding a window when a bound is missing#

Both reference implementations use the same four-way shape. Historian and the reference Generator:

Pick a default lookback you can serve cheaply. The no-bounds case is reached by any caller that sends a bare predicate, and by the transforms rowset fetch.

Return the leading boundary row, do not trim exactly#

This is counter-intuitive enough to be worth stating on its own.

Atlas relies on a connector returning a bounding sample at or before the window start. The shipped Historian fetches [start, end], strips the timestamp comparisons into the residual, and BuildRangeResponse returns every row the historian yielded without re-filtering. Because the surfaced residual no longer carries the timestamp comparisons, Atlas's local re-application never trims that leading row away either.

That leading row is what makes step-interpolated charts correct across a slice seam, and it is what RangeSegmentation.ApplyLeakGuardAndSeam re-stamps up to a strategy boundary when a changeover lands mid-window.

The design plan is explicit that this is a connector-side property, and that connectors which do not provide it degrade rather than fail: "a Bound segment backed by a connector that returns no leading bounding value simply has no synthesized point at T" (out of scope).

Practical rule. Ask your upstream for the window plus its bounding sample, return whatever it gives you, and let the residual tell Atlas which filtering you did not do. A connector that helpfully trims to exactly [from, to) and then reports "I applied the whole time filter" loses the boundary sample on every slice.

Pushdown: what to apply, what to hand back#

PushdownResidual is a record, not an enum. PushdownResidual.None, .Predicates and .All do not exist and never did - Its four fields mirror PushdownRequest exactly.

The rule, in one line: return the sub-request you did NOT satisfy.

Atlas applies the residual over the rows you returned. UnwindPropertyOperator calls PushdownEvaluator.Apply when the residual is non-empty, and Apply runs a fixed pipeline of filter, sort, skip, limit. If you report something as applied that you did not apply, nobody applies it.

The three recipes#

StripTimestampComparisons returns the original tree minus the consumed timestamp comparisons, with SortKeys, Limit and Skip preserved verbatim. That is the "partial" answer the three-value enum in older drafts could not express.

PushdownResidual.Empty is not the safe default. Returning Empty from a path you do not actually serve tells Atlas "I filtered this correctly and zero rows is the answer", which suppresses the local re-application. Every shipped unknown-path fallback returns PushdownResidual.From(...) instead.

residual: null is illegal#

It compiles, it serializes, and it throws a NullReferenceException inside Atlas, where you cannot see it. Two call sites dereference residual.IsEmpty unguarded: and. Use PushdownResidual.Empty.

Limit and Skip are hints, and there is no pagination#

There is no continuation token, no cursor and no follow-up call anywhere in the connector protocol. Two things follow:

• Limit and Skip are usually null on the time-series path. RangeSegmentation.ClipRequestToSegment strips them per segment because the stitch layer owns them globally, and AtlasHub.WindowRequest always sends Limit: null, Skip: null. Do not conclude data is missing because Limit is absent, and do not build a paging strategy on it.
• Your response must be complete for the window you were asked for. Atlas will not come back for "the rest". Chunking is Atlas-side time-slicing, which you experience as repeated calls, not as pagination you drive.

The one place Limit is load-bearing is Limit == 0. See Special request shapes.

If you do honour Limit or Skip, remove them from the residual. If you do not, leave them in, which is what StripTimestampComparisons does for you.

Sort keys#

SortDirection is { Asc, Desc } and ordering arrives as request.Pushdown.SortKeys, a list of PushdownSortKey(FieldRefExpr Field, SortDirection Direction). There is no scalar OrderByTime.

Sort keys are populated only from a Cypher ORDER BY over the unwound alias. The value-hub and history paths always send an empty list. Because a sort key can name any leaf, check the field path matches your axis before acting on it:

public static bool WantsDescendingOnAxis(PushdownRequest pushdown, string axisField)
{
    foreach (var key in pushdown.SortKeys)
    {
        if (key.Field.Path.Count == 1 &&
            string.Equals(key.Field.Path[0], axisField, StringComparison.Ordinal))
        {
            return key.Direction == SortDirection.Desc;
        }
    }
    return false;
}

Leaving sort keys in the residual is always safe: Atlas sorts locally.

Special request shapes#

Limit = 0 is schema introspection#

BindingPathSchemaIntrospector.GetSchema issues:

var request = new PushdownRequest(
    Predicate: null,
    SortKeys:  Array.Empty<PushdownSortKey>(),
    Limit:     0,
    Skip:      0);

and then reads only response.Rows.Columns, mapping each to a ColumnSchema(Name, Kind). Rows and residual are discarded. It runs during instance-load and hot-reload validation for the transforms-layer semantic rules, cached per validation pass. Its own comment records the expectation: "Most connectors short-circuit Limit=0 to 'return columns, no rows'."

Answer it with your column headers and zero rows. Every shipped connector does this, skipping the upstream round-trip entirely because the shape is data-independent:, and, , and,.

The shipped empty-schema helper looks like this:

public static CompoundRowSet EmptyTvqSchema() => new(
    0, new Column[]
    {
        new DateTimeColumn("Timestamp", null, Array.Empty<long>()),
        new DoubleColumn("Value", null, Array.Empty<double>()),
        new IntColumn("Quality", null, Array.Empty<int>()),
    });

Getting this wrong breaks transform validation for that binding path, not the data. Returning zero columns makes Atlas believe the path advertises no columns at all.

Pair it with PushdownResidual.From(request.Pushdown), which is what all eight shipped connectors do. The introspector discards the residual, so the choice is about consistency rather than behaviour.

The no-pushdown rowset fetch#

Catalog-shaped and rowset-shaped binding paths (a SQL query, a tag catalog) are fetched with no pushdown at all. RowsetFetcher.FetchAsync sends PushdownRequest(Predicate: null, SortKeys: empty, Limit: null, Skip: null) and takes resp.Rows whole. Its comment states the design position:

No pushdown on the fetch: the connector returns whatever its pre-filter produced. Sort, limit, skip stay null; transforms are applied host-side over the full result. Atlas does not standardise a cross-connector filter syntax, so any narrowing happens connector-side via the connector's own filter syntax.

So a rowset path must return its full pre-filtered rowset, and any narrowing you want must be configured on your side (the Historian's catalog Filter option, for example). The residual is ignored on this path.

Empty results, and how to signal an error#

Two different empty responses#

They are not interchangeable.

// (a) You serve this path; there is simply nothing in the window.
//     Keep the column headers so a Limit=0 probe still reads a schema.
return new ConnectorRangeResponse(
    Rows: EmptyTvqSchema(),                 // CompoundRowSet(0, [Timestamp, Value, Quality])
    Residual: residual);                    // the residual you would normally return

// (b) You do not serve this path at all.
return new ConnectorRangeResponse(
    Rows: new CompoundRowSet(0, Array.Empty<Column>()),
    Residual: PushdownResidual.From(request.Pushdown));

Shape (b) is the shipped unknown-path fallback in three independent places:,.

Either way RowCount is 0, which is what actually short-circuits Atlas: CompoundRowSetAssembler.Assemble early-returns on rows.RowCount == 0.

Never let an exception escape GetRange#

There is no catch and no retry anywhere on the Atlas range path. ConnectorBackfillDispatcher.GetRangeAsync is a bare InvokeAsync<ConnectorRangeResponse>; ConnectorDataStore (82 lines) has no catch; neither do PropertyRetrievalService.ResolveRangeBoundAsync nor UnwindPropertyOperator.ResolveValuesAsync. An exception thrown out of your handler faults the whole invocation, which faults the entire query or value-hub stream - not just the one binding path. Every other property in the same query goes down with it.

The reference implementation handles an upstream failure by logging a warning, returning an empty rowset, and returning the residual you would have returned anyway, so Atlas re-applies the filtering it expects to re-apply.

Historian GetDataAsync failed for {Dataset}/{Tag} window [{Start:O}, {End:O}]; returning empty + Timestamp-stripped residual.

Do the same for timeouts, connection failures, authentication failures and malformed upstream responses. Surface the detail in your own logs and in the connector UI, not as a thrown exception.

Ordering#

Return rows ascending by the time-axis field, unless a sort key told you otherwise.

• Atlas does not sort on the single-segment fast path. RangeRowSetBuilder.Project documents its rows parameter as "one CompoundValue per sample, in ascending time order" and performs no sort of its own.
• RangeSegmentation.ApplyLeakGuardAndSeam does defensively re-sort, but only on the stitched path (two or more strategy edges in the window). Do not rely on it.
• PointNormalizer builds its own sorted, de-duplicated anchor grid for instance-mode alignment, so cross-property ordering is handled for you.
• If the residual carries sort keys, PushdownEvaluator.Apply sorts locally. That is the only place a wrong order gets corrected.

Column names: the contract in one line#

Column Name must equal the dotted leaf path of the compound type descriptor the binding is declared against - Timestamp, Value, Quality for the shipped TVQ compound, and parent.child.leaf for a nested one. Not t / v / q. Names are matched with an ordinal dictionary, so case matters, and column order is irrelevant on the compound path.

A name that does not match is a no match: CompoundRowSetAssembler logs "Compound {Uri} leaf '{Leaf}' missing from connector response; will be null" once per missing leaf and every value for that leaf comes through as null.

The full rules, the seven-member Column union, the sparse-Nulls convention, the RowCount-is-authoritative rule and the scalar-ranged-property rule are in Connector Protocol Reference.

Range versus subscription#

Both halves of the protocol carry the identical PushdownRequest, on purpose: SubscriptionRequest is documented as "conceptually a streaming ConnectorRangeRequest anchored at 'now forward' - the same PushdownRequest shape Atlas already builds for range queries so plugins reuse their query-translation logic across pull and push". Factor your query translation once and share it.

The behavioural split:

Two consequences worth stating plainly:

• A current value never reaches Atlas through GetRange. Point-in-time scalar bound values are served out of Atlas's own SoA store, populated by the subscription orchestrator. From: "Range/rowset only. Point-in-time scalar bound values come from the SoA column populated by the subscription orchestrator; no wire pull path remains." A connector that implements only GetRange can serve every chart and will never show a current value.
• A historical window never reaches Atlas through a subscription. If you implement only Subscribe, trends start empty and fill forward from the moment you connected.

Worked implementation#

Compiled against the shipped DTO and abstraction assemblies on (Timebase.Atlas.Shared.DTOs, Timebase.Connect.Abstractions, net10.0). It follows the same shape as HistorianInstance.GetTagDataRange with the branches inlined.

The handler#

public ConnectorRangeResponse GetRange(ConnectorRangeRequest request)
{
    // 1. A path this datasource does not serve: return nothing, and hand the
    //    whole request back so Atlas knows you filtered nothing.
    if (!_tagByPath.TryGetValue(request.BindingPath, out var tag))
        return new ConnectorRangeResponse(
            Rows: new CompoundRowSet(0, Array.Empty<Column>()),
            Residual: PushdownResidual.From(request.Pushdown));

    // 2. Schema introspection. Atlas wants the column list, not data.
    if (request.Pushdown.Limit == 0)
        return new ConnectorRangeResponse(
            Rows: EmptyTvqSchema(),
            Residual: PushdownResidual.From(request.Pushdown));

    // 3. The window lives in the predicate tree, not in a field.
    var (minTicks, maxTicks) = TimestampBounds.Extract(request.Pushdown.Predicate);
    bool hasMin = minTicks != long.MinValue;
    bool hasMax = maxTicks != long.MaxValue;

    var nowUtc = DateTime.UtcNow;
    DateTime startUtc, endUtc;
    if (hasMin && hasMax)
    {
        startUtc = new DateTime(minTicks, DateTimeKind.Utc);
        endUtc   = new DateTime(maxTicks, DateTimeKind.Utc);
    }
    else if (hasMin)
    {
        startUtc = new DateTime(minTicks, DateTimeKind.Utc);
        endUtc   = nowUtc;
    }
    else if (hasMax)
    {
        endUtc   = new DateTime(maxTicks, DateTimeKind.Utc);
        startUtc = endUtc - DefaultLookback;
    }
    else
    {
        endUtc   = nowUtc;
        startUtc = endUtc - DefaultLookback;
    }

    // 4. "I applied the Timestamp comparisons; you apply everything else."
    var residual = TimestampBounds.StripTimestampComparisons(request.Pushdown);

    if (startUtc > endUtc)
        return new ConnectorRangeResponse(EmptyTvqSchema(), residual);

    // 5. Never let an exception escape GetRange.
    IReadOnlyList<UpstreamSample> samples;
    try
    {
        samples = _upstream.Read(tag, startUtc, endUtc);
    }
    catch (Exception ex)
    {
        _log.LogWarning(ex,
            "Range read failed for {Tag} over [{Start:O}, {End:O}]; returning empty + residual.",
            tag, startUtc, endUtc);
        return new ConnectorRangeResponse(EmptyTvqSchema(), residual);
    }

    // Return whatever the upstream gave, including any leading bounding sample.
    return BuildTvqResponse(samples, residual);
}

Building the rowset#

private static ConnectorRangeResponse BuildTvqResponse(
    IReadOnlyList<UpstreamSample> samples, PushdownResidual residual)
{
    int n = samples.Count;
    var ticks   = new long[n];
    var values  = new double[n];
    var quality = new int[n];

    for (int i = 0; i < n; i++)
    {
        ticks[i]   = samples[i].TimestampUtc.ToUniversalTime().Ticks;
        values[i]  = samples[i].Value;
        quality[i] = samples[i].Quality;
    }

    return new ConnectorRangeResponse(
        Rows: new CompoundRowSet(
            RowCount: n,
            Columns: new Column[]
            {
                new DateTimeColumn(Name: "Timestamp", Nulls: null, Ticks:  ticks),
                new DoubleColumn  (Name: "Value",     Nulls: null, Values: values),
                new IntColumn     (Name: "Quality",   Nulls: null, Values: quality),
            }),
        Residual: residual);
}

// The zero-row form of the same shape. Keep the headers so Limit=0 reads a schema.
private static CompoundRowSet EmptyTvqSchema() => new(
    RowCount: 0,
    Columns: new Column[]
    {
        new DateTimeColumn("Timestamp", Nulls: null, Ticks:  Array.Empty<long>()),
        new DoubleColumn  ("Value",     Nulls: null, Values: Array.Empty<double>()),
        new IntColumn     ("Quality",   Nulls: null, Values: Array.Empty<int>()),
    });

RowCount is authoritative: Atlas reads exactly RowCount cells from every column and never inspects array Length. Set it to your row count and make every array at least that long.

Things to get right#

Related reading#

• [Connector Protocol Reference - all wire shapes, key numbers, union tags, and the MessagePack client setup this article assumes.
• Connector Overview and Lifecycle - connect, handshake, reconcile, subscribe, disconnect, reconnect.