Incremental sync mechanics

Common to all models (NFS-e, CT-e, NF-e). This document covers what is identical across the /api/integration/dfe/{modelo}/inbound/{documentos|eventos} endpoints of any model: cursor-based pagination, token rules, polling semantics, rate limit and the error pattern. Each model has its own page with the endpoint, specific filters, payload examples and the numeric values that apply to it — see the "By model" section in Inbound.

1. PrerequisitesCopy Anchor Link

• Valid JWT (Bearer token) authentication. The customerId is automatically extracted from the token.
• All timestamps must be provided in epoch milliseconds UTC.

2. Cursor-based pagination (keyset pagination)Copy Anchor Link

The inbound endpoints return records ordered by creation date (createdAt), using keyset pagination: with each response the API returns a nextToken that encodes the position of the last record returned (internally, (createdAt, id)). This token is sent back in subsequent calls to advance from that point — it is not a traditional OFFSET.

Late records are silently skipped. If a record is persisted with a createdAt earlier than the position already passed by the cursor (e.g., out-of-order processing in the backend), it does not appear in any future page of the sequence — the cursor has already moved past that point. This is an inherent property of the keyset-by-createdAt design, not a bug. In practice this is rare (the window between creation and indexing is short), but the client should not assume a 100% record delivery guarantee from this mechanism alone; for critical reconciliation, consider periodically reprocessing a window with an earlier createdFrom.

hasMore semanticsCopy Anchor Link

3. Token rulesCopy Anchor Link

OpacityCopy Anchor Link

The nextToken is an encoded string that internally contains the cursor position and a hash of the filters used to generate it. The client must not interpret or manipulate the token content — only store it and send it back.

The nextToken key is omitted when there is no value — it never appears as nullCopy Anchor Link

The API does not serialize fields whose value is null. When there is no next page (empty response, or end of a sequence), the data object simply does not have the nextToken key — it is not "nextToken": null. A client that reads the JSON with direct access (data["nextToken"] in Python, data.nextToken typed as non-optional, etc.) breaks in this case; use a read that tolerates a missing key (data.get("nextToken"), Optional, a default value) — see §9.

Filter immutabilityCopy Anchor Link

All filter parameters used to generate a nextToken must remain identical across all calls of that pagination sequence (the exact set of filters varies by model/endpoint — see the specific page). If any filter is changed while a token is active, the API rejects the request with HTTP 400 and a {MODELO}_LISTING_TOKEN_FILTER_MISMATCH code (e.g., CTE_LISTING_TOKEN_FILTER_MISMATCH, NFSE_LISTING_TOKEN_FILTER_MISMATCH). To change the filters, discard the token and start a new sequence.

The token of one endpoint (documents) is not interchangeable with that of another (events), nor across models.

4. How to paginate step by stepCopy Anchor Link

Step 1 — First callCopy Anchor Link

Define the time window (createdFrom required) and the desired filters. Do not send nextToken.

Step 2 — Subsequent pagesCopy Anchor Link

Repeat the call adding the returned nextToken, with the same filters. Continue while hasMore == true, overwriting the token with each response.

Step 3 — End of dataCopy Anchor Link

At some point the response will come with fewer records than size, or empty — items: [], hasMore: false and without the nextToken key (see the note in §3 on omission of null fields). Do not discard the last non-null token you already had — it will be used for continuous polling.

Step 4 — Continuous pollingCopy Anchor Link

Resend the last call using the last non-null nextToken stored, with the same filters. New records created since the last query will be returned. If there is nothing new, the response comes back empty — wait an interval and try again.

5. What the client must storeCopy Anchor Link

Tip: store filters + token as a unit (a database record or JSON on disk), to restore the exact state after failures/restarts without risk of inconsistency.

6. When there are no more recordsCopy Anchor Link

items: [] and hasMore: false mean that at this instant there is no more data beyond the cursor — not that the process has ended. Incremental polling mechanics:

1. Keep the last non-null nextToken.
2. Wait an interval (respecting the rate limit).
3. Repeat the call with the token and the same filters.
4. Process the new records, if any.
5. Repeat.

7. Rate limitCopy Anchor Link

Isolated by customerId (tenant): each tenant has its own quota, with no interference between different clients. The value (requests per window) is specific to each model/endpoint — see the corresponding page — but the mechanism is always the same:

• Quota exceeded → HTTP 429, code {MODELO}_LISTING_RATE_LIMITED.
• It is recommended to spread polling across the window (e.g., with a 5/60s quota, an interval of 12–15s between calls avoids bursts).
• On a 429, apply backoff (e.g., wait for the full refresh period before trying again).

8. Error patternCopy Anchor Link

All inbound endpoints follow the same set of errors, with the code prefixed by the model (NFSE_LISTING_*, CTE_LISTING_*, etc.):

9. Generic sync loop exampleCopy Anchor Link

FILTERS = { "createdFrom": 1710400000000, "size": 50 }   # + filtros específicos do modelo/endpoint
POLL_INTERVAL_SECONDS = 15
next_token = storage.load("<modelo>_<endpoint>_token")   # None na primeira execução

while True:
    params = {**FILTERS}
    if next_token is not None:
        params["nextToken"] = next_token

    response = http_get("/api/integration/dfe/{modelo}/inbound/{documentos|eventos}", params)
    if response.status == 429:
        wait(60); continue

    data = response.json()["data"]
    for item in data["items"]:
        process(item)

    # nextToken pode estar AUSENTE do JSON (não é uma chave "null") — use .get(), nunca data["nextToken"]
    new_token = data.get("nextToken")
    if new_token is not None:
        next_token = new_token
        storage.save("<modelo>_<endpoint>_token", next_token)

    if data["hasMore"]:
        continue          # próxima página (respeitando rate limit)
    wait(POLL_INTERVAL_SECONDS)   # sem mais dados — polling

Key pointsCopy Anchor Link

• Initialization: on the first run the local next_token is None; the nextToken key is not sent in the first call, and the API returns the oldest records from createdFrom onward, in chronological order.
• Tolerant read: always read nextToken with .get() (or equivalent) — the API omits the key when there is no token, it never sends it as null.
• Draining: while hasMore == true, fetch consecutive pages (respecting the rate limit).
• Polling: when hasMore == false, wait and repeat.
• Idempotency: on failure/restart, reload the nextToken from storage and resume exactly where you left off — without duplicates.