Transports

The central object: move this table, this way, to here.

A transport is the central object of BARRY. It captures one decision: "this table, from this source, lands in this sink, in this format, using this load strategy." Everything else in the product exists to be composed into a transport, and to start or watch its runs.

A transport is a thin row that references a source connection, a sink connection, and a couple of templates, plus a few per-table values:

• Source — the source connection (Connections).
• Load — the load and write strategy (see below).
• Sink — the sink connection (Connections).
• Output Format — the format, for classic object-store sinks (Output Formats).

When a transport runs is decided by its Triggers — a schedule, a message, a file arrival, and so on — or by a manual Run now.

Plus three values that are specific to the one table and therefore live on the transport itself, not on a reusable template:

• Data object — the qualified table name or file path. This is the transport's identity and is read-only after creation.
• Key columns — the columns used by the Merge write mode.
• Last-modified column — the watermark column used by incremental loads.

Creating transports — the wizard

The Transports page opens a multi-step wizard. Because you browse and select tables, the wizard can create many transports in one pass — one per selected table.

1. Source — pick a source connection. A preview shows its details and last test result.
2. Tables — the wizard asks the source's client to enumerate the available tables. You get a grid (schema, name) with a checkbox per row. For each row you can set the key columns (auto-filled from the reported primary key when available) and the last-modified column (auto-filled from a conventionally-named date column when present). Tick the tables you want.
3. Load — pick a Load Template (load mode + write mode).
4. Sink — pick a sink connection.
5. Output Format — pick the format. Skipped for self-contained sinks.
6. Review — choose a naming pattern that expands per table (tokens {table}, {schema}, {schema_table}), see one row per selected table with its templates and overrides, and create them all at once.

Every template dropdown has a + New … entry at the bottom, so a missing template can be created inline without leaving the wizard.

A Test run on the review step runs a single-table dry run against the first selected table, useful for catching misconfiguration before committing many transports. You can also Save as draft at any step and resume later.

Load modes and write modes

The Load strategy combines two independent choices.

Load mode — how much to read each run:

• Full — every run reads the whole table.
• Incremental — each run reads only what changed, using a watermark (see below). Requires a last-modified column.

Write mode — how to land it at the destination:

• Append — add the new rows.
• Overwrite — replace the dataset. (For file-format destinations this is the only valid mode — see Output Formats.)
• Merge — upsert by key columns. Requires key columns and a destination with a transaction log (Delta, not plain Parquet).

The wizard cross-checks these on the review step: a Merge without key columns, or an Incremental without a last-modified column, is flagged as a row-level error before you can create the transport.

Incremental loading (watermarks)

For incremental loads, the runtime tracks a watermark per transport so each run only pulls newer rows. Two strategies are available:

• Max observed value — the client scans the watermark column across the streamed rows and reports the highest value seen; the next run filters WHERE [column] > [max]. Precise: only rows strictly newer than anything already seen are re-pulled.
• Run-started timestamp — at run start the client reads the source database's own clock; the next run filters WHERE [column] > [previous run start]. Robust — no row can slip through even if it changed mid-run — but it re-pulls rows that changed during the previous read window, so duplicates are unavoidable by design. The UI warns you when this mode is selected.

The watermark value is stored on the server and handed to the client in the run; the client reports the new value back only after the data has been successfully shipped.

Change Tracking (continuous CDC)

For SQL Server sources that have Change Tracking enabled, a transport can run as a continuous data movement instead of a batch. The client polls the change table every few seconds and ships inserts, updates, and deletes as small Arrow batches, each carrying an operation marker and a change-tracking watermark. The first supported destination for this is Databricks managed Delta, whose MERGE applies the deletes too.

The runtime consumes change tracking; it never enables it on your database. Regular Full and Incremental batch transports are unaffected.

Where a continuous run writes

Like any run, a continuous change-feed run writes wherever its sink is placed (see Connections). A cloud-direct sink is applied by the server. A client-placed sink is written on the sink's own client: for a cross-client relay — the source on one client, the sink on another — the source client streams each change micro-batch to the server and the sink client picks it up as a fresh finite write and applies it. That cross-client relay is the continuous shape that lands on a client-placed sink today.

The detail page

Opening a transport shows its detail page. The Configuration tab lays out each slot as a section card — Source, Table, Load, Sink, Output Format — each with its own Edit affordance. Triggers and alerts attached to the transport appear on their own cards too.

• The Source, Load, Sink, and Output Format cards let you swap to a different connection or template, or spawn a new one inline. You never edit a connection's or template's properties from here — that happens on its own page.
• The Table card edits the per-transport values: key columns, last-modified column, load mode, and write mode. The data object itself is read-only — to move a different table, create a new transport.

No edit affects a run that is already in progress; changes apply to the next run.

Other tabs give an at-a-glance Overview, the full Runs history, the inferred Schema (source versus destination, with drift highlighted), and the transport's Logs.

Schema evolution

Sources change. Somebody adds a column, widens a decimal, makes a column nullable. Schema evolution is the per-transport policy that decides what the runtime does when the schema the source sends no longer matches what the destination holds.

Every run compares the incoming schema (what the source sent this run) against the baseline (what the destination held after the last run). The policy lives on the transport's Schema evolution card:

• Strict (the default) — abort the run on any detected change. Nothing is written, nothing is altered, and the destination stays exactly as it was. The change is yours to review and act on. This is the safe default: a silent schema change is how a downstream report quietly starts producing wrong numbers.
• AutoAlter — the runtime issues the DDL to evolve the target itself: type widening and adding a column with a NULL default. It never drops and never renames a column automatically — those are destructive and stay a manual decision.
• SourceHandlesEvolution — the sink adapts to the new schema on its own, without DDL. Typical for table-format sinks (Delta, Iceberg) whose readers handle schema evolution natively.

Some sink and format combinations are locked to Strict in this version — plain file formats (Parquet, CSV), the raw object stores, and the streaming sinks. The card says so when that applies.

Seeing what changed

The Schema changes section on the transport detail page is the history of every run whose schema differed from the baseline: when it was detected, the run it belongs to, the action taken (Aborted / Auto-altered / Propagated / No change), how many columns moved, and the error if the run was aborted.

Compare on a row opens the side-by-side view: the two schemas aligned column by column, baseline on the left, incoming on the right, with a marker per row — Added, Removed, Type changed, Nullability changed, Renamed. Unchanged columns are collapsed behind a toggle when there are many, so what moved is what you see. The very first run of a transport has no baseline; it is shown as such — that is the schema BARRY materialized, not a set of additions.

The same comparison appears on a run's own details when that run produced one. When a run failed with a Strict abort, this is the block that names the column responsible.

Running a transport

Press Run now on the detail page (or the list row) to queue a run. The owning client picks it up on its next poll — within a second or two, thanks to long polling — and executes it. You can also Pause a transport to keep it from being scheduled. Watch the run live and review its history on the Monitoring & Runs surfaces.

Cloning

The list row's Clone action opens the wizard pre-filled from an existing transport (all five template references plus the per-table values), so you can stand up a near-identical transport quickly.