Change data capture

BARRY moves data three ways: a full load, a watermark-based incremental load, or true log-based change data capture. Which one you get depends on the source, and you choose it per transport rather than for the whole system.

On SQL Server, enable Change Tracking on the database and on the tables you want to follow. BARRY reads the change-tracking version, asks the database which rows changed since the last run, and fetches only those. It needs no triggers and no shadow tables, and it behaves the same on SQL Server 2016 and later, Azure SQL Database, Managed Instance and RDS.

On PostgreSQL, BARRY reads the write-ahead log through a logical replication slot. You grant replication, BARRY uses or creates a slot, and inserts, updates and deletes flow straight from the WAL — including on managed Postgres. One thing to watch: an inactive slot holds WAL on the server, so BARRY manages its slot and you should drop any you no longer use.

On MySQL and MariaDB, enable the binary log in row format. BARRY tails the binlog using either GTIDs or file positions, so a table stays current without being repeatedly rescanned; MariaDB's three-part GTIDs are supported. Where log-based capture is not available at all — Oracle through the managed driver, for example — BARRY uses a watermark instead: you nominate a monotonically increasing column, an id or a modified timestamp, and each run reads only the rows past the last high-water mark.

Whichever route applies, the guarantees are the same. Extraction is read-only and deliberately low-impact: no triggers on your tables, no changes to your source schema, and no long-held locks on production. Once a transport is set up it simply stays fresh, and you stop thinking about the mechanism underneath it.

← All documentation