Output formats

File format, compression and folder layout for object-store sinks.

An Output Format template describes the format a classic storage destination is written in: the file or table format, its per-format settings, and the folder layout. Output formats are managed on the Output Format page and paired with a destination inside a transport.

When you need one

• Classic storage destinations (Parquet → ADLS Gen2, Parquet → OneLake) require an output format — it decides how the bytes are written.
• Self-contained destinations (Databricks managed Delta, Microsoft Fabric) do not need one. They pin both the staging format and the downstream apply step themselves, so the transport wizard skips the output-format step when you pick one of them.

What an output format carries

• Format — the file or table format:
  • File formats: Parquet (columnar, the default) and CSV.
  • Table formats: Delta and DuckLake (a transaction log/metadata layer over the files).

Iceberg is not an output format — it ships as a self-contained sink Connection (Apache Iceberg, AWS S3 Tables) that owns its own layout.

• Compression — e.g. Snappy for Parquet.
• Format-specific settings — for table formats, things like the Z-Order columns for Delta.

Folder layout (partition strategy) decides how files are organised in the target folder:

• Flat — every file lands in one folder.
• Hive-partitioned — files are organised into year=YYYY/month=MM/day=DD/... folders at a chosen granularity.

Hive partitioning is implemented; Flat is the user-selectable option in the current release.

Write modes and file formats

Not every write mode is valid for every format. Merge requires a transaction log to merge into, which plain file formats (Parquet, CSV) do not have. For file-format destinations the write mode is therefore locked to Overwrite: at write time the runtime clears the existing files in the target folder and writes the fresh dataset. The transport wizard enforces this — an invalid combination such as Merge into Parquet is surfaced as a row-level error on the review step. See Transports for how load and write modes are chosen.

Reuse

Like all template entities, an output format is defined once and reused across many transports. Typical organisation-level templates are things like Parquet + Snappy + Flat or Delta + Z-Order.