Documentation

Python wheel

Install the BARRY Python package.

This page walks through getting the BARRY PySpark connector wheel and using it in a notebook to read on-prem data into a Spark DataFrame. For the architecture and the serving contract, see the PySpark Connector page.

1. Download the wheel

On the Transports page, click ↓ Barry Wheel vX.Y.Z (top right). It always serves the latest published wheel — there is no version to pick and no promotion step. The file is a universal Python wheel:

barry_io_pyspark-<version>-py3-none-any.whl

If the button is not shown, no wheel has been published yet (or the server's wheel storage is not configured) — ask an administrator.

2. Install it in your notebook

Upload the wheel to a location your workspace can read (a Databricks Volume or workspace files), then install it and restart Python:

%pip install /Volumes/<catalog>/<schema>/<volume>/barry_io_pyspark-<version>-py3-none-any.whl
dbutils.library.restartPython()

Requirements: Databricks Runtime 15.4 LTS+ (or serverless environment v2), or Apache Spark 4.0+ — the Python Data Source API is required. pyarrow and requests are already on the Databricks runtime.

3. Register and read a table

from barry_io_pyspark import BarrySparkDataSource
spark.dataSource.register(BarrySparkDataSource)

df = (spark.read.format("barry")
        .option("base_url", "https://barry.example.com")
        .option("token", dbutils.secrets.get("barry", "serve-token"))
        .option("table", "Customers")
        .load())
df.show()

Options

Option	Required	Meaning
`base_url`	yes	the BARRY server origin
`token`	yes	a per-group access token (issue it in Transports → Manage groups → Serving access)
`table`	yes	the served table name
`verify_ssl`	no (default `true`)	set `"false"` only for dev servers with self-signed certs

Store the token in a Databricks secret scope (as shown) — don't inline it. A table is available only when a transport uses the Spark Direct Barry destination, belongs to a group, and that group has a token.

4. What to expect

Each read triggers a fresh live query against the on-prem source (snapshot; no change-data-capture). The first rows may take a few seconds while the on-prem client picks up the request.
A .filter(...) / .select(...) runs in Spark after the snapshot arrives — there is no pushdown to the source in this version.
A single table reads on one task (no in-table parallelism); read multiple tables as separate .load() calls to parallelize across them.

Troubleshooting

HTTP 404 on every call — the token is wrong, expired, or revoked, or the base_url is wrong. Re-issue the token from Manage groups → Serving access. (An authentication failure is reported as HTTP 404.)
format("barry") not found — the runtime is older than DBR 15.4 LTS / Spark 4.0, or spark.dataSource.register(BarrySparkDataSource) was not run in this session (re-run after restartPython()).
The download button is missing — no wheel is published, or serving storage is not configured on the server.

← PySpark connector

Mobile app →