Delivery & Formats

How data is delivered and what formats are available.

Delivery Methods

Cloud Storage

Delivery to your cloud storage bucket:

Amazon S3: Direct delivery to your S3 bucket
Google Cloud Storage: Delivery to your Google Cloud Storage bucket
Azure Blob Storage: Delivery to your Azure Blob Storage container

Secure File Transfer (SFTP)

Traditional SFTP delivery to your server:

Secure, encrypted transfer
Automated delivery scheduling
Compatible with all standard SFTP clients

Snowflake Marketplace

The person and company data is also listed on Snowflake Marketplace. A listing mounts as a database in your own Snowflake account, so you query and join it without running a pipeline, and each build replaces the data in place. See Snowflake for the listings and how to get them. We’ll work with you to set up the delivery method that works best for your infrastructure.

File Formats

CSV

Comma-separated values format:

Easy to import into spreadsheets and databases
Flat structure (nested data flattened)
Compatible with Excel, Google Sheets, and all major databases

Best for: Quick analysis, spreadsheet tools, simple database imports

JSON

Structured JSON format:

Preserves nested object structure
Arrays maintained as arrays
Human-readable format

Best for: Applications, APIs, structured data processing

Parquet

Columnar Parquet format:

Optimized for analytics and data warehouses
Efficient compression
Schema embedded in file

Best for: Data warehouses (Snowflake, BigQuery, Redshift), analytics platforms, large-scale processing

Delta

Delta Lake format:

Schema enforcement
Built on Parquet
Native support in Databricks, Spark, and other lakehouse platforms

Best for: Databricks, lakehouse architectures, teams already using Delta Lake

Update Frequency

Monthly Refreshes

Each delivery is a complete, full-file snapshot of all records matching your criteria. There are no incremental or diff files. Deliveries run monthly by default; quarterly and one-time schedules are also available.

Data Organization

Folder Layout

Each delivery lands in its own folder under the path you give us:

<your path>/<dataset>/<build-version>/part-*.<extension>
<your path>/<dataset>/<build-version>/_SUCCESS

For example, gs://your-bucket/datalegion/person/1.5.0_2026-10-15/part-00000-….json.gz. CSV deliveries, and deliveries flattened into related tables, have one folder per table instead:

<your path>/<dataset>/<build-version>/<table>/part-*.<extension>
<your path>/<dataset>/<build-version>/<table>/_SUCCESS
<your path>/<dataset>/<build-version>/_SUCCESS

Delta deliveries also include a _delta_log/ folder next to each set of part files.

Knowing a Delivery Is Complete

A _SUCCESS file is written last, after every data file is in place. Start processing when the one at the top of the delivery folder appears. If a build is ever re-delivered, its folder is replaced: files from the earlier run are removed first, including the old _SUCCESS.

Versioning

Each delivery includes:

build_version field in each record, naming the data build it came from (for example 1.5.0_2026-10-15), matching the folder name

Integration Examples

Loading into Database

PostgreSQL:

COPY data_legion FROM '/path/to/data.csv' WITH (FORMAT csv, HEADER true);

BigQuery:

LOAD DATA INTO `project.dataset.data_legion`
FROM FILES (format='PARQUET', uris=['gs://your-bucket/datalegion/person/1.5.0_2026-10-15/part-*.parquet']);

Processing with Python

The examples read one delivery folder, for example person/1.5.0_2026-10-15/. CSV:

import glob
import pandas as pd
df = pd.concat(pd.read_csv(f) for f in sorted(glob.glob('person/1.5.0_2026-10-15/part-*.csv*')))

JSON (gzip-compressed JSON Lines):

import glob
import gzip
import json
for path in sorted(glob.glob('person/1.5.0_2026-10-15/part-*.json.gz')):
    with gzip.open(path, 'rt') as f:
        for line in f:
            record = json.loads(line)

Parquet:

import pandas as pd
df = pd.read_parquet('person/1.5.0_2026-10-15/')  # skips _SUCCESS

Delta:

from deltalake import DeltaTable
dt = DeltaTable('data-legion-delta/')
df = dt.to_pandas()

Quickstart - Get started guide
Person Schema - Person data structure
Company Schema - Company data structure
Snowflake - Data Legion on Snowflake Marketplace
Get Sample Data - Get a sample dataset

Getting Started

Person Data

Company Data

Data Quality

Data Standardization

SDKs

Integrations

Delivery Methods

Cloud Storage

Secure File Transfer (SFTP)

Snowflake Marketplace

File Formats

CSV

JSON

Parquet

Delta

Update Frequency

Monthly Refreshes

Data Organization

Folder Layout

Knowing a Delivery Is Complete

Versioning

Integration Examples

Loading into Database

Processing with Python

​Delivery Methods

​Cloud Storage

​Secure File Transfer (SFTP)

​Snowflake Marketplace

​File Formats

​CSV

​JSON

​Parquet

​Delta

​Update Frequency

​Monthly Refreshes

​Data Organization

​Folder Layout

​Knowing a Delivery Is Complete

​Versioning

​Integration Examples

​Loading into Database

​Processing with Python

​Related Documentation

Delivery Methods

Cloud Storage

Secure File Transfer (SFTP)

Snowflake Marketplace

File Formats

CSV

JSON

Parquet

Delta

Update Frequency

Monthly Refreshes

Data Organization

Folder Layout

Knowing a Delivery Is Complete

Versioning

Integration Examples

Loading into Database

Processing with Python

Related Documentation