A Firmographic Dataset on Hugging Face: 28M+ Companies

If you work with data in Python, there's a good chance Hugging Face is already where your datasets come from. You sign in once, point a library at a repo name, and the rows are there. Company data has rarely worked that way. It usually arrives as a vendor export, a CSV someone emailed, or an API you have to page through before you can ask your first question.

Our free company dataset, Company Firmographics (Free), is now on Hugging Face as datalegionai/company-firmographics. It covers 28M+ companies worldwide, each with a domain, a LinkedIn URL, an industry and a size range, plus company type and founding year where we have them. On Hugging Face it's six Parquet files, about 1.5 GB in all, which means you can query it where it sits before deciding whether to download it.

This post covers getting access, querying the files in place with DuckDB, Polars or the datasets library, and the point where downloading a copy becomes the better choice. The launch post covers what's in the data and the license in more detail.

Getting access

The dataset is gated, so the files open once you've asked for them. On the dataset page, give your company and use case and accept the license. Hugging Face shares your username and email with us, and access is granted right away, with no wait for a review.

Then let your tools authenticate as you. Sign in from a terminal, or set a token from your Hugging Face settings as HF_TOKEN:

Shellhf auth login
# or, in a job or notebook:
export HF_TOKEN=hf_...

Every example below works with either one.

Query it where it sits

The files are Parquet, a columnar format, so a query reads only the columns it needs. DuckDB can read them straight from Hugging Face with an hf:// path. The credential_chain secret picks up your login or HF_TOKEN:

SQLCREATE SECRET hf (TYPE huggingface, PROVIDER credential_chain);

SELECT size, COUNT(*) AS companies
FROM 'hf://datasets/datalegionai/company-firmographics/data/*.parquet'
WHERE industry = 'software development'
GROUP BY size
ORDER BY companies DESC;

That counts every software company in the set by employee range, reading only the industry and size columns. In our test it finished in about 20 seconds, without downloading the full files. Reading in place works best for questions shaped like this one: counts, distributions, the top values of a column.

Polars reads the same path. pl.scan_parquet("hf://datasets/datalegionai/company-firmographics/data/*.parquet") builds a lazy frame, and only the columns your filter and group-by touch come over the network.

To look at rows without fetching the files, stream them with the datasets library. load_dataset("datalegionai/company-firmographics", split="train", streaming=True) yields records as it reads them, so the first ones arrive in seconds, and .take(5) stops after five. Drop streaming=True and load_dataset downloads and caches all six files instead, which is the next option.

When to download a copy

Reading in place has one weak spot: looking up specific companies. A filter on a domain can't skip any rows, because a domain could be anywhere in 28M records. So DuckDB has to scan the whole domain column across all six files and then fetch the matching rows. In our test, a lookup of a few domains took about three minutes over the network.

If you match lists against the data regularly, download it. It's about 1.5 GB:

Shellhf download datalegionai/company-firmographics \
  --repo-type dataset --local-dir company-firmographics

Against the local files, the same lookup takes a fraction of a second. Here it is with a list of account domains, joined in DuckDB:

Pythonimport duckdb

con = duckdb.connect()

accounts = ["airtable.com", "canva.com", "not-a-company.example"]
rows = con.execute(
    """
    SELECT domain, legion_id, name, industry, size, type, founded
    FROM 'company-firmographics/data/*.parquet'
    WHERE domain IN (SELECT unnest(?))
    """,
    [accounts],
).fetchall()

for domain, legion_id, name, industry, size, kind, founded in rows:
    print(domain, legion_id, name, industry, size, kind, founded)

JavaScriptimport { DuckDBInstance, LIST, listValue, VARCHAR } from "@duckdb/node-api";

const db = await DuckDBInstance.create();
const con = await db.connect();

const accounts = ["airtable.com", "canva.com", "not-a-company.example"];
const reader = await con.runAndReadAll(
  `SELECT domain, legion_id, name, industry, size, type, founded
   FROM 'company-firmographics/data/*.parquet'
   WHERE domain IN (SELECT unnest($1))`,
  [listValue(accounts)],
  [LIST(VARCHAR)],
);

for (const row of reader.getRowObjects()) {
  console.log(
    row.domain, row.legion_id, row.name, row.industry, row.size, row.type, row.founded,
  );
}

A domain that isn't in the set returns no row, so not-a-company.example drops out. A domain can also return more than one row, since a brand's locations sometimes share a website. Keep every row a domain matches and pick the one you want by name or LinkedIn URL.

Keep the legion_id

Every record carries a legion_id, the same company ID our API and paid company data use. A company you found here joins to the full profile later without matching it again: pass the ID or the domain to Company Enrichment and you get workforce analytics, growth and turnover, monthly headcount history and financial data on top of these nine fields. The workforce signals post shows what that monthly series is good for.

The dataset is refreshed every quarter, and each record's build_version names the build it came from, so a pipeline can tell when the files have moved on. It's also on Snowflake Marketplace and as a CSV or JSONL download from the dataset page. It's free for internal and commercial use with attribution, and the full terms cover the details.

Build it with a coding agent

To turn this into a script that matches your own account list, paste this into Claude Code, Codex, or Cursor:

TextBuild a script that matches a list of company domains against Data Legion's
free firmographic dataset on Hugging Face.

- Dataset: datalegionai/company-firmographics (gated: request access on the
  dataset page first, then authenticate with `hf auth login` or HF_TOKEN).
- Files: data/*.parquet, six Parquet files, about 1.5 GB in all.
- Columns: legion_id, name, domain, linkedin_url, industry, type, size,
  founded, build_version.
- Download once with:
  hf download datalegionai/company-firmographics --repo-type dataset
  --local-dir company-firmographics
- Query with DuckDB. For one-off counts, DuckDB can read the remote files in
  place via 'hf://datasets/datalegionai/company-firmographics/data/*.parquet'
  after CREATE SECRET hf (TYPE huggingface, PROVIDER credential_chain).

References for context (read these first):
- The walkthrough this is based on:
  https://www.datalegion.ai/blog/post/firmographic-dataset-on-hugging-face
- The dataset: https://huggingface.co/datasets/datalegionai/company-firmographics
- Dataset docs: https://www.datalegion.ai/docs/company-data/free-dataset

Requirements:
- Read the input domains from a CSV, lowercase them, and match on domain.
- Keep every record a domain matches; report domains with no match.
- Write the matches to a CSV with legion_id, name, domain, industry, size,
  type and founded.
- Skip the download when the local files already exist.

Before you start, ask me which language and framework to use. Sketch a short
plan, then build it with tests and a short README. When you finish, check the
result against each requirement above and tell me what is covered.