Guide · Purview to OneLake Migration

How to inventory a Purview governance catalog before migrating

Updated 2026-11-21Python and DuckDBRuns shown on generated records, PDF by email

The short answer

Save every list the Unified Catalog API returns, from governance domains to policies. Run one Python and DuckDB script on the saved files. The script counts every object by type and domain, gives each object a first decision by its type (move, bridge, redesign or retire) for people to confirm, and checks that the export is whole.

Why inventory before anything moves

Microsoft announced on November 12, 2026 that Purview Data Governance is retiring: “Support for Purview Data Governance will end Nov. 12, 2027” (Microsoft Security blog, read 2026-11-21). Microsoft's transition guide (read 2026-11-21) says to move in waves, and its first step is to inventory the current footprint. The same guide gives a table: each Purview outcome, the Fabric target that covers it today, and the guidance for the move.

An inventory answers two questions before any wave is planned: how much of each kind of object each governance domain holds, and what the transition guide says should happen to each kind. This guide counts both from your own export.

What you will have at the end

  • Counts of every object in your Unified Catalog export, by object type and by governance domain.
  • A first decision for every object (move, bridge, redesign or retire), taken from the transition guide's row for the object's type, for people to confirm or change.
  • Four checks, each one PASS or FAIL, that the export is whole before anyone plans from it.
  • Two CSV files: objects.csv (one row per object, with its first decision and the reason) and inventory.csv (one row per domain).

Step 1. Save the export

The commands read the Unified Catalog API, version 2026-03-20-preview (Unified Catalog API, in public preview, read 2026-11-21), one list per object type: business domains, glossary terms, data products, critical data elements, objectives, data assets and policies. The identity that runs them needs a role in every governance domain. Microsoft's authentication page (read 2026-11-21) lists Data Catalog Reader for objectives, business domains, critical data elements, data products and terms, and lists policies only under Data Steward, Data Product Owner and Governance domain owner. The authentication page names no role for data assets. Policies answer with a values list and page with skipToken (Policies - List, read 2026-11-21); every other list answers value and pages with nextLink.

We did not run these commands. We have no Purview tenant to run them against, so they are written from Microsoft's reference pages, and the runs further down use generated records in the same shape. They need curl and jq.

# Not run by us: written from Microsoft's Unified Catalog API reference (read 2026-11-21).
# 1. A token for Purview (an app registration that holds Data Steward or Data Product Owner in every governance domain)
token() {
  curl -s -X POST "https://login.microsoftonline.com/$TENANT_ID/oauth2/token" \
    -d grant_type=client_credentials -d client_id="$CLIENT_ID" \
    --data-urlencode client_secret="$CLIENT_SECRET" -d resource=https://purview.azure.net \
    | jq -r .access_token
}
fetch() {   # fetch URL FILE: save the answer; on 401 get a new token and try once more
  code=$(curl -s -o "$2" -w '%{http_code}' -H "Authorization: Bearer $TOKEN" "$1")
  if [ "$code" = 401 ]; then
    TOKEN=$(token)
    code=$(curl -s -o "$2" -w '%{http_code}' -H "Authorization: Bearer $TOKEN" "$1")
  fi
  [ "$code" = 200 ] || echo "HTTP $code: $2 holds an error, run the commands again" >&2
}
TOKEN=$(token)
EP="https://api.purview-service.microsoft.com/datagovernance/catalog"   # the endpoint Microsoft's reference shows
API="api-version=2026-03-20-preview"
OUT="inventory-$(date +%Y%m%d-%H%M%S)"   # a new, empty folder for every export

# 2. Every object type, one file per page, following nextLink until there is none
for type in businessdomains terms dataProducts criticalDataElements objectives dataAssets; do
  mkdir -p "$OUT/$type"; url="$EP/$type?$API"; n=1
  while [ -n "$url" ]; do
    f=$(printf "$OUT/$type/page-%03d.json" "$n")
    fetch "$url" "$f"
    url=$(jq -r '.nextLink // empty' "$f"); n=$((n + 1))
  done
done

# 3. Policies answer "values" and page with skipToken
mkdir -p "$OUT/policies"; url="$EP/policies?$API"; n=1
while [ -n "$url" ]; do
  f=$(printf "$OUT/policies/page-%03d.json" "$n")
  fetch "$url" "$f"
  tok=$(jq -r '.skipToken // empty' "$f"); n=$((n + 1))
  url=""; [ -n "$tok" ] && url="$EP/policies?$API&skipToken=$(jq -rn --arg t "$tok" '$t|@uri')"
done
echo "Saved to $OUT"

Each run writes a new folder named inventory- with the date and time. When Purview answers 401 because the token expired, the commands get a new token and try the request once more. If a request fails for another reason, the commands print the file's name, and the inventory stops on that file; run the commands again.

Step 2. Run the inventory

Download inventory_catalog.py, install DuckDB (pip install duckdb) and run the script on the folder. The script reads the files with Python, loads them into DuckDB in memory and counts in SQL. Nothing is sent anywhere.

python3 inventory_catalog.py inventory-20261121-093000

The first decisions come from a table at the top of the script, DECISIONS. Each row names an object type, a decision and the transition-guide row the decision comes from:

Object typeFirst decisionWhy, from the transition guide
businessdomainsmoveCollections, domains, ownership -> Fabric domains and subdomains
termsredesignGlossary -> tags, descriptions, endorsement; the guide: do not claim full glossary parity
criticalDataElementsredesignCritical data elements -> tags, descriptions, endorsement; no CDE parity
dataProductsredesignData products -> domains, endorsement, sharing; define the product contract outside the catalog where needed
objectivesretireObjectives and key results: not in the transition guide's table; keep them in an archive of the export
policiesbridgeAccess policy -> Fabric Policies (preview) and permissions; the list also holds governance-domain role assignments, which map to domain admins and owners; do not assume universal coverage
dataAssets:AzureSqlTablebridgeCatalog search -> OneLake catalog for Fabric items; keep discovery for sources outside Fabric, or mirror the source
dataAssets:ADLSGen2PathbridgeCatalog search -> OneLake catalog for Fabric items; keep discovery for sources outside Fabric, or add a shortcut
dataAssets:GeneralbridgeCatalog search -> OneLake catalog for Fabric items; keep discovery for sources outside Fabric

A first decision is a starting point, not the answer. A data asset that already sits in Fabric moves rather than bridges; a policy that only one workspace needs may move to item permissions. Change a row in DECISIONS and the whole inventory follows.

CheckWhat a failure means
Every page of every object type was savedA page number is missing, or the last page still points to a next page, so objects are missing from every count.
No object id appears twice within its typePages overlapped, so an object is counted twice.
Every object's domain is a domain in the exportThe object belongs to a governance domain the export does not hold, often a domain the identity has no role in.
Every object type in the export has a decisionThe export holds a kind of object the DECISIONS table does not cover, so the object would get no decision.

Run 1. A generated export, clean

The script writes a generated export when you ask for one: three invented governance domains with their terms, data products, critical data elements and objectives, plus invented data assets and policies, with a fixed seed so every run gives the same records.

python3 inventory_catalog.py --make-demo demo
Wrote a generated export to demo: 132 objects of 7 types (seed 274)
python3 inventory_catalog.py demo
Python 3.14.8 duckdb 1.5.6
Export: demo
Read 132 objects of 7 types: businessdomains, criticalDataElements, dataAssets, dataProducts, objectives, policies, terms

Objects by type and first decision
  businessdomains                3  move
  criticalDataElements          18  redesign
  dataAssets:ADLSGen2Path        9  bridge
  dataAssets:AzureSqlTable      18  bridge
  dataAssets:General             3  bridge
  dataProducts                  12  redesign
  objectives                     6  retire
  policies                       3  bridge
  terms                         60  redesign
Objects by domain and decision
  Domain                    move  bridge  redesign  retire  total
  (no domain field)            0      33         0       0     33
  Finance                      1       0        30       2     33
  Sales                        1       0        30       2     33
  Supply Chain                 1       0        30       2     33

Checks
  PASS  Every page of every object type was saved
  PASS  No object id appears twice within its type
  PASS  Every object's domain is a domain in the export
  PASS  Every object type in the export has a decision

Wrote objects.csv (132 rows) and inventory.csv (4 rows) to .
RESULT: 0 findings. Every object has a first decision for people to confirm.

How to read it

Data assets and policies carry no domain field in the API, so they are counted under “(no domain field)”. Every row of the second table is the work one domain brings to a wave: what moves as it is, what needs a bridge to a source outside Fabric, what needs redesign because the Fabric target holds less, and what retires to an archive of the export. In run 2, the total of the “(no domain field)” row includes the custom metadata object, which has no decision.

Run 2. The same export with four planted errors

python3 inventory_catalog.py --make-demo demo-planted --plant-errors
Wrote a generated export to demo-planted: 134 objects of 8 types (seed 274)
Planted one error for each check: DUPLICATE_ID, NO_DECISION, PAGES_MISSING, UNKNOWN_DOMAIN
python3 inventory_catalog.py demo-planted --tag planted
Python 3.14.8 duckdb 1.5.6
Export: demo-planted
Read 109 objects of 8 types: businessdomains, criticalDataElements, customMetadata, dataAssets, dataProducts, objectives, policies, terms

Objects by type and first decision
  businessdomains                3  move
  criticalDataElements          18  redesign
  customMetadata                 1  (none)
  dataAssets:ADLSGen2Path        9  bridge
  dataAssets:AzureSqlTable      18  bridge
  dataAssets:General             3  bridge
  dataProducts                  12  redesign
  objectives                     6  retire
  policies                       3  bridge
  terms                         36  redesign
Objects by domain and decision
  Domain                    move  bridge  redesign  retire  total
  (no domain field)            0      33         0       0     34
  5d521927-7a23-4d91-bbad-341e415c497b     0       0         1       0      1
  Finance                      1       0        30       2     33
  Sales                        1       0        15       2     18
  Supply Chain                 1       0        20       2     23

Checks
  FAIL  Every page of every object type was saved  (1)
  FAIL  No object id appears twice within its type  (1)
  FAIL  Every object's domain is a domain in the export  (1)
  FAIL  Every object type in the export has a decision  (1)

Findings
  PAGES_MISSING   terms: terms/page-002.json was not saved
  DUPLICATE_ID    terms: Finance term 5 id on 2 rows
  UNKNOWN_DOMAIN  dataProducts: Finance dataProduct 1 domain 5d521927-7a23-4d91-bbad-341e415c497b is not in businessdomains
  NO_DECISION     customMetadata: objects of type customMetadata with no row in DECISIONS: 1

Wrote objects-planted.csv (109 rows) and inventory-planted.csv (5 rows) to .
RESULT: 4 findings. Fix the export or DECISIONS, then re-run.
CheckThe error planted in run 2
Every page savedPage 2 of the glossary terms was never saved.
No duplicate idsOne glossary term is repeated.
Known domainsOne data product belongs to a domain id the export does not hold.
A decision for every typeThe export holds a folder of custom metadata, a type the table does not cover.

Run the inventory on your own export

To run the inventory on your own catalog, save your export with the commands in step 1 into one folder, then run python3 inventory_catalog.py followed by that folder's path. Read the four checks before the counts, because a failed check means the counts or the first decisions cannot be trusted yet. You do not edit the script. The script reads saved files and changes nothing in Purview. Add --fail-on-findings to make it exit with code 1 when a check fails, and --out <folder> to write the CSVs somewhere other than the script's folder.

We tested a run on a reader's own export: a second generated export with other domain and object names, upper-case ids, empty fields, two pages of policies paged with skipToken, and one planted error for each check. All four checks failed as planted, and nothing else was flagged.

Download inventory_catalog.py · Get the full guide (PDF)

Results from the runs above: objects.csv and inventory.csv from run 1.

Get the full guide as a PDF

We'll email you a link to it. Please use your work email.

Limits

  • The runs on this page use generated records, not a real Purview tenant, and the export commands in step 1 were not run. The field names come from Microsoft's reference pages and the API specification, read on 2026-11-21.
  • The inventory covers what the Unified Catalog API lists. Data columns have no list operation, only a query, and this guide does not export data columns. Classic Data Map content, lineage and scan settings need their own export.
  • The first decisions are one reading of Microsoft's transition guide, by object type. People decide object by object; the script only gives them a starting point and the counts.
  • Dates are as of 2026-11-21. This guide is about Purview Data Governance, whose support ends on November 12, 2027; after that date the guide describes history.

Tested on

Tested on

Platform
DuckDB 1.5.6, run from Python 3.14.8
Edition
Open-source DuckDB library (pip install duckdb)
Date
2026-11-21
Records
Generated: an invented Unified Catalog of three governance domains, fixed seed 274, in the shape of API version 2026-03-20-preview (specification at commit be93aca, read 2026-11-21). Not a run against a Purview tenant.
Code
inventory_catalog.py
Results
objects.csv, inventory.csv

When this is not enough

The inventory gives counts and a first decision by type. A real plan needs a decision for each object, an owner and a steward for each domain, a list of the reports and pipelines that read the catalog, and a switch-off date for each domain.

Need a decision for every object, not only for every object type?

Purview to OneLake Migration is in early access. The offering starts with the four-day Catalog Transition Assessment: one fixed fee of $12,000, and no commitment to anything else. You get an inventory of every object in your Purview governance catalog and a decision for each object. AI does the first pass, and your stewards sign every domain.

See Purview to OneLake Migration