How to inventory a Purview governance catalog before migrating
The short answer
Save every list the Unified Catalog API returns, from governance domains to policies. Run one Python and DuckDB script on the saved files. The script counts every object by type and domain, gives each object a first decision by its type (move, bridge, redesign or retire) for people to confirm, and checks that the export is whole.
Why inventory before anything moves
Microsoft announced on November 12, 2026 that Purview Data Governance is retiring: “Support for Purview Data Governance will end Nov. 12, 2027” (Microsoft Security blog, read 2026-11-21). Microsoft's transition guide (read 2026-11-21) says to move in waves, and its first step is to inventory the current footprint. The same guide gives a table: each Purview outcome, the Fabric target that covers it today, and the guidance for the move.
An inventory answers two questions before any wave is planned: how much of each kind of object each governance domain holds, and what the transition guide says should happen to each kind. This guide counts both from your own export.
What you will have at the end
- Counts of every object in your Unified Catalog export, by object type and by governance domain.
- A first decision for every object (move, bridge, redesign or retire), taken from the transition guide's row for the object's type, for people to confirm or change.
- Four checks, each one PASS or FAIL, that the export is whole before anyone plans from it.
- Two CSV files:
objects.csv(one row per object, with its first decision and the reason) andinventory.csv(one row per domain).
Step 1. Save the export
The commands read the Unified Catalog API, version 2026-03-20-preview (Unified Catalog API, in public preview, read 2026-11-21), one list per object type: business domains, glossary terms, data products, critical data elements, objectives, data assets and policies. The identity that runs them needs a role in every governance domain. Microsoft's authentication page (read 2026-11-21) lists Data Catalog Reader for objectives, business domains, critical data elements, data products and terms, and lists policies only under Data Steward, Data Product Owner and Governance domain owner. The authentication page names no role for data assets. Policies answer with a values list and page with skipToken (Policies - List, read 2026-11-21); every other list answers value and pages with nextLink.
We did not run these commands. We have no Purview tenant to run them against, so they are written from Microsoft's reference pages, and the runs further down use generated records in the same shape. They need curl and jq.
# Not run by us: written from Microsoft's Unified Catalog API reference (read 2026-11-21).
# 1. A token for Purview (an app registration that holds Data Steward or Data Product Owner in every governance domain)
token() {
curl -s -X POST "https://login.microsoftonline.com/$TENANT_ID/oauth2/token" \
-d grant_type=client_credentials -d client_id="$CLIENT_ID" \
--data-urlencode client_secret="$CLIENT_SECRET" -d resource=https://purview.azure.net \
| jq -r .access_token
}
fetch() { # fetch URL FILE: save the answer; on 401 get a new token and try once more
code=$(curl -s -o "$2" -w '%{http_code}' -H "Authorization: Bearer $TOKEN" "$1")
if [ "$code" = 401 ]; then
TOKEN=$(token)
code=$(curl -s -o "$2" -w '%{http_code}' -H "Authorization: Bearer $TOKEN" "$1")
fi
[ "$code" = 200 ] || echo "HTTP $code: $2 holds an error, run the commands again" >&2
}
TOKEN=$(token)
EP="https://api.purview-service.microsoft.com/datagovernance/catalog" # the endpoint Microsoft's reference shows
API="api-version=2026-03-20-preview"
OUT="inventory-$(date +%Y%m%d-%H%M%S)" # a new, empty folder for every export
# 2. Every object type, one file per page, following nextLink until there is none
for type in businessdomains terms dataProducts criticalDataElements objectives dataAssets; do
mkdir -p "$OUT/$type"; url="$EP/$type?$API"; n=1
while [ -n "$url" ]; do
f=$(printf "$OUT/$type/page-%03d.json" "$n")
fetch "$url" "$f"
url=$(jq -r '.nextLink // empty' "$f"); n=$((n + 1))
done
done
# 3. Policies answer "values" and page with skipToken
mkdir -p "$OUT/policies"; url="$EP/policies?$API"; n=1
while [ -n "$url" ]; do
f=$(printf "$OUT/policies/page-%03d.json" "$n")
fetch "$url" "$f"
tok=$(jq -r '.skipToken // empty' "$f"); n=$((n + 1))
url=""; [ -n "$tok" ] && url="$EP/policies?$API&skipToken=$(jq -rn --arg t "$tok" '$t|@uri')"
done
echo "Saved to $OUT"
Each run writes a new folder named inventory- with the date and time. When Purview answers 401 because the token expired, the commands get a new token and try the request once more. If a request fails for another reason, the commands print the file's name, and the inventory stops on that file; run the commands again.
Step 2. Run the inventory
Download inventory_catalog.py, install DuckDB (pip install duckdb) and run the script on the folder. The script reads the files with Python, loads them into DuckDB in memory and counts in SQL. Nothing is sent anywhere.
python3 inventory_catalog.py inventory-20261121-093000
The first decisions come from a table at the top of the script, DECISIONS. Each row names an object type, a decision and the transition-guide row the decision comes from:
| Object type | First decision | Why, from the transition guide |
|---|---|---|
businessdomains | move | Collections, domains, ownership -> Fabric domains and subdomains |
terms | redesign | Glossary -> tags, descriptions, endorsement; the guide: do not claim full glossary parity |
criticalDataElements | redesign | Critical data elements -> tags, descriptions, endorsement; no CDE parity |
dataProducts | redesign | Data products -> domains, endorsement, sharing; define the product contract outside the catalog where needed |
objectives | retire | Objectives and key results: not in the transition guide's table; keep them in an archive of the export |
policies | bridge | Access policy -> Fabric Policies (preview) and permissions; the list also holds governance-domain role assignments, which map to domain admins and owners; do not assume universal coverage |
dataAssets:AzureSqlTable | bridge | Catalog search -> OneLake catalog for Fabric items; keep discovery for sources outside Fabric, or mirror the source |
dataAssets:ADLSGen2Path | bridge | Catalog search -> OneLake catalog for Fabric items; keep discovery for sources outside Fabric, or add a shortcut |
dataAssets:General | bridge | Catalog search -> OneLake catalog for Fabric items; keep discovery for sources outside Fabric |
A first decision is a starting point, not the answer. A data asset that already sits in Fabric moves rather than bridges; a policy that only one workspace needs may move to item permissions. Change a row in DECISIONS and the whole inventory follows.
| Check | What a failure means |
|---|---|
| Every page of every object type was saved | A page number is missing, or the last page still points to a next page, so objects are missing from every count. |
| No object id appears twice within its type | Pages overlapped, so an object is counted twice. |
| Every object's domain is a domain in the export | The object belongs to a governance domain the export does not hold, often a domain the identity has no role in. |
| Every object type in the export has a decision | The export holds a kind of object the DECISIONS table does not cover, so the object would get no decision. |
Run 1. A generated export, clean
The script writes a generated export when you ask for one: three invented governance domains with their terms, data products, critical data elements and objectives, plus invented data assets and policies, with a fixed seed so every run gives the same records.
python3 inventory_catalog.py --make-demo demo
Wrote a generated export to demo: 132 objects of 7 types (seed 274)
python3 inventory_catalog.py demo
Python 3.14.8 duckdb 1.5.6 Export: demo Read 132 objects of 7 types: businessdomains, criticalDataElements, dataAssets, dataProducts, objectives, policies, terms Objects by type and first decision businessdomains 3 move criticalDataElements 18 redesign dataAssets:ADLSGen2Path 9 bridge dataAssets:AzureSqlTable 18 bridge dataAssets:General 3 bridge dataProducts 12 redesign objectives 6 retire policies 3 bridge terms 60 redesign Objects by domain and decision Domain move bridge redesign retire total (no domain field) 0 33 0 0 33 Finance 1 0 30 2 33 Sales 1 0 30 2 33 Supply Chain 1 0 30 2 33 Checks PASS Every page of every object type was saved PASS No object id appears twice within its type PASS Every object's domain is a domain in the export PASS Every object type in the export has a decision Wrote objects.csv (132 rows) and inventory.csv (4 rows) to . RESULT: 0 findings. Every object has a first decision for people to confirm.
How to read it
Data assets and policies carry no domain field in the API, so they are counted under “(no domain field)”. Every row of the second table is the work one domain brings to a wave: what moves as it is, what needs a bridge to a source outside Fabric, what needs redesign because the Fabric target holds less, and what retires to an archive of the export. In run 2, the total of the “(no domain field)” row includes the custom metadata object, which has no decision.
Run 2. The same export with four planted errors
python3 inventory_catalog.py --make-demo demo-planted --plant-errors
Wrote a generated export to demo-planted: 134 objects of 8 types (seed 274) Planted one error for each check: DUPLICATE_ID, NO_DECISION, PAGES_MISSING, UNKNOWN_DOMAIN
python3 inventory_catalog.py demo-planted --tag planted
Python 3.14.8 duckdb 1.5.6 Export: demo-planted Read 109 objects of 8 types: businessdomains, criticalDataElements, customMetadata, dataAssets, dataProducts, objectives, policies, terms Objects by type and first decision businessdomains 3 move criticalDataElements 18 redesign customMetadata 1 (none) dataAssets:ADLSGen2Path 9 bridge dataAssets:AzureSqlTable 18 bridge dataAssets:General 3 bridge dataProducts 12 redesign objectives 6 retire policies 3 bridge terms 36 redesign Objects by domain and decision Domain move bridge redesign retire total (no domain field) 0 33 0 0 34 5d521927-7a23-4d91-bbad-341e415c497b 0 0 1 0 1 Finance 1 0 30 2 33 Sales 1 0 15 2 18 Supply Chain 1 0 20 2 23 Checks FAIL Every page of every object type was saved (1) FAIL No object id appears twice within its type (1) FAIL Every object's domain is a domain in the export (1) FAIL Every object type in the export has a decision (1) Findings PAGES_MISSING terms: terms/page-002.json was not saved DUPLICATE_ID terms: Finance term 5 id on 2 rows UNKNOWN_DOMAIN dataProducts: Finance dataProduct 1 domain 5d521927-7a23-4d91-bbad-341e415c497b is not in businessdomains NO_DECISION customMetadata: objects of type customMetadata with no row in DECISIONS: 1 Wrote objects-planted.csv (109 rows) and inventory-planted.csv (5 rows) to . RESULT: 4 findings. Fix the export or DECISIONS, then re-run.
| Check | The error planted in run 2 |
|---|---|
| Every page saved | Page 2 of the glossary terms was never saved. |
| No duplicate ids | One glossary term is repeated. |
| Known domains | One data product belongs to a domain id the export does not hold. |
| A decision for every type | The export holds a folder of custom metadata, a type the table does not cover. |
Run the inventory on your own export
To run the inventory on your own catalog, save your export with the commands in step 1 into one folder, then run python3 inventory_catalog.py followed by that folder's path. Read the four checks before the counts, because a failed check means the counts or the first decisions cannot be trusted yet. You do not edit the script. The script reads saved files and changes nothing in Purview. Add --fail-on-findings to make it exit with code 1 when a check fails, and --out <folder> to write the CSVs somewhere other than the script's folder.
We tested a run on a reader's own export: a second generated export with other domain and object names, upper-case ids, empty fields, two pages of policies paged with skipToken, and one planted error for each check. All four checks failed as planted, and nothing else was flagged.
Download inventory_catalog.py · Get the full guide (PDF)
Results from the runs above: objects.csv and inventory.csv from run 1.
Get the full guide as a PDF
We'll email you a link to it. Please use your work email.
Thanks, . We're sending the link to now. If it isn't there in a few minutes, check your spam or junk folder, or email info@datameaning.com.
Limits
- The runs on this page use generated records, not a real Purview tenant, and the export commands in step 1 were not run. The field names come from Microsoft's reference pages and the API specification, read on 2026-11-21.
- The inventory covers what the Unified Catalog API lists. Data columns have no list operation, only a query, and this guide does not export data columns. Classic Data Map content, lineage and scan settings need their own export.
- The first decisions are one reading of Microsoft's transition guide, by object type. People decide object by object; the script only gives them a starting point and the counts.
- Dates are as of 2026-11-21. This guide is about Purview Data Governance, whose support ends on November 12, 2027; after that date the guide describes history.
Tested on
Tested on
- Platform
- DuckDB 1.5.6, run from Python 3.14.8
- Edition
- Open-source DuckDB library (
pip install duckdb) - Date
- 2026-11-21
- Records
- Generated: an invented Unified Catalog of three governance domains, fixed seed 274, in the shape of API version 2026-03-20-preview (specification at commit be93aca, read 2026-11-21). Not a run against a Purview tenant.
- Code
- inventory_catalog.py
- Results
- objects.csv, inventory.csv
When this is not enough
The inventory gives counts and a first decision by type. A real plan needs a decision for each object, an owner and a steward for each domain, a list of the reports and pipelines that read the catalog, and a switch-off date for each domain.
Need a decision for every object, not only for every object type?
Purview to OneLake Migration is in early access. The offering starts with the four-day Catalog Transition Assessment: one fixed fee of $12,000, and no commitment to anything else. You get an inventory of every object in your Purview governance catalog and a decision for each object. AI does the first pass, and your stewards sign every domain.