How to keep Purview Data Governance evidence after the switch-off
The short answer
Before Microsoft ends support for Purview Data Governance on November 12, 2027, save the last governance catalog export and check that every page was saved. A Python and DuckDB script writes the export as read-only Parquet files, each with a SHA-256 hash. The script answers ten audit questions only while every hash matches. Purview audit stays in Purview.
Why keep an archive
Microsoft announced on November 12, 2026 that Purview Data Governance is retiring: “Support for Purview Data Governance will end Nov. 12, 2027” (Microsoft Security blog, read 2026-11-21). Microsoft's transition guide (read 2026-11-21) lists evidence-retention requirements among the things to inventory before the move, and its advice for the glossary is to export and preserve terms and ownership.
After the switch-off, an auditor can still ask who owned a term, when a term expired or which data a term described. The answer has to come from something that cannot change: the last export of the governance catalog, kept read-only, with a way to show the files have not changed since they were saved.
What you will have at the end
- An archive: one Parquet file per object type (terms, links from terms to data assets, domains, data products, critical data elements, objectives, data assets, policies), each file made read-only.
- A manifest with each file's rows, size and SHA-256, a hash of the manifest itself, and one hash for all the export files the archive came from, the relationships files included.
- A verify step that recomputes every hash and reports any file that changed, went missing or was added.
- Ten audit questions in
audit_questions.sql, answered from the archive alone, and only after the verify step passes.
Step 1. Save the last export
Save the export as the guide How to inventory a Purview governance catalog before migrating shows (one folder per object type), and the links from terms to data assets as the guide How to export Purview glossary terms with their relationships shows (a relationships folder). Run both checks on the export before you archive it: an archive of an incomplete export only preserves the gap. The archive stops if a page of any object type is missing or the last page still points to a next one. The archive does not check the relationships folder; the export-check guide does, so run that guide's checks first.
Step 2. Archive, verify, ask
Download archive_evidence.py and audit_questions.sql into one folder, and install DuckDB (pip install duckdb). Then run three commands:
python3 archive_evidence.py archive inventory-20261121-093000 purview-archive-20261121
python3 archive_evidence.py verify purview-archive-20261121
python3 archive_evidence.py ask purview-archive-20261121
- archive writes the Parquet files, the manifest and the manifest's hash into a new, empty folder, then makes every file read-only. The script refuses a folder that already holds files, so an archive is written once.
- verify recomputes the SHA-256 of the manifest and of every Parquet file, counts each file's rows, and lists any file that changed, is missing, or is in the folder but not in the manifest.
- ask runs verify first and refuses to answer from an archive that is not intact. Then it runs each question in
audit_questions.sqland prints up to ten rows of each answer.
The questions are plain SQL over the archive's tables. Add your own: each question starts with a line -- Q<n>. <question>, followed by its query.
Run 1. Archive a generated export, then ask the ten questions
The script writes a generated export when you ask for one: three invented governance domains with eight terms each, links to invented data assets, and invented data products, critical data elements, objectives and policies, with a fixed seed.
python3 archive_evidence.py --make-demo demo
Wrote a generated export to demo: 24 terms in 3 domains, 48 term links, 12 data assets (seed 277)
python3 archive_evidence.py archive demo archive
Python 3.14.8 duckdb 1.5.6 Archived 8 export pages and 24 relationship files into 8 Parquet files in archive: businessdomains.parquet 3 rows terms.parquet 24 rows dataProducts.parquet 3 rows criticalDataElements.parquet 3 rows objectives.parquet 3 rows dataAssets.parquet 12 rows policies.parquet 2 rows term_links.parquet 48 rows Wrote manifest.csv and manifest.sha256; every file in archive is now read-only.
python3 archive_evidence.py ask archive
Python 3.14.8 duckdb 1.5.6 Verified 8 files against manifest.csv in archive RESULT: the archive is intact. Q1. How many glossary terms did each governance domain hold at the last export, by status? (9 rows) domain | status | terms Finance | DRAFT | 1 Finance | EXPIRED | 1 Finance | PUBLISHED | 6 Sales | DRAFT | 1 Sales | EXPIRED | 1 Sales | PUBLISHED | 6 Supply Chain | DRAFT | 1 Supply Chain | EXPIRED | 1 Supply Chain | PUBLISHED | 6 Q2. Which published terms had no owner? (3 rows) domain | term Finance | Finance term 6 Sales | Sales term 6 Supply Chain | Supply Chain term 6 Q3. Who owned the most published terms (Microsoft Entra object ids)? (5 rows) owner_id | published_terms 807d8acc-9359-4d9c-9fa1-595294dbad49 | 5 f193cfd9-4da5-49aa-b867-bb689ef3bc60 | 4 2fdfc10e-eb4b-49fb-96d3-1981e6c3abcc | 3 202dd20a-96a6-4d6a-82c0-7d98796b15b4 | 2 2be62dd0-0a2c-4b73-a994-1cac3786044f | 1 Q4. Which terms were changed last, and by whom? (10 rows) term | last_modified_at | last_modified_by Finance term 8 | 2026-08-15T10:00:00Z | 2fdfc10e-eb4b-49fb-96d3-1981e6c3abcc Sales term 8 | 2026-08-15T10:00:00Z | 202dd20a-96a6-4d6a-82c0-7d98796b15b4 Supply Chain term 8 | 2026-08-15T10:00:00Z | 807d8acc-9359-4d9c-9fa1-595294dbad49 Finance term 7 | 2026-07-15T10:00:00Z | 2fdfc10e-eb4b-49fb-96d3-1981e6c3abcc Sales term 7 | 2026-07-15T10:00:00Z | 2fdfc10e-eb4b-49fb-96d3-1981e6c3abcc Supply Chain term 7 | 2026-07-15T10:00:00Z | 807d8acc-9359-4d9c-9fa1-595294dbad49 Finance term 6 | 2026-06-15T10:00:00Z | 807d8acc-9359-4d9c-9fa1-595294dbad49 Sales term 6 | 2026-06-15T10:00:00Z | 2fdfc10e-eb4b-49fb-96d3-1981e6c3abcc Supply Chain term 6 | 2026-06-15T10:00:00Z | 202dd20a-96a6-4d6a-82c0-7d98796b15b4 Finance term 5 | 2026-05-15T10:00:00Z | f193cfd9-4da5-49aa-b867-bb689ef3bc60 Q5. Which terms had expired by the last export? (3 rows) domain | term | last_modified_at Finance | Finance term 8 | 2026-08-15T10:00:00Z Sales | Sales term 8 | 2026-08-15T10:00:00Z Supply Chain | Supply Chain term 8 | 2026-08-15T10:00:00Z Q6. Which data assets was each published term linked to? (36 rows) term | data_asset Finance term 1 | asset_02 Finance term 1 | asset_05 Finance term 2 | asset_05 Finance term 2 | asset_06 Finance term 3 | asset_01 Finance term 3 | asset_09 Finance term 4 | asset_01 Finance term 4 | asset_02 Finance term 5 | asset_06 Finance term 5 | asset_08 ... 26 more rows Q7. Which data assets were linked to more than one term? (12 rows) data_asset | terms asset_10 | 6 asset_12 | 6 asset_01 | 5 asset_02 | 5 asset_06 | 5 asset_05 | 4 asset_08 | 4 asset_09 | 4 asset_11 | 3 asset_03 | 2 ... 2 more rows Q8. Which data products existed, in which domain, and who owned each one? (3 rows) domain | data_product | status | owners Finance | Finance reporting | PUBLISHED | f193cfd9-4da5-49aa-b867-bb689ef3bc60 Sales | Sales reporting | PUBLISHED | 807d8acc-9359-4d9c-9fa1-595294dbad49 Supply Chain | Supply Chain reporting | PUBLISHED | f193cfd9-4da5-49aa-b867-bb689ef3bc60 Q9. Which access policies were recorded? (2 rows) policy | id Invented access policy 1 | e5f914ef-e91c-4a7b-8601-eeac2383fb68 Invented access policy 2 | 86e98513-c2ac-4db2-b948-1cdee1ea30d4 Q10. How many objects of each type does the archive hold? (8 rows) object_type | objects businessdomains | 3 criticalDataElements | 3 dataAssets | 12 dataProducts | 3 objectives | 3 policies | 2 term_links | 48 terms | 24
Run 2. Change one archived value, then verify
Run 2 copies the archive and changes one value in the copy, the way a careless fix would: one expired term is marked published, with the same number of rows as before. Then it verifies the copy, and asks the questions of the copy.
cp -r archive archive-copy
python3 archive_evidence.py --tamper archive-copy
Tampered with archive-copy/terms.parquet: one expired term marked PUBLISHED (24 rows, as before)
python3 archive_evidence.py verify archive-copy
Python 3.14.8 duckdb 1.5.6 Verified 8 files against manifest.csv in archive-copy FAIL terms.parquet changed after it was archived: its SHA-256 no longer matches RESULT: the archive is not intact (problems: 1).
python3 archive_evidence.py ask archive-copy
Python 3.14.8 duckdb 1.5.6 Verified 8 files against manifest.csv in archive-copy FAIL terms.parquet changed after it was archived: its SHA-256 no longer matches RESULT: the archive is not intact (problems: 1). CANNOT CONTINUE: the archive is not intact, so its answers are not evidence
The row count did not change, so only the hash shows the change. That is why the manifest records a SHA-256 for every file, and why ask refuses a damaged archive instead of answering from the changed data.
Run the archive on your own export
Save your last export into one folder, laid out the way the two guides in step 1 show. Run python3 archive_evidence.py archive with your export folder and a new, empty archive folder, then python3 archive_evidence.py verify and python3 archive_evidence.py ask with the archive folder. You do not edit the script.
We tested a run on a reader's own export: a second generated export with other names, upper-case ids, empty fields, a second page of terms and a second page of links. The archive verified as intact; then one byte was added to one file and a stray file was put in the folder, and verify reported both, and ask refused to answer.
Download archive_evidence.py · Download audit_questions.sql · Get the full guide (PDF)
Result from run 1: manifest.csv.
Get the full guide as a PDF
We'll email you a link to it. Please use your work email.
Thanks, . We're sending the link to now. If it isn't there in a few minutes, check your spam or junk folder, or email info@datameaning.com.
Limits
- The runs on this page use generated records, not a real Purview tenant. The field names come from Microsoft's API specification and reference pages, read on 2026-11-21.
- A hash proves a file has not changed since the manifest was written. A hash does not prove the export was complete or correct when it was saved: run the export checks first, and keep the manifest's hash somewhere the archive's owners cannot edit, for example in your records system.
- Read-only file permissions stop accidents, not an administrator. Store the archive where your retention rules apply, for example storage with an immutability policy.
- The archive holds what the Unified Catalog API returns. Classic Data Map content, lineage, scan settings and audit logs need their own export. Purview audit stays in Purview.
- A question that reads an object type your export does not hold prints “cannot run” instead of an answer.
- Parquet files written by different DuckDB versions can differ byte for byte, so compare hashes only against the manifest written with the archive.
- Dates are as of 2026-11-21. This guide is about Purview Data Governance, whose support ends on November 12, 2027.
Tested on
Tested on
- Platform
- DuckDB 1.5.6, run from Python 3.14.8
- Edition
- Open-source DuckDB library (
pip install duckdb) - Date
- 2026-11-21
- Records
- Generated: an invented Unified Catalog of three governance domains, fixed seed 277, in the shape of API version 2026-03-20-preview (specification at commit be93aca, read 2026-11-21). Not a run against a Purview tenant.
- Code
- archive_evidence.py, audit_questions.sql
- Results
- manifest.csv
When this is not enough
An archive of the catalog is one part of the evidence. An audit can also ask which reports read the catalog, who approved each switch-off, and what the archive could not hold, such as lineage across systems and the history of each change.
Need an evidence archive for your whole Purview governance catalog?
Purview to OneLake Migration is in early access. The offering starts with the four-day Catalog Transition Assessment: one fixed fee of $12,000, and no commitment to anything else. You get an inventory of every object in your Purview governance catalog and a decision for each object. AI does the first pass, and your stewards sign every domain.