Guide · Purview to OneLake Migration

How to keep Purview Data Governance evidence after the switch-off

Updated 2026-11-21Python and DuckDBRuns shown on generated records, PDF by email

The short answer

Before Microsoft ends support for Purview Data Governance on November 12, 2027, save the last governance catalog export and check that every page was saved. A Python and DuckDB script writes the export as read-only Parquet files, each with a SHA-256 hash. The script answers ten audit questions only while every hash matches. Purview audit stays in Purview.

Why keep an archive

Microsoft announced on November 12, 2026 that Purview Data Governance is retiring: “Support for Purview Data Governance will end Nov. 12, 2027” (Microsoft Security blog, read 2026-11-21). Microsoft's transition guide (read 2026-11-21) lists evidence-retention requirements among the things to inventory before the move, and its advice for the glossary is to export and preserve terms and ownership.

After the switch-off, an auditor can still ask who owned a term, when a term expired or which data a term described. The answer has to come from something that cannot change: the last export of the governance catalog, kept read-only, with a way to show the files have not changed since they were saved.

What you will have at the end

  • An archive: one Parquet file per object type (terms, links from terms to data assets, domains, data products, critical data elements, objectives, data assets, policies), each file made read-only.
  • A manifest with each file's rows, size and SHA-256, a hash of the manifest itself, and one hash for all the export files the archive came from, the relationships files included.
  • A verify step that recomputes every hash and reports any file that changed, went missing or was added.
  • Ten audit questions in audit_questions.sql, answered from the archive alone, and only after the verify step passes.

Step 1. Save the last export

Save the export as the guide How to inventory a Purview governance catalog before migrating shows (one folder per object type), and the links from terms to data assets as the guide How to export Purview glossary terms with their relationships shows (a relationships folder). Run both checks on the export before you archive it: an archive of an incomplete export only preserves the gap. The archive stops if a page of any object type is missing or the last page still points to a next one. The archive does not check the relationships folder; the export-check guide does, so run that guide's checks first.

Step 2. Archive, verify, ask

Download archive_evidence.py and audit_questions.sql into one folder, and install DuckDB (pip install duckdb). Then run three commands:

python3 archive_evidence.py archive inventory-20261121-093000 purview-archive-20261121
python3 archive_evidence.py verify purview-archive-20261121
python3 archive_evidence.py ask purview-archive-20261121
  • archive writes the Parquet files, the manifest and the manifest's hash into a new, empty folder, then makes every file read-only. The script refuses a folder that already holds files, so an archive is written once.
  • verify recomputes the SHA-256 of the manifest and of every Parquet file, counts each file's rows, and lists any file that changed, is missing, or is in the folder but not in the manifest.
  • ask runs verify first and refuses to answer from an archive that is not intact. Then it runs each question in audit_questions.sql and prints up to ten rows of each answer.

The questions are plain SQL over the archive's tables. Add your own: each question starts with a line -- Q<n>. <question>, followed by its query.

Run 1. Archive a generated export, then ask the ten questions

The script writes a generated export when you ask for one: three invented governance domains with eight terms each, links to invented data assets, and invented data products, critical data elements, objectives and policies, with a fixed seed.

python3 archive_evidence.py --make-demo demo
Wrote a generated export to demo: 24 terms in 3 domains, 48 term links, 12 data assets (seed 277)
python3 archive_evidence.py archive demo archive
Python 3.14.8 duckdb 1.5.6
Archived 8 export pages and 24 relationship files into 8 Parquet files in archive:
  businessdomains.parquet             3 rows
  terms.parquet                      24 rows
  dataProducts.parquet                3 rows
  criticalDataElements.parquet        3 rows
  objectives.parquet                  3 rows
  dataAssets.parquet                 12 rows
  policies.parquet                    2 rows
  term_links.parquet                 48 rows
Wrote manifest.csv and manifest.sha256; every file in archive is now read-only.
python3 archive_evidence.py ask archive
Python 3.14.8 duckdb 1.5.6
Verified 8 files against manifest.csv in archive
RESULT: the archive is intact.

Q1. How many glossary terms did each governance domain hold at the last export, by status?  (9 rows)
  domain | status | terms
  Finance | DRAFT | 1
  Finance | EXPIRED | 1
  Finance | PUBLISHED | 6
  Sales | DRAFT | 1
  Sales | EXPIRED | 1
  Sales | PUBLISHED | 6
  Supply Chain | DRAFT | 1
  Supply Chain | EXPIRED | 1
  Supply Chain | PUBLISHED | 6

Q2. Which published terms had no owner?  (3 rows)
  domain | term
  Finance | Finance term 6
  Sales | Sales term 6
  Supply Chain | Supply Chain term 6

Q3. Who owned the most published terms (Microsoft Entra object ids)?  (5 rows)
  owner_id | published_terms
  807d8acc-9359-4d9c-9fa1-595294dbad49 | 5
  f193cfd9-4da5-49aa-b867-bb689ef3bc60 | 4
  2fdfc10e-eb4b-49fb-96d3-1981e6c3abcc | 3
  202dd20a-96a6-4d6a-82c0-7d98796b15b4 | 2
  2be62dd0-0a2c-4b73-a994-1cac3786044f | 1

Q4. Which terms were changed last, and by whom?  (10 rows)
  term | last_modified_at | last_modified_by
  Finance term 8 | 2026-08-15T10:00:00Z | 2fdfc10e-eb4b-49fb-96d3-1981e6c3abcc
  Sales term 8 | 2026-08-15T10:00:00Z | 202dd20a-96a6-4d6a-82c0-7d98796b15b4
  Supply Chain term 8 | 2026-08-15T10:00:00Z | 807d8acc-9359-4d9c-9fa1-595294dbad49
  Finance term 7 | 2026-07-15T10:00:00Z | 2fdfc10e-eb4b-49fb-96d3-1981e6c3abcc
  Sales term 7 | 2026-07-15T10:00:00Z | 2fdfc10e-eb4b-49fb-96d3-1981e6c3abcc
  Supply Chain term 7 | 2026-07-15T10:00:00Z | 807d8acc-9359-4d9c-9fa1-595294dbad49
  Finance term 6 | 2026-06-15T10:00:00Z | 807d8acc-9359-4d9c-9fa1-595294dbad49
  Sales term 6 | 2026-06-15T10:00:00Z | 2fdfc10e-eb4b-49fb-96d3-1981e6c3abcc
  Supply Chain term 6 | 2026-06-15T10:00:00Z | 202dd20a-96a6-4d6a-82c0-7d98796b15b4
  Finance term 5 | 2026-05-15T10:00:00Z | f193cfd9-4da5-49aa-b867-bb689ef3bc60

Q5. Which terms had expired by the last export?  (3 rows)
  domain | term | last_modified_at
  Finance | Finance term 8 | 2026-08-15T10:00:00Z
  Sales | Sales term 8 | 2026-08-15T10:00:00Z
  Supply Chain | Supply Chain term 8 | 2026-08-15T10:00:00Z

Q6. Which data assets was each published term linked to?  (36 rows)
  term | data_asset
  Finance term 1 | asset_02
  Finance term 1 | asset_05
  Finance term 2 | asset_05
  Finance term 2 | asset_06
  Finance term 3 | asset_01
  Finance term 3 | asset_09
  Finance term 4 | asset_01
  Finance term 4 | asset_02
  Finance term 5 | asset_06
  Finance term 5 | asset_08
  ... 26 more rows

Q7. Which data assets were linked to more than one term?  (12 rows)
  data_asset | terms
  asset_10 | 6
  asset_12 | 6
  asset_01 | 5
  asset_02 | 5
  asset_06 | 5
  asset_05 | 4
  asset_08 | 4
  asset_09 | 4
  asset_11 | 3
  asset_03 | 2
  ... 2 more rows

Q8. Which data products existed, in which domain, and who owned each one?  (3 rows)
  domain | data_product | status | owners
  Finance | Finance reporting | PUBLISHED | f193cfd9-4da5-49aa-b867-bb689ef3bc60
  Sales | Sales reporting | PUBLISHED | 807d8acc-9359-4d9c-9fa1-595294dbad49
  Supply Chain | Supply Chain reporting | PUBLISHED | f193cfd9-4da5-49aa-b867-bb689ef3bc60

Q9. Which access policies were recorded?  (2 rows)
  policy | id
  Invented access policy 1 | e5f914ef-e91c-4a7b-8601-eeac2383fb68
  Invented access policy 2 | 86e98513-c2ac-4db2-b948-1cdee1ea30d4

Q10. How many objects of each type does the archive hold?  (8 rows)
  object_type | objects
  businessdomains | 3
  criticalDataElements | 3
  dataAssets | 12
  dataProducts | 3
  objectives | 3
  policies | 2
  term_links | 48
  terms | 24

Run 2. Change one archived value, then verify

Run 2 copies the archive and changes one value in the copy, the way a careless fix would: one expired term is marked published, with the same number of rows as before. Then it verifies the copy, and asks the questions of the copy.

cp -r archive archive-copy
python3 archive_evidence.py --tamper archive-copy
Tampered with archive-copy/terms.parquet: one expired term marked PUBLISHED (24 rows, as before)
python3 archive_evidence.py verify archive-copy
Python 3.14.8 duckdb 1.5.6
Verified 8 files against manifest.csv in archive-copy
  FAIL  terms.parquet changed after it was archived: its SHA-256 no longer matches
RESULT: the archive is not intact (problems: 1).
python3 archive_evidence.py ask archive-copy
Python 3.14.8 duckdb 1.5.6
Verified 8 files against manifest.csv in archive-copy
  FAIL  terms.parquet changed after it was archived: its SHA-256 no longer matches
RESULT: the archive is not intact (problems: 1).
CANNOT CONTINUE: the archive is not intact, so its answers are not evidence

The row count did not change, so only the hash shows the change. That is why the manifest records a SHA-256 for every file, and why ask refuses a damaged archive instead of answering from the changed data.

Run the archive on your own export

Save your last export into one folder, laid out the way the two guides in step 1 show. Run python3 archive_evidence.py archive with your export folder and a new, empty archive folder, then python3 archive_evidence.py verify and python3 archive_evidence.py ask with the archive folder. You do not edit the script.

We tested a run on a reader's own export: a second generated export with other names, upper-case ids, empty fields, a second page of terms and a second page of links. The archive verified as intact; then one byte was added to one file and a stray file was put in the folder, and verify reported both, and ask refused to answer.

Download archive_evidence.py · Download audit_questions.sql · Get the full guide (PDF)

Result from run 1: manifest.csv.

Get the full guide as a PDF

We'll email you a link to it. Please use your work email.

Limits

  • The runs on this page use generated records, not a real Purview tenant. The field names come from Microsoft's API specification and reference pages, read on 2026-11-21.
  • A hash proves a file has not changed since the manifest was written. A hash does not prove the export was complete or correct when it was saved: run the export checks first, and keep the manifest's hash somewhere the archive's owners cannot edit, for example in your records system.
  • Read-only file permissions stop accidents, not an administrator. Store the archive where your retention rules apply, for example storage with an immutability policy.
  • The archive holds what the Unified Catalog API returns. Classic Data Map content, lineage, scan settings and audit logs need their own export. Purview audit stays in Purview.
  • A question that reads an object type your export does not hold prints “cannot run” instead of an answer.
  • Parquet files written by different DuckDB versions can differ byte for byte, so compare hashes only against the manifest written with the archive.
  • Dates are as of 2026-11-21. This guide is about Purview Data Governance, whose support ends on November 12, 2027.

Tested on

Tested on

Platform
DuckDB 1.5.6, run from Python 3.14.8
Edition
Open-source DuckDB library (pip install duckdb)
Date
2026-11-21
Records
Generated: an invented Unified Catalog of three governance domains, fixed seed 277, in the shape of API version 2026-03-20-preview (specification at commit be93aca, read 2026-11-21). Not a run against a Purview tenant.
Code
archive_evidence.py, audit_questions.sql
Results
manifest.csv

When this is not enough

An archive of the catalog is one part of the evidence. An audit can also ask which reports read the catalog, who approved each switch-off, and what the archive could not hold, such as lineage across systems and the history of each change.

Need an evidence archive for your whole Purview governance catalog?

Purview to OneLake Migration is in early access. The offering starts with the four-day Catalog Transition Assessment: one fixed fee of $12,000, and no commitment to anything else. You get an inventory of every object in your Purview governance catalog and a decision for each object. AI does the first pass, and your stewards sign every domain.

See Purview to OneLake Migration