How to export Purview scan rules and custom classifications
The short answer
Call the Purview Scanning API and save four lists, one page per file, following every nextLink: classification rules, scan rule sets, data sources and scans. Add samples.csv with values each custom rule should and should not match. One Python and DuckDB script runs eight checks, including missing pages and rule sets that name unsaved rules, and tests every rule's patterns against the test values.
Why save the scan settings first
Microsoft announced on November 12, 2026 that Purview Data Governance is retiring: “Support for Purview Data Governance will end Nov. 12, 2027” (Microsoft Security blog, read 2026-11-21). Microsoft's classification best practices page (read 2026-11-21) says the classic data governance capabilities stay in maintenance mode through November 12, 2027, and that Purview's data security and compliance capabilities are not affected.
Microsoft's transition guide (read 2026-11-21) tells you to capture registered sources, scans and network runtimes, keeps classification and sensitive-data protection in Purview, and says data connectivity in Fabric is not enterprise metadata scanning: sources Fabric does not cover need a bridge or partner approach. Whatever the destination, the custom classification rules and scan rule sets hold your decisions about what counts as an employee id or a policy number in your data. Save the rules and rule sets, and test each rule, before the scans stop.
What you will have at the end
- The scan settings as saved files: classification rules, scan rule sets, data sources and scans, in the Scanning API's own answers, one file per page.
- A rule list,
rules.csv: one row per custom classification rule, with its patterns, threshold, the rule sets that include the rule, the scans that use those rule sets and how many test values ran against the rule. - Eight checks, each one PASS or FAIL, with every finding in
findings.csv.
What a custom classification rule holds
Microsoft's classification best practices page describes the parts: a data pattern, a regular expression for the values to classify; an optional column pattern, a regular expression for the column name, with several names separated by the pipe character; and a minimum match threshold from 1% to 100%. The column pattern is checked as an AND condition with the data pattern, and the threshold is turned off when a rule has more than one data pattern. The script tests each of your test values the same way: a value matches when a data pattern finds the value and, if the rule has column patterns, a column pattern finds the column name.
Step 1. Save the scan settings through the Scanning API
The Scanning REST API (api-version 2023-09-01) lists classification rules at /scan/classificationrules, scan rule sets at /scan/scanrulesets, data sources at /scan/datasources and each data source's scans under /scan/datasources/{dataSourceName}/scans. Each list answers a page with a nextLink to the next page (List Classification Rules reference, read 2026-11-21). The commands below follow every nextLink and save each page as its own file.
We did not run these commands. We have no tenant to run them against, so they are written from Microsoft's Scanning REST reference, and the runs further down use generated records in the same shape. Set ENDPOINT to your Purview account's endpoint and sign in with the Azure CLI first.
# Not run by us: written from Microsoft's Scanning REST reference, api-version 2023-09-01 (read 2026-11-21).
# Needs the Azure CLI (az login) and an account that can read the scan settings.
ENDPOINT="https://<your-account>.purview.azure.com" # your Purview account's endpoint
OUT="scan-settings-$(date +%Y%m%d-%H%M%S)"; mkdir -p "$OUT"
TOKEN=$(az account get-access-token --resource https://purview.azure.net --query accessToken -o tsv)
get_pages() { # every page of one list, given its URL and a folder; the next page's URL comes from nextLink
mkdir -p "$OUT/$2"; URL="$1"; N=1
while [ -n "$URL" ]; do
F="$OUT/$2/page-$(printf %03d $N).json"
curl -sf -H "Authorization: Bearer $TOKEN" "$URL" -o "$F" || { echo "FAILED: $F was not saved"; rm -f "$F"; return; }
URL=$(python3 -c "import json,sys; print(json.load(open(sys.argv[1])).get('nextLink') or '')" "$F"); N=$((N + 1))
done
}
API="$ENDPOINT/scan"; V="api-version=2023-09-01"
get_pages "$API/classificationrules?$V" classificationrules
get_pages "$API/scanrulesets?$V" scanrulesets
get_pages "$API/datasources?$V" datasources
python3 -c "import json,glob,sys; [print(d['name']) for f in sorted(glob.glob(sys.argv[1] + '/datasources/page-*.json')) for d in json.load(open(f))['value']]" "$OUT" |
while IFS= read -r DS; do # a name can hold spaces: quote it in the folder, encode it in the URL
ENC=$(python3 -c "import sys, urllib.parse; print(urllib.parse.quote(sys.argv[1], safe=''))" "$DS")
get_pages "$API/datasources/$ENC/scans?$V" "scans/$DS"
done
echo "Saved to $OUT. Now write $OUT/samples.csv with your test values."
Then write samples.csv in the same folder, with the columns rule, column_name, value and expect (match or no match): for each rule, at least one value the rule should classify and one it should not. Use invented values in the same format as your data, not real ones.
Step 2. Check the settings and test every rule
Download check_scan_rules.py, install DuckDB (pip install duckdb) and run the script on the folder the commands wrote:
python3 check_scan_rules.py scan-settings-20261121-093000
| Check | What a failure means |
|---|---|
| Every page of every list was saved | A list stops at a page that still points to a next one, or a page number is missing, so the export is incomplete. |
| Every scan's custom rule set is in the export | A scan names a custom rule set that was not saved, so what the scan classified cannot be rebuilt. |
| Every custom classification rule a rule set includes is in the export | A rule set names a classification rule that was not saved. |
| No rule set includes a disabled classification rule | A rule set still lists a rule that no longer runs; decide whether the rule moves. |
| Every enabled custom classification rule is in at least one rule set | An enabled rule that no scan can use; a candidate to retire rather than move. |
| Every pattern can be read as a regular expression | A pattern Python cannot read; fix the pattern, or rewrite the pattern for the engine the rule moves to. |
| Every custom classification rule in use has a test value that should match | Nothing proves the rule still classifies what the rule was written for. |
| Every test value gives the result samples.csv expects | The rule and the test value disagree: the rule is wrong, or the expected result is. |
Run 1. Generated scan settings, clean
The script writes a generated export when you ask for one: five custom classification rules, two custom scan rule sets, three data sources, four scans and ten test values, the same every run.
python3 check_scan_rules.py --make-demo demo
Wrote a generated Scanning API export to demo: 5 custom and 3 system classification rules, 2 scan rule sets, 3 data sources, 4 scans, 10 test values
python3 check_scan_rules.py demo
Python 3.14.8 duckdb 1.5.6 Export: demo Read 5 custom classification rules (3 system), 2 scan rule sets, 3 data sources, 4 scans Custom classification rule Status Rule sets Scans Tests COST_CENTER Enabled 1 2 2 EMPLOYEE_ID Enabled 1 2 2 POLICY_NUMBER Enabled 1 1 2 PROJECT_CODE Enabled 1 1 2 SUPPLIER_CODE Enabled 1 2 2 Scans by rule set type: Custom 3, System 1 Test values: 10 run, 10 gave the expected result Checks PASS Every page of every list was saved PASS Every scan's custom rule set is in the export PASS Every custom classification rule a rule set includes is in the export PASS No rule set includes a disabled classification rule PASS Every enabled custom classification rule is in at least one rule set PASS Every pattern can be read as a regular expression PASS Every custom classification rule in use has a test value that should match PASS Every test value gives the result samples.csv expects Wrote rules.csv (5 rules) and findings.csv (0 rows) to . RESULT: 0 findings. Every rule is saved, in use and tested.
How to read it
Each row is one custom classification rule: how many rule sets include the rule, how many scans use those rule sets and how many test values the rule ran. All eight checks pass, so every rule is saved, used by a scan and tested. The Power BI scan uses a system rule set, so no custom rule runs in that scan.
Run 2. The same settings with eight planted problems
python3 check_scan_rules.py --make-demo demo-planted --plant-errors
Wrote a generated Scanning API export to demo-planted: 6 custom and 3 system classification rules, 2 scan rule sets, 3 data sources, 5 scans, 8 test values Planted one error for each check: BAD_PATTERN, DISABLED_RULE_IN_USE, NO_SAMPLE, PAGES_MISSING, SAMPLE_FAILS, UNKNOWN_RULE, UNKNOWN_RULESET, UNUSED_RULE
python3 check_scan_rules.py demo-planted --tag planted
Python 3.14.8 duckdb 1.5.6
Export: demo-planted
Read 6 custom classification rules (3 system), 2 scan rule sets, 3 data sources, 5 scans
Custom classification rule Status Rule sets Scans Tests
COST_CENTER Disabled 1 2 2
EMPLOYEE_ID Enabled 1 2 2
POLICY_NUMBER Enabled 1 1 0
PROJECT_CODE Enabled 1 1 0
SUPPLIER_CODE Enabled 1 2 2
TAX_REGION Enabled 0 0 0
Scans by rule set type: Custom 4, System 1
Test values: 6 run, 5 gave the expected result
Checks
FAIL Every page of every list was saved (1)
FAIL Every scan's custom rule set is in the export (1)
FAIL Every custom classification rule a rule set includes is in the export (1)
FAIL No rule set includes a disabled classification rule (1)
FAIL Every enabled custom classification rule is in at least one rule set (1)
FAIL Every pattern can be read as a regular expression (1)
FAIL Every custom classification rule in use has a test value that should match (1)
FAIL Every test value gives the result samples.csv expects (1)
PAGES_MISSING scans/pbi-tenant: page-001.json still points to a next page
UNKNOWN_RULESET scan lake-claims/curated-zone: uses custom rule set lake-claims-rules-v2, which is not in the export
UNKNOWN_RULE rule set lake-claims-rules: includes CLAIM_ID, which is not in the export
DISABLED_RULE_IN_USE rule set sql-finance-rules: includes COST_CENTER, which is disabled
UNUSED_RULE rule TAX_REGION: enabled, but in no scan rule set
BAD_PATTERN rule PROJECT_CODE: data pattern '^PRJ-([A-Z]{3}-\d{3}$': missing ), unterminated subpattern at position 5
SAMPLE_FAILS rule EMPLOYEE_ID: 'E-104233' in column 'employee_id': expected match, got no match
NO_SAMPLE rule POLICY_NUMBER: in use, with no test value that should match
Wrote rules-planted.csv (6 rules) and findings-planted.csv (8 rows) to .
RESULT: 8 findings. Fix them in Purview or in samples.csv, export again and re-run.
| Check | The problem planted in run 2 |
|---|---|
| Pages | The Power BI data source's scans page points to a next page that was never saved. |
| Rule sets | A curated-zone scan names the rule set lake-claims-rules-v2, which is not in the export. |
| Rules | The lake-claims-rules rule set includes CLAIM_ID, which is not in the export. |
| Disabled rules | COST_CENTER is disabled while sql-finance-rules still includes the rule. |
| Unused rules | A new TAX_REGION rule, enabled and in no rule set. |
| Patterns | PROJECT_CODE's data pattern with an unclosed parenthesis. |
| Test values | POLICY_NUMBER's test values removed from samples.csv. |
| Results | An employee id written as E-104233 and expected to match. |
Run the check on your own scan settings
Run the commands in step 1 with ENDPOINT set to your Purview account's endpoint. The commands write a folder named scan-settings- followed by the date and time, for example scan-settings-20261121-093000, holding one folder of pages each for classificationrules, scanrulesets and datasources, and a scans folder with one folder of pages for each data source. Write samples.csv in that same folder, with the columns rule, column_name, value and expect. Then run python3 check_scan_rules.py followed by the folder's name. The script writes rules.csv and findings.csv to the script's own folder; add --out followed by a folder name to write the two files somewhere else. You do not edit the script.
We also tested a run on a second generated export, saved the way you would save your own: other rule names, three pages of classification rules, a rule with two data patterns, column names separated by the pipe character, a data source with no scans, a samples.csv saved from a spreadsheet with an extra column, and one planted problem for each check. All eight checks failed as planted, and nothing else was flagged.
Download check_scan_rules.py · Get the full guide (PDF)
Result from run 1: rules.csv.
Get the full guide as a PDF
We'll email you a link to it. Please use your work email.
Thanks, . We're sending the link to now. If it isn't there in a few minutes, check your spam or junk folder, or email info@datameaning.com.
Limits
- The runs on this page use generated records, not a Purview tenant, and the commands in step 1 were not run. The field names come from Microsoft's Scanning REST specification, api-version 2023-09-01, read on 2026-11-21.
- The script tests patterns with Python's regular expressions. Microsoft's page does not name the engine Purview uses (its pipe example links to the .NET regular expression reference), and rare syntax can behave differently, so test the rule again on the platform the rule moves to.
- A test value checks the patterns, not the minimum match threshold, which Purview applies to the share of matching values in a column.
- The scan pages name the credential and integration runtime each scan uses, but do not hold the credentials themselves, the runtimes' setup or the scan schedules. Microsoft's transition guide lists network runtimes among the things to capture, so record the runtimes, credentials and schedules separately.
- The script tests regular expression patterns only. Microsoft's page also describes dictionary rules, built from an uploaded file; record those rules separately.
- Dates are as of 2026-11-21. This guide is about Purview Data Governance, whose support ends on November 12, 2027.
Tested on
Tested on
- Platform
- DuckDB 1.5.6, run from Python 3.14.8
- Edition
- Open-source DuckDB library (
pip install duckdb) - Date
- 2026-11-21
- Records
- Generated: invented scan settings with five custom classification rules, two scan rule sets, three data sources and four scans, the same on every run, in the shape of the Scanning API's answers (api-version 2023-09-01). Not a run against a tenant.
- Code
- check_scan_rules.py
- Results
- rules.csv
When this is not enough
The check shows which rules are saved, used and tested. Each rule then needs a home after the switch-off: a sensitivity label or data loss prevention policy in Purview, a rule in another scanner for sources Fabric does not cover, or retirement. Sensitivity labels stay in Purview, so check each item's label after the move as well.
Moving your whole Purview governance catalog, not only the scan rules?
Purview to OneLake Migration is in early access. The offering starts with the four-day Catalog Transition Assessment: one fixed fee of $12,000, and no commitment to anything else. You get an inventory of every object in your Purview governance catalog and a decision for each object. AI does the first pass, and your stewards sign every domain.