Guide · Purview to OneLake Migration

How to export Purview scan rules and custom classifications

Updated 2026-11-21Python and DuckDBRuns shown on generated records, PDF by email

The short answer

Call the Purview Scanning API and save four lists, one page per file, following every nextLink: classification rules, scan rule sets, data sources and scans. Add samples.csv with values each custom rule should and should not match. One Python and DuckDB script runs eight checks, including missing pages and rule sets that name unsaved rules, and tests every rule's patterns against the test values.

Why save the scan settings first

Microsoft announced on November 12, 2026 that Purview Data Governance is retiring: “Support for Purview Data Governance will end Nov. 12, 2027” (Microsoft Security blog, read 2026-11-21). Microsoft's classification best practices page (read 2026-11-21) says the classic data governance capabilities stay in maintenance mode through November 12, 2027, and that Purview's data security and compliance capabilities are not affected.

Microsoft's transition guide (read 2026-11-21) tells you to capture registered sources, scans and network runtimes, keeps classification and sensitive-data protection in Purview, and says data connectivity in Fabric is not enterprise metadata scanning: sources Fabric does not cover need a bridge or partner approach. Whatever the destination, the custom classification rules and scan rule sets hold your decisions about what counts as an employee id or a policy number in your data. Save the rules and rule sets, and test each rule, before the scans stop.

What you will have at the end

  • The scan settings as saved files: classification rules, scan rule sets, data sources and scans, in the Scanning API's own answers, one file per page.
  • A rule list, rules.csv: one row per custom classification rule, with its patterns, threshold, the rule sets that include the rule, the scans that use those rule sets and how many test values ran against the rule.
  • Eight checks, each one PASS or FAIL, with every finding in findings.csv.

What a custom classification rule holds

Microsoft's classification best practices page describes the parts: a data pattern, a regular expression for the values to classify; an optional column pattern, a regular expression for the column name, with several names separated by the pipe character; and a minimum match threshold from 1% to 100%. The column pattern is checked as an AND condition with the data pattern, and the threshold is turned off when a rule has more than one data pattern. The script tests each of your test values the same way: a value matches when a data pattern finds the value and, if the rule has column patterns, a column pattern finds the column name.

Step 1. Save the scan settings through the Scanning API

The Scanning REST API (api-version 2023-09-01) lists classification rules at /scan/classificationrules, scan rule sets at /scan/scanrulesets, data sources at /scan/datasources and each data source's scans under /scan/datasources/{dataSourceName}/scans. Each list answers a page with a nextLink to the next page (List Classification Rules reference, read 2026-11-21). The commands below follow every nextLink and save each page as its own file.

We did not run these commands. We have no tenant to run them against, so they are written from Microsoft's Scanning REST reference, and the runs further down use generated records in the same shape. Set ENDPOINT to your Purview account's endpoint and sign in with the Azure CLI first.

# Not run by us: written from Microsoft's Scanning REST reference, api-version 2023-09-01 (read 2026-11-21).
# Needs the Azure CLI (az login) and an account that can read the scan settings.
ENDPOINT="https://<your-account>.purview.azure.com"   # your Purview account's endpoint
OUT="scan-settings-$(date +%Y%m%d-%H%M%S)"; mkdir -p "$OUT"
TOKEN=$(az account get-access-token --resource https://purview.azure.net --query accessToken -o tsv)
get_pages() {   # every page of one list, given its URL and a folder; the next page's URL comes from nextLink
  mkdir -p "$OUT/$2"; URL="$1"; N=1
  while [ -n "$URL" ]; do
    F="$OUT/$2/page-$(printf %03d $N).json"
    curl -sf -H "Authorization: Bearer $TOKEN" "$URL" -o "$F" || { echo "FAILED: $F was not saved"; rm -f "$F"; return; }
    URL=$(python3 -c "import json,sys; print(json.load(open(sys.argv[1])).get('nextLink') or '')" "$F"); N=$((N + 1))
  done
}
API="$ENDPOINT/scan"; V="api-version=2023-09-01"
get_pages "$API/classificationrules?$V" classificationrules
get_pages "$API/scanrulesets?$V" scanrulesets
get_pages "$API/datasources?$V" datasources
python3 -c "import json,glob,sys; [print(d['name']) for f in sorted(glob.glob(sys.argv[1] + '/datasources/page-*.json')) for d in json.load(open(f))['value']]" "$OUT" |
while IFS= read -r DS; do   # a name can hold spaces: quote it in the folder, encode it in the URL
  ENC=$(python3 -c "import sys, urllib.parse; print(urllib.parse.quote(sys.argv[1], safe=''))" "$DS")
  get_pages "$API/datasources/$ENC/scans?$V" "scans/$DS"
done
echo "Saved to $OUT. Now write $OUT/samples.csv with your test values."

Then write samples.csv in the same folder, with the columns rule, column_name, value and expect (match or no match): for each rule, at least one value the rule should classify and one it should not. Use invented values in the same format as your data, not real ones.

Step 2. Check the settings and test every rule

Download check_scan_rules.py, install DuckDB (pip install duckdb) and run the script on the folder the commands wrote:

python3 check_scan_rules.py scan-settings-20261121-093000
CheckWhat a failure means
Every page of every list was savedA list stops at a page that still points to a next one, or a page number is missing, so the export is incomplete.
Every scan's custom rule set is in the exportA scan names a custom rule set that was not saved, so what the scan classified cannot be rebuilt.
Every custom classification rule a rule set includes is in the exportA rule set names a classification rule that was not saved.
No rule set includes a disabled classification ruleA rule set still lists a rule that no longer runs; decide whether the rule moves.
Every enabled custom classification rule is in at least one rule setAn enabled rule that no scan can use; a candidate to retire rather than move.
Every pattern can be read as a regular expressionA pattern Python cannot read; fix the pattern, or rewrite the pattern for the engine the rule moves to.
Every custom classification rule in use has a test value that should matchNothing proves the rule still classifies what the rule was written for.
Every test value gives the result samples.csv expectsThe rule and the test value disagree: the rule is wrong, or the expected result is.

Run 1. Generated scan settings, clean

The script writes a generated export when you ask for one: five custom classification rules, two custom scan rule sets, three data sources, four scans and ten test values, the same every run.

python3 check_scan_rules.py --make-demo demo
Wrote a generated Scanning API export to demo: 5 custom and 3 system classification rules, 2 scan rule sets, 3 data sources, 4 scans, 10 test values
python3 check_scan_rules.py demo
Python 3.14.8 duckdb 1.5.6
Export: demo
Read 5 custom classification rules (3 system), 2 scan rule sets, 3 data sources, 4 scans

  Custom classification rule   Status    Rule sets  Scans  Tests
  COST_CENTER                  Enabled           1      2      2
  EMPLOYEE_ID                  Enabled           1      2      2
  POLICY_NUMBER                Enabled           1      1      2
  PROJECT_CODE                 Enabled           1      1      2
  SUPPLIER_CODE                Enabled           1      2      2
Scans by rule set type: Custom 3, System 1
Test values: 10 run, 10 gave the expected result

Checks
  PASS  Every page of every list was saved
  PASS  Every scan's custom rule set is in the export
  PASS  Every custom classification rule a rule set includes is in the export
  PASS  No rule set includes a disabled classification rule
  PASS  Every enabled custom classification rule is in at least one rule set
  PASS  Every pattern can be read as a regular expression
  PASS  Every custom classification rule in use has a test value that should match
  PASS  Every test value gives the result samples.csv expects

Wrote rules.csv (5 rules) and findings.csv (0 rows) to .
RESULT: 0 findings. Every rule is saved, in use and tested.

How to read it

Each row is one custom classification rule: how many rule sets include the rule, how many scans use those rule sets and how many test values the rule ran. All eight checks pass, so every rule is saved, used by a scan and tested. The Power BI scan uses a system rule set, so no custom rule runs in that scan.

Run 2. The same settings with eight planted problems

python3 check_scan_rules.py --make-demo demo-planted --plant-errors
Wrote a generated Scanning API export to demo-planted: 6 custom and 3 system classification rules, 2 scan rule sets, 3 data sources, 5 scans, 8 test values
Planted one error for each check: BAD_PATTERN, DISABLED_RULE_IN_USE, NO_SAMPLE, PAGES_MISSING, SAMPLE_FAILS, UNKNOWN_RULE, UNKNOWN_RULESET, UNUSED_RULE
python3 check_scan_rules.py demo-planted --tag planted
Python 3.14.8 duckdb 1.5.6
Export: demo-planted
Read 6 custom classification rules (3 system), 2 scan rule sets, 3 data sources, 5 scans

  Custom classification rule   Status    Rule sets  Scans  Tests
  COST_CENTER                  Disabled          1      2      2
  EMPLOYEE_ID                  Enabled           1      2      2
  POLICY_NUMBER                Enabled           1      1      0
  PROJECT_CODE                 Enabled           1      1      0
  SUPPLIER_CODE                Enabled           1      2      2
  TAX_REGION                   Enabled           0      0      0
Scans by rule set type: Custom 4, System 1
Test values: 6 run, 5 gave the expected result

Checks
  FAIL  Every page of every list was saved  (1)
  FAIL  Every scan's custom rule set is in the export  (1)
  FAIL  Every custom classification rule a rule set includes is in the export  (1)
  FAIL  No rule set includes a disabled classification rule  (1)
  FAIL  Every enabled custom classification rule is in at least one rule set  (1)
  FAIL  Every pattern can be read as a regular expression  (1)
  FAIL  Every custom classification rule in use has a test value that should match  (1)
  FAIL  Every test value gives the result samples.csv expects  (1)
  PAGES_MISSING         scans/pbi-tenant: page-001.json still points to a next page
  UNKNOWN_RULESET       scan lake-claims/curated-zone: uses custom rule set lake-claims-rules-v2, which is not in the export
  UNKNOWN_RULE          rule set lake-claims-rules: includes CLAIM_ID, which is not in the export
  DISABLED_RULE_IN_USE  rule set sql-finance-rules: includes COST_CENTER, which is disabled
  UNUSED_RULE           rule TAX_REGION: enabled, but in no scan rule set
  BAD_PATTERN           rule PROJECT_CODE: data pattern '^PRJ-([A-Z]{3}-\d{3}$': missing ), unterminated subpattern at position 5
  SAMPLE_FAILS          rule EMPLOYEE_ID: 'E-104233' in column 'employee_id': expected match, got no match
  NO_SAMPLE             rule POLICY_NUMBER: in use, with no test value that should match

Wrote rules-planted.csv (6 rules) and findings-planted.csv (8 rows) to .
RESULT: 8 findings. Fix them in Purview or in samples.csv, export again and re-run.
CheckThe problem planted in run 2
PagesThe Power BI data source's scans page points to a next page that was never saved.
Rule setsA curated-zone scan names the rule set lake-claims-rules-v2, which is not in the export.
RulesThe lake-claims-rules rule set includes CLAIM_ID, which is not in the export.
Disabled rulesCOST_CENTER is disabled while sql-finance-rules still includes the rule.
Unused rulesA new TAX_REGION rule, enabled and in no rule set.
PatternsPROJECT_CODE's data pattern with an unclosed parenthesis.
Test valuesPOLICY_NUMBER's test values removed from samples.csv.
ResultsAn employee id written as E-104233 and expected to match.

Run the check on your own scan settings

Run the commands in step 1 with ENDPOINT set to your Purview account's endpoint. The commands write a folder named scan-settings- followed by the date and time, for example scan-settings-20261121-093000, holding one folder of pages each for classificationrules, scanrulesets and datasources, and a scans folder with one folder of pages for each data source. Write samples.csv in that same folder, with the columns rule, column_name, value and expect. Then run python3 check_scan_rules.py followed by the folder's name. The script writes rules.csv and findings.csv to the script's own folder; add --out followed by a folder name to write the two files somewhere else. You do not edit the script.

We also tested a run on a second generated export, saved the way you would save your own: other rule names, three pages of classification rules, a rule with two data patterns, column names separated by the pipe character, a data source with no scans, a samples.csv saved from a spreadsheet with an extra column, and one planted problem for each check. All eight checks failed as planted, and nothing else was flagged.

Download check_scan_rules.py · Get the full guide (PDF)

Result from run 1: rules.csv.

Get the full guide as a PDF

We'll email you a link to it. Please use your work email.

Limits

  • The runs on this page use generated records, not a Purview tenant, and the commands in step 1 were not run. The field names come from Microsoft's Scanning REST specification, api-version 2023-09-01, read on 2026-11-21.
  • The script tests patterns with Python's regular expressions. Microsoft's page does not name the engine Purview uses (its pipe example links to the .NET regular expression reference), and rare syntax can behave differently, so test the rule again on the platform the rule moves to.
  • A test value checks the patterns, not the minimum match threshold, which Purview applies to the share of matching values in a column.
  • The scan pages name the credential and integration runtime each scan uses, but do not hold the credentials themselves, the runtimes' setup or the scan schedules. Microsoft's transition guide lists network runtimes among the things to capture, so record the runtimes, credentials and schedules separately.
  • The script tests regular expression patterns only. Microsoft's page also describes dictionary rules, built from an uploaded file; record those rules separately.
  • Dates are as of 2026-11-21. This guide is about Purview Data Governance, whose support ends on November 12, 2027.

Tested on

Tested on

Platform
DuckDB 1.5.6, run from Python 3.14.8
Edition
Open-source DuckDB library (pip install duckdb)
Date
2026-11-21
Records
Generated: invented scan settings with five custom classification rules, two scan rule sets, three data sources and four scans, the same on every run, in the shape of the Scanning API's answers (api-version 2023-09-01). Not a run against a tenant.
Code
check_scan_rules.py
Results
rules.csv

When this is not enough

The check shows which rules are saved, used and tested. Each rule then needs a home after the switch-off: a sensitivity label or data loss prevention policy in Purview, a rule in another scanner for sources Fabric does not cover, or retirement. Sensitivity labels stay in Purview, so check each item's label after the move as well.

Moving your whole Purview governance catalog, not only the scan rules?

Purview to OneLake Migration is in early access. The offering starts with the four-day Catalog Transition Assessment: one fixed fee of $12,000, and no commitment to anything else. You get an inventory of every object in your Purview governance catalog and a decision for each object. AI does the first pass, and your stewards sign every domain.

See Purview to OneLake Migration