Guide · Metadata enrichment

Use a Business Glossary to Generate Column Descriptions with AI

DuckDB2026-11-05Script and CSVs downloadable, PDF by email

The short answer

Link each column to the glossary terms that govern it, then give the AI those terms plus a profile of the column: counts, and values only for short codes. Plain code checks every description against the glossary's own words, banned words and numbers. The owner of each term approves the exact text before it is saved.

What you will have at the end

A Python script, glossary_descriptions.py, that loads a table into DuckDB, profiles every column, asks Claude to describe each column from your glossary, checks each description with plain code, and writes a review file for the owners. When the owners decide, it writes only the approved descriptions into the table as column comments. The files it writes are the review file glossary_descriptions-review.csv, the owners’ decisions glossary_descriptions-decisions.csv, the published descriptions glossary_descriptions-approved.csv, the same descriptions as glossary_descriptions-comments.sql for your own database, and every request and answer in glossary_descriptions-answers.json. Get the full guide (PDF) for the same steps with the complete code and a checklist.

We do not sell or reuse this data. A public FEC file stands in for a company’s own table. Names and addresses of individual contributors taken from FEC reports may not be sold or used for solicitation or commercial purposes (FEC: sale or use of contributor information (read 2026-11-05)). The script sends Claude the glossary, a sentence about the table and counts for each column. It sends a column’s values only when the column has at most twelve distinct values of at most eight characters and is not in your NO_VALUES list, so none of the FEC’s names, cities or ZIP codes leave your machine, and the script prints none of them.

Step 1. Load the table and profile every column

The table is the FEC’s file of payments that committees made to candidates and to other committees, called pas2, for the closed 2020 election cycle (FEC: file description (read 2026-11-05)). Its column names are short codes such as TRANSACTION_TP, RPT_TP and MEMO_CD, which is why it needs descriptions. The script downloads the file and its header file from the FEC bulk data page with Python’s urllib, prints each file’s byte size, Last-Modified header and the first 16 characters of its SHA-256 so you can tell whether the source changed since our run, unzips it into a folder you give it (it refuses a folder that is not empty), and loads every column as text (DuckDB: CSV overview (read 2026-11-05)).

For each column the script then counts the rows, the filled rows, the blank rows and the distinct values. A blank means empty or only spaces. It keeps the filled count next to the blank count because a description that mixes the two up is wrong and still uses numbers that are in the profile.

Step 2. Write the glossary as data, with owners and links

Each glossary term has an id, the term, its definition, an owner (the person or role who approves descriptions that use it), the columns it governs and the words descriptions must avoid. The links are what make the glossary useful to the AI: a column linked to a term is described in that term’s words. The terms are ours, written from the FEC’s file description, and the owners are invented roles, not people. A column no term governs goes to DEFAULT_OWNER, here the Data steward, and the AI is told to describe only what the column name and the profile show.

GLOSSARY = [
    {"id": "G1", "term": "Committee", "owner": "Filings steward", "columns": ["CMTE_ID"], "avoid": [],
     "definition": "An organization registered with the FEC that raises or spends money in federal elections."},
    {"id": "G2", "term": "Candidate", "owner": "Candidate steward", "columns": ["CAND_ID"], "avoid": [],
     "definition": "A person seeking a federal office, identified by a candidate id."},
    {"id": "G3", "term": "Contribution", "owner": "Payments steward", "columns": ["TRANSACTION_AMT", "TRANSACTION_DT"],
     "avoid": ["donation", "donor", "gift"],
     "definition": "Money or something of value that a committee gives to a candidate or to another committee."},
    ...
]

The script checks the glossary before it asks anything. It reports a term with a blank field, a term id used twice, a link to a column the table does not have, and a column linked to two terms. It leaves the offending term or link out and carries on, so one bad row in the glossary does not stop the other columns.

Step 3. Ask Claude for each description, and save every answer

For each column the script sends one request to claude-haiku-5-5: the whole glossary, one sentence about the table, the ids of the terms linked to that column, and the column’s profile. The instructions tell Claude to use each linked term itself, never a word in a term’s Never-say list, and no number that is not in the profile, the table sentence or a linked definition. They ask for a JSON reply with a description and the list of term ids used. Anthropic describes Claude Haiku 5.5 as a model for high-volume, latency-sensitive tasks such as classification, extraction and routing (read 2026-11-05); we did not compare larger models. Its price for prompts up to 100,000 tokens is $0.10 per million input tokens and $0.50 per million output tokens (Anthropic: pricing (read 2026-11-05)), and the script prints the cost of the run.

The script saves each answer to glossary_descriptions-answers.json as soon as it arrives, and records an API error against the one column it hit, so a failure on column 12 never loses columns 1 to 11. A live run may word descriptions differently from ours. --replay runs every check again from the saved file with no key and no cost, and reproduces the output below exactly, as long as the FEC file has not changed.

Step 4. Check every description with plain code

The check is ordinary Python, never a second AI call. It never decides that a description is true. It decides whether the description stays inside what the glossary and the profile say. The last column of this table is the fault we planted in run 2 to prove each check can fail.

What each check refuses, and the planted fault that trips it
Check	It refuses a description that…	Planted fault that trips it
Known terms only	cites a term id that is not in the glossary, or one not linked to the column	`CMTE_ID` also cites `G99`
Every linked term cited	does not cite a term the steward linked to the column	`TRANSACTION_TP` cites no term
The term’s own words	cites a term but never writes it	`MEMO_CD` says only “Marks the row.”
Banned words	uses a word a term says to avoid	`TRANSACTION_AMT` ends with “a donation”
Numbers	states a number that is not in the profile, the table sentence, the column name or a linked definition	`TRANSACTION_DT` gains “about 4,500 per day”
Length	is longer than 300 characters	`SUB_ID` repeated until it is 413 characters
Reply parses	is not the JSON the script asked for, or is not text	`RPT_TP` cut in half; `CAND_ID` not text
Same prompt	was saved for a different prompt than the one the script would send now	`ENTITY_TP` prompt changed by one space
Glossary checks	blank field, duplicate id, link to a missing column, column linked twice	Four glossary faults, one for each

The banned-word check matches the exact word, not plurals or variants, and the number check reads digits, not numbers written as words. List each form you want refused.

Step 5. Send the descriptions to their owners, and publish only what they approve

The review file has one row per column: the column, its owner, the terms cited, the description, the check’s verdict and a short fingerprint of the description (the first 12 characters of its SHA-256), plus three empty cells for the owner: decision, reviewer and edited_description. Every description goes to the owner of its term, including a description the check caught, because the owner may fix it by editing it. An owner answers with approve, edit or reject.

The script publishes a description as a DuckDB column comment (DuckDB: COMMENT ON (read 2026-11-05)) only when all of these hold. It refuses everything else and says why.

The reviewer is the column’s owner. A steward who does not own the column cannot approve it. Planted: MEMO_TEXT decided by the Payments steward.
An approval names the fingerprint the owner was shown, so a description changed after review is refused. Planted: OTHER_ID approved against a changed description.
An approval is refused for a description the check caught. The owner has to edit it. Planted: seven of the caught descriptions are also approved.
An owner’s edit has to pass the same check. Planted: TRANSACTION_TP edited into “donation” wording.
A column has one decision. Planted: CITY approved twice.

The reviewer cell must match the owner role, apart from letter case. A column with no decision, or whose row the owner left blank, stays pending and gets no comment. In a real run the owners fill in the review file and you pass it back with --decisions. The script reads the owners’ file before writing anything, never writes over the owners’ file, and also reads a file saved from Excel as CSV UTF-8. So that the whole flow runs in one command, --demo-owner plays an invented owner for each role: it approves every description once and edits one, TRANSACTION_AMT, into its own wording. That owner does not read anything, so it shows what the script enforces, not what an owner catches.

Step 6. Run 1: the live run

One command downloads the file, asks Claude about all 22 columns, checks them and applies the invented owner’s decisions:

python3 glossary_descriptions.py --data-dir fec-data --demo-owner

The start of the output: the two source files, the table, the glossary, and the cost of asking about every column.

source  https://www.fec.gov/files/bulk-downloads/data_dictionaries/pas2_header_file.csv  bytes=212  last-modified=Wed, 10 Jan 2018 18:30:12 GMT  sha256=9ce58b0576635ddb
source  https://www.fec.gov/files/bulk-downloads/2020/pas220.zip  bytes=28978598  last-modified=Sun, 29 Dec 2024 22:08:51 GMT  sha256=7a8355e2cdae6c4b
loaded 887829 rows, 22 columns, every column read as text
glossary: 11 usable terms, 14 column links
columns with no glossary term: 8 (NAME, CITY, STATE, ZIP_CODE, EMPLOYER, OCCUPATION, OTHER_ID, TRAN_ID)
model claude-haiku-5-5, answers dated 2026-11-05, 22 columns
tokens in 19488, tokens out 6893, cost $0.0054

Two of the descriptions. TRANSACTION_TP is linked to the glossary term Transaction type, and the description says “Transaction type”. OCCUPATION has no term, so it is described from its name and profile and goes to the Data steward.

TRANSACTION_TP  [owner: Payments steward]  linked: G7
    profile: rows 887829; filled 887829 rows; blank 0 rows (0.0%); distinct values 8; the values are: 24A, 24C, 24E, 24F, 24K, 24N, 24R, 24Z
    description: Transaction type: the FEC code for the kind of payment each row records. All 887829 rows have a code, and 8 distinct codes appear in the column.   cites: G7
    -> OK

OCCUPATION  [owner: Data steward]  linked: none
    profile: rows 887829; filled 17 rows; blank 887812 rows (100.0%); distinct values 13
    description: Occupation of the person linked to a payment, as recorded on the row. It is filled in for only 17 of 887829 rows, with 13 distinct values, so it is almost always empty.   cites: nothing
    -> OK, but no glossary term is linked: the steward decides

The end of the run: the count of descriptions by verdict, the owners’ decisions and the result. The comments are stored in the table, and glossary_descriptions-comments.sql holds the same statements for your own database.

descriptions: 14 ok, 8 ok with no glossary term, 0 caught for a person to read
review file for the owners: glossary_descriptions-review.csv (fill in decision, reviewer and, for an edit, edited_description)

owner decisions: 22 rows (invented owner)
APPROVED CMTE_ID: approve by Filings steward
APPROVED AMNDT_IND: approve by Filings steward
APPROVED RPT_TP: approve by Filings steward
APPROVED TRANSACTION_PGI: approve by Filings steward
APPROVED IMAGE_NUM: approve by Filings steward
APPROVED TRANSACTION_TP: approve by Payments steward
APPROVED ENTITY_TP: approve by Payments steward
APPROVED NAME: approve by Data steward
APPROVED CITY: approve by Data steward
APPROVED STATE: approve by Data steward
APPROVED ZIP_CODE: approve by Data steward
APPROVED EMPLOYER: approve by Data steward
APPROVED OCCUPATION: approve by Data steward
APPROVED TRANSACTION_DT: approve by Payments steward
EDITED   TRANSACTION_AMT: edit by Payments steward: the owner's wording is published
APPROVED OTHER_ID: approve by Data steward
APPROVED CAND_ID: approve by Candidate steward
APPROVED TRAN_ID: approve by Data steward
APPROVED FILE_NUM: approve by Filings steward
APPROVED MEMO_CD: approve by Filings steward
APPROVED MEMO_TEXT: approve by Filings steward
APPROVED SUB_ID: approve by Filings steward
published as DuckDB column comments: 22 of 22 columns
RESULT: PASSED, every published description passed the check and was approved by its owner

Read the descriptions before you trust this run. In an earlier draft of this script, Claude described a mostly blank column as filled in for the number of blank rows. The number was in the profile, so the check passed it. That is why the profile now carries the filled count beside the blank count, and why the owner still has to read.

Step 7. Run 2: plant faults and see what catches them

The same script, replaying the saved answers, with nine faults planted in the answers, four in the glossary and four in the owners’ decisions. The script stops if it cannot plant a fault where it expects to, and it checks that the banned word and the invented number are not already in the answer.

python3 glossary_descriptions.py --data-dir fec-data2 --demo-owner --replay glossary_descriptions-answers.json --plant-error

The four glossary faults, reported before anything is asked:

loaded 887829 rows, 22 columns, every column read as text
glossary: 11 usable terms, 14 column links
GLOSSARY CHECK: term G3 links column 'TRANSACTION_AMOUNT' which is not in the table
GLOSSARY CHECK: column 'FILE_NUM' is linked to both G9 and G10; the link from G10 is left out
GLOSSARY CHECK: term G12 has no definition, so it is left out
GLOSSARY CHECK: term id G2 is used twice; the second one (Recipient) is left out
columns with no glossary term: 8 (NAME, CITY, STATE, ZIP_CODE, EMPLOYER, OCCUPATION, OTHER_ID, TRAN_ID)

Three planted answers, each caught for its own reason:

CMTE_ID  [owner: Filings steward]  linked: G1
    profile: rows 887829; filled 887829 rows; blank 0 rows (0.0%); distinct values 5566
    description: The identifier of the committee that made the payment. It is filled in every row and takes 5,566 distinct values across the table's 887,829 rows.   cites: G1, G99
    -> CAUGHT: cites G99, which is not in the glossary

TRANSACTION_AMT  [owner: Payments steward]  linked: G3
    profile: rows 887829; filled 887829 rows; blank 0 rows (0.0%); distinct values 19373
    description: The dollar amount of each Contribution, the money a committee gave to a candidate or another committee. The amount is filled in on all 887829 rows, a donation.   cites: G3
    -> CAUGHT: uses 'donation', which the glossary says to avoid (G3 Contribution)

MEMO_CD  [owner: Filings steward]  linked: G10
    profile: rows 887829; filled 13239 rows; blank 874590 rows (98.5%); distinct values 2; the values are: X, Y
    description: Marks the row.   cites: G10
    -> CAUGHT: cites G10 but does not use its term 'Memo entry'

The owners’ decisions. Every caught description that the invented owner approved is refused. So are the approval by the wrong steward, the approval of a changed description, the edit that brings back the banned word, and the second decision for one column. Only 12 of 22 comments are published.

descriptions: 5 ok, 8 ok with no glossary term, 9 caught for a person to read
review file for the owners: glossary_descriptions-planted-review.csv (fill in decision, reviewer and, for an edit, edited_description)

owner decisions: 23 rows (invented owner)
REFUSED  CMTE_ID: approve by Filings steward: the check caught this description (cites G99, which is not in the glossary); edit it instead
APPROVED AMNDT_IND: approve by Filings steward
REFUSED  RPT_TP: approve by Filings steward: the check caught this description (reply is not the expected JSON (JSONDecodeError)); edit it instead
APPROVED TRANSACTION_PGI: approve by Filings steward
APPROVED IMAGE_NUM: approve by Filings steward
REFUSED  ENTITY_TP: approve by Payments steward: the check caught this description (the saved answer was made from a different prompt (the glossary, the links or the data changed)); edit it instead
APPROVED NAME: approve by Data steward
APPROVED CITY: approve by Data steward
APPROVED STATE: approve by Data steward
APPROVED ZIP_CODE: approve by Data steward
APPROVED EMPLOYER: approve by Data steward
APPROVED OCCUPATION: approve by Data steward
REFUSED  TRANSACTION_DT: approve by Payments steward: the check caught this description (states 4500, which is not in the profile, the table sentence or a linked definition); edit it instead
EDITED   TRANSACTION_AMT: edit by Payments steward: the owner's wording is published
REFUSED  OTHER_ID: approve by Data steward: the description is not the one the owner was shown (hash 000000000000, now 740aa3dbacba)
REFUSED  CAND_ID: approve by Candidate steward: the check caught this description (no reply text); edit it instead
APPROVED TRAN_ID: approve by Data steward
APPROVED FILE_NUM: approve by Filings steward
REFUSED  MEMO_CD: approve by Filings steward: the check caught this description (cites G10 but does not use its term 'Memo entry'); edit it instead
REFUSED  MEMO_TEXT: approve by Payments steward: the owner of MEMO_TEXT is Filings steward
REFUSED  SUB_ID: approve by Filings steward: the check caught this description (the description is 413 characters (limit 300)); edit it instead
REFUSED  TRANSACTION_TP: edit by Payments steward: the edit fails the same check: cites G7 but does not use its term 'Transaction type'; uses 'donation', which the glossary says to avoid (G3 Contribution)
REFUSED  CITY: approve by Data steward: this column already has a decision
published as DuckDB column comments: 12 of 22 columns
RESULT: 24 things to read before anything is trusted: 9 descriptions caught, 11 decisions refused, 4 glossary problems

Run it on your own data

To run it on your own table, export the table to a CSV file with a header row. In the script, set DATA_CSV to that file's path and set TABLE to the table's name in your database. Replace TABLE_CONTEXT with one sentence that says what each row of your table is. Replace everything inside GLOSSARY with your own terms: give each term an id that no other term uses, the term, its definition, an owner (the person or role who approves descriptions that use it), the columns it governs under columns, and the words descriptions must avoid under avoid. Replace the names in NO_VALUES with the names of your columns that hold personal data, so that their values are never sent to Claude, and replace everything inside DEMO_EDITS with {}. Set DEFAULT_OWNER to whoever approves the descriptions of columns that no term governs. Then run the script without --data-dir: add --demo-owner for a first look, or leave it out to write the review file for your owners. When your owners have filled in the review file, run the script again with --replay glossary_descriptions-answers.json --decisions followed by their file, and it publishes only what passes.

TABLE may also name a schema, such as analytics.claims. Each part is quoted on its own in glossary_descriptions-comments.sql, so write each part in the letter case your database stores that part in: Snowflake stores a name written without quotes in capitals, such as ANALYTICS.CLAIMS.

The script reads every column as text, so a column of numbers or dates is profiled the same way. A glossary link written in another letter case still finds its column. We ran exactly these changes on a made-up claims file with other column names, mixed letter case, a blank key, a column of personal data and a glossary with the four faults above. Every glossary fault was reported, and the decisions with a wrong reviewer, a changed fingerprint, a failing edit, a second decision and an unknown column were refused. The published comments were the owner’s valid edit only.

Get the full guide as a PDF

We'll email you a link to it. Please use your work email.

Method

Dataset: FEC “contributions from committees to candidates and other committees” (pas2), cycle 2020, from https://www.fec.gov/files/bulk-downloads/2020/pas220.zip, with column names from https://www.fec.gov/files/bulk-downloads/data_dictionaries/pas2_header_file.csv. US government data, public. Size, Last-Modified and the first 16 characters of the SHA-256 of each file are in the run 1 output.
One script: every count is over all rows. No sampling, no random seed. The AI step is the only part that can word things differently on a re-run.
The glossary, the links, the owner roles, the banned words and the invented owner’s decisions are written out in the script and shown in steps 2 and 5. They are our judgment, and you can disagree with them.
Model: claude-haiku-5-5, set in one constant. We set no temperature, so a live run can differ from ours.
Self-checks: every row of step 4’s table has a planted fault in run 2 that trips it.

Limits

This is not a client result and not a data set for sale. It describes the columns of a public government file. Your tables have other columns, owners and words to avoid.
A description that passes the check is not proven true. The check proves it stays inside the glossary and the profile. It cannot tell whether Claude read a number correctly, and it cannot tell whether a guess about a column with no term is right. Descriptions of columns with no glossary term, such as OTHER_ID and TRAN_ID, are the AI’s reading of a column name and a profile.
The check cannot see a hedge or a number written as a word. In run 1 the CITY description says “About 1,327 rows”, which hedges a number the prompt told Claude to copy exactly. The digits match the profile, so the check passed it.
The check does not decide whether the glossary is right. Our definitions are ours, written from the FEC’s file description. The FEC remains the source for what its columns mean.
The invented owner reads nothing. Run 1 shows that the approval rules work. It does not show that real owners will catch errors.
The source can change. The FEC republishes its bulk files. Our numbers are for the files as they stood on 2026-11-05.
A column comment is stored in the table’s own database. In-memory DuckDB forgets it when the script ends. Run glossary_descriptions-comments.sql against the database you keep.

Tested on

Test environment

Platform: DuckDB 1.5.6, Python 3.12, run from Python with the duckdb and anthropic modules
Edition: Open-source DuckDB, in-memory
Model: claude-haiku-5-5 (Anthropic: models overview (read 2026-11-05))
Date: 2026-11-05
Dataset: FEC committee contributions to candidates and other committees (pas2), cycle 2020, 28,978,598 bytes for the data file (US government data, public; source)
Code: glossary_descriptions.py, saved answers glossary_descriptions-answers.json

When this is not enough

A passing description is not a true one. The check proves wording, grounding in the glossary and numbers from the profile, so owners must still read each description.
The glossary sets the limit. Eleven terms linked 14 of the 22 columns here, and the other eight go to a data steward to decide.
One script is not a program for hundreds of tables. This guide documents one file of 22 columns, and a large estate needs the same discipline repeated across every table.

Get descriptions of your tables ready for owner approval with Metadata Enrichment Engine

Metadata Enrichment Engine delivers AI-driven contextualized business descriptions, sensitivity classifications and data quality rules for your tables.

Your data owners approve every description before it is published. They review a finished draft instead of writing each one from a blank page.

See Metadata Enrichment Engine

Advise

Build

Govern

Operate

Sectors