Does NYC Local Law 144 apply when HR uses Claude?
The short answer
It depends, and Legal decides. New York City's rule asks whether a tool's score, tag, ranking or recommendation is the only factor, weighs most, or overrules other conclusions. Those are three tests. Our script sorts each Claude use in HR against them, marks the ones Legal should look at, and works out impact ratios. It is not a bias audit.
What the law and the City's rule say
Local Law 144 makes it unlawful for an employer or employment agency to use an automated employment decision tool to screen candidates for employment or employees for promotion within the city unless the tool has had a bias audit no more than one year before use, a summary of the latest audit is public, and candidates and employees have had notice (Administrative Code § 20-870 and § 20-871, NYC Council: Local Law 144 of 2021 (read 2026-10-31), Legistar file ID 4344524). The City's Department of Consumer and Worker Protection (DCWP) adopted a rule that says what "substantially assist or replace discretionary decision making" means. Its section 5-300 gives three tests (DCWP: adopted rule text (read 2026-10-31)):
| Test | What the rule says | Register answer (output_use) |
|---|---|---|
| i | “to rely solely on a simplified output (score, tag, classification, ranking, etc.), with no other factors considered” | decides alone |
| ii | “to use a simplified output as one of a set of criteria where the simplified output is weighted more than any other criterion in the set” | weighted most |
| iii | “to use a simplified output to overrule conclusions derived from other factors including human decision-making” | overrules other factors |
A "simplified output" is a prediction or classification: the rule's examples are a score, a tag or categorization, a recommendation (whether a candidate should be given an interview) and a ranking (arranging candidates by how well their cover letters match the job description). The rule says the term does not refer to the output from analytical tools that translate or transcribe existing text. The rule's definitions of "simplified output" and "impact ratio" and the wording of the three tests are in the same adopted rule text (read 2026-10-31); the rule page gives its effective date as July 5, 2023 (NYC Rules: adopted rule (read 2026-10-31)).
Three further points from the City's own pages shape what a script can do. The FAQ says the law applies only to employers and employment agencies that use a tool "in the city"; that means the job's location is a New York City office, at least part time; or the job is fully remote but the location tied to it is a New York City office; or the employment agency using the tool is in New York City (or, if it is outside the city, one of the first two is true). It also says the law does not require any specific action based on the results of a bias audit, and does not require auditors to be approved by DCWP (DCWP: AEDT frequently asked questions (read 2026-10-31)). So the script applies no pass mark: it prints numbers, and what to do about them is for you and your counsel. DCWP's own page lists what must be public and when notice is due (DCWP: automated employment decision tools (read 2026-10-31)).
What this covers, and what it does not
This guide does not tell you whether Local Law 144 applies to any use of Claude at your company. That is a legal question for your lawyers, and the City's rule has more in it than three tests: the tool must also be derived from "machine learning, statistical modeling, data analytics, or artificial intelligence" as the rule defines those words, the use must be an "employment decision" in the city, and the audit must be done by an independent auditor on historical data in the cases the rule describes. The script does not test any of that. It does two things that sit beside it: it sorts your own record of each use against the three tests, so Legal sees which uses to read first, and it works out selection rates and impact ratios by the rule's formulas, so you can see the selection-rate and impact-ratio calculations the rule defines and check your own numbers against the City's worked example.
This is not a bias audit. Local Law 144 requires an independent auditor, an audit within one year before use, a published summary, and notice to candidates at least 10 business days before use (Administrative Code § 20-871 in Local Law 144 of 2021 (read 2026-10-31), and sections 5-301, 5-303 and 5-304 of the adopted rule text (read 2026-10-31)). The script uses generated records. Data Meaning is not the auditor.
What you will have at the end
A Python script, nyc_ll144_claude_check.py, that reads your Claude use-case register and your selection records. It pins the City's rule PDF by size and SHA-256, recomputes the rule's own worked example as a check on its formulas, runs nine input checks, then builds two lists and prints each with its filters, its row count and a SHA-256 of its rows. It also writes each list as a CSV. The full detail is in a PDF: Download the full guide (PDF).
- The use-case sort: every register row with a status.
FOR LEGAL TO CHECKcomes with a reason such asmay meet test ii.OUTSIDE THE THREE TESTS AS RECORDEDmeans the register says the output is weighted equally or less, is not a criterion, is not a score, tag, recommendation or ranking, or only translates or transcribes.NOT A HIRING OR PROMOTION USE AS RECORDEDandNOT USED IN THE CITY AS RECORDEDfollow the register's own answers. An empty answer, or one the script does not recognize, is never read as outside the three tests: it sends the row to Legal, unless an earlier answer in the order in Step 2 already sets the row aside. The script never says a use is an automated employment decision tool and never says a use is exempt. - Selection rates and impact ratios: for each category by sex, by race/ethnicity and by both together, the applicants, the number selected, the selection rate, the impact ratio, and the number of people in an unknown category who are left out. The 2% option is shown in a separate column beside the full calculation.
The records we ran it on are generated: a seeded stand-in of 2,000 invented applicants and seven invented register rows. They are not a real organization's data and not a measurement of anything. We have not run the script on a real company's records.
What you will need
- A register of your Claude uses in HR as a CSV with a header row: an id, the use, whether it supports a hiring decision, a promotion decision or neither, how its output enters the decision (one of the answers in the table above, or
weighted equal or less,not a criterion,no score, tag, recommendation or ranking,translates or transcribes), whether the use is "in the city" as the City's FAQ defines it (any of its three cases counts as yes), and an owner. - Your selection records as a CSV with a header row: one row for each candidate or employee considered, with an id, sex, race/ethnicity as the EEO-1 categories the rule uses, and whether the person was selected to move forward (1 or 0). These are records about people: keep them under your own access controls. The script runs on your machine and sends nothing to Data Meaning; it does fetch the City's rule PDF.
- Python 3 and the
duckdbmodule. The script reads each CSV with DuckDB'sread_csvusing a header row andall_varchar(DuckDB: CSV overview (read 2026-10-31)).
Step 1. Pin the City's text, and set the answers you can disagree with
The script downloads the adopted rule and prints its size, its Last-Modified header and its SHA-256. If the size or the SHA-256 differs from the pin, an input check fails, so you know the City re-published the file and the quoted definitions need reading again. The Last-Modified header is shown for you to read; the script does not compare it. If the download fails, the check fails too: a check that could not run is not reported as ok. The mapping from your answers to the three tests is a table at the top of the script. It is ours, and you can disagree with it.
OUTPUT_USE_WORDS = {
"decides alone": "i", # test i: relies solely on a simplified output, no other factors considered
"weighted most": "ii", # test ii: one of a set of criteria, weighted more than any other
"overrules other factors": "iii", # test iii: overrules conclusions from other factors, including a person's decision
"weighted equal or less": "none", # one criterion, weighted no more than another
"not a criterion": "none", # read for background, no weight in the decision
"no score, tag, recommendation or ranking": "none", # no simplified output at all
"translates or transcribes": "none", # the rule excludes tools that translate or transcribe existing text
}
The answers are matched after trimming spaces and ignoring case. Any answer in your files that is in none of the lists is counted and fails an input check; it is never guessed. In the register, it sends the row to Legal unless an earlier answer in the order in Step 2 already sets the row aside (a decision of neither, for example).
Step 2. Sort each use by the three tests, as recorded
The sort takes the first status that applies, in this order: a decision answer that is empty or unknown goes to Legal; a use that supports neither hiring nor promotion is set aside; an output-use or in-the-city answer that is empty or unknown goes to Legal; a use recorded as not used in the city is set aside; a use whose output enters the decision by test i, ii or iii goes to Legal with that test; the rest are outside the three tests as recorded. The statuses say what the register records. They do not say what the law requires.
CASE WHEN d.meaning IS NULL THEN 'FOR LEGAL TO CHECK'
WHEN d.meaning = 'neither' THEN 'NOT A HIRING OR PROMOTION USE AS RECORDED'
WHEN o.meaning IS NULL THEN 'FOR LEGAL TO CHECK'
WHEN n.meaning IS NULL THEN 'FOR LEGAL TO CHECK'
WHEN n.meaning = 'no' THEN 'NOT USED IN THE CITY AS RECORDED'
WHEN o.meaning IN ('i', 'ii', 'iii') THEN 'FOR LEGAL TO CHECK'
ELSE 'OUTSIDE THE THREE TESTS AS RECORDED' END AS status,
Step 3. Work out the selection rate and the impact ratio
The rule defines the selection rate as the number of individuals in a category moving forward divided by the number in the category who applied or were considered. Its example: if 100 Hispanic women apply and 40 are selected for an interview after use of a tool, the selection rate is 40/100 or 40%. The impact ratio is the selection rate for a category divided by the selection rate of the most selected category (adopted rule text (read 2026-10-31), section 5-300). The function below is the whole calculation. A table with no one selected has no impact ratio, and the script prints n/a rather than dividing by zero.
def impact_ratios(rows, wrong=False):
"""6 RCNY 5-300: impact ratio = the selection rate of a category / the selection rate of the most selected category.
rows: (category, applicants, selected). Returns {category: (selection rate, impact ratio or None)}.
wrong=True divides by the LOWEST rate instead, only to show the self-check below catching it."""
rates = {c: s / n for c, n, s in rows if n > 0}
ref = (min if wrong else max)(rates.values()) if rates else 0
return {c: (r, r / ref if ref > 0 else None) for c, r in rates.items()}
The script makes three tables, as section 5-301(b) asks: by sex, by race/ethnicity, and by sex and race/ethnicity together. People whose sex or race/ethnicity is blank are in an unknown category: the script leaves them out of the calculation and prints how many, because the rule asks the audit to indicate that number. So are people who declined to self-identify: list such an answer with the meaning None (the defaults list "I do not wish to self-identify"), and the script counts it as unknown, never as a category, and prints how many declined. An answer that is in none of your lists fails an input check. Until you add it to a list, the script leaves that person out of the calculation and counts them with the unknown category, so read the unknown count together with the input checks.
Step 4. Check the formulas against the City's own example
Section 5-301(b) of the rule prints a worked example: 1390 male applicants with 667 selected, and 1181 female applicants with 555 selected, giving selection rates of 48% and 47% and an impact ratio of 0.979 for women; it also gives a race/ethnicity table and a table by sex and race/ethnicity together. The script recomputes the first two tables from the counts and compares each rounded figure with the one the rule prints. It does not recompute the third. If any figure differs, an input check fails.
def check_formulas(wrong):
"""Recompute the City's own worked example and compare each figure with the rule's printed one."""
off = []
for dim, table in RULE_EXAMPLE.items():
got = impact_ratios([(c, n, s) for c, n, s, _, _ in table], wrong)
for c, n, s, pct, ratio in table:
rate, ir = got[c]
if f"{rate * 100:.0f}" != pct or ir is None or f"{ir:.{len(ratio.split('.')[1])}f}" != ratio:
off.append(f"{dim}: {c}")
return off
Step 5. Show the 2% option beside the full calculation
Section 5-301(d) lets an independent auditor leave out of the impact-ratio calculation a category that represents less than 2% of the data used for the audit. The summary must then give the auditor's justification and the excluded category's number of applicants and rate. The script prints both calculations side by side and leaves the choice to you and your auditor. It takes a category's share of the data as its share of the people with a known category in that table, which is our reading. When a small category has the highest rate, leaving it out moves every other ratio, as the first run shows.
Step 6. Run nine input checks before trusting either list
- The City's rule is not the pinned file (trips on: the City re-publishing the rule, or a failed fetch).
- The City's worked example recomputes to different figures (trips on: an impact ratio divided by anything but the most selected category's rate; run 3 shows it).
- Register ids listed more than once, and register rows with no id (trip on: a duplicated or unnumbered use).
- Register answers in no list (trips on: an answer such as
overridesthat is not one of your words). - Candidates listed more than once, and rows with no candidate id (trip on: a duplicated or unnumbered applicant).
- Selected values that are not in your list (trips on: a value such as
maybe; the row is left out and counted). - Sex or race/ethnicity answers in no list (trips on: an answer such as
Hisp.).
The script also checks that every selection row is in a category, in an unknown category or left out as unreadable, and that the shares of the data add to 100% in each table. Those two guard the script's own arithmetic: no planted fault trips them, so we say so rather than claim a run that does not exist. A word in your list that no row uses is printed as information. A failed input check marks both lists INCOMPLETE.
Run 1: clean generated records
Seed 225, 2,000 invented applicants, seven invented register rows. The input checks pass. One of the seven uses is for Legal: the use that ranks resumes against a job description, recorded as weighted most, so it may meet test ii.
The blocks below are excerpts of the run; the PDF has the full output.
input : GENERATED records (not a real organization), seed 225, mode=clean == The City's rule (pinned: a different size or hash means the City re-published the file) 6 RCNY 5-300 to 5-304 as adopted (DCWP, effective 2023-07-05): 462675 bytes, Last-Modified Thu, 06 Apr 2023 13:36:03 GMT, sha256 4ecc4463b29440f9b876b0b7319faf2d4c4710b8ad28db9cfa8ae15a123ece2c == Input checks (register rows: 7; selection rows: 2000) [ok ] City rule that is not the pinned file: 0 [ok ] the City's worked example recomputed to different figures: 0 [ok ] register ids listed more than once: 0 [ok ] register rows with no use case id: 0 [ok ] register answers in no list (decision, output use, New York City): 0 [ok ] selection candidates listed more than once: 0 [ok ] selection rows with no candidate id: 0 [ok ] selected values that are not in SELECTED_WORDS: 0 [ok ] sex and race/ethnicity answers in no list: 0 [info] output_use answers in your word list that no row uses: overrules other factors == 6 RCNY 5-300: the three tests: use_case_sort filters : every register row, none dropped; first applicable status; an empty or unrecognized answer is FOR LEGAL TO CHECK, never outside rows : 7 sha256 : 8ede27ebdd69332068b3c9d6f267760731c697c273e5866392c7525b1b9c6748 1 | UC-101 | OUTSIDE THE THREE TESTS AS RECORDED | 2 | UC-102 | FOR LEGAL TO CHECK | may meet test ii 3 | UC-103 | NOT A HIRING OR PROMOTION USE AS RECORDED | 4 | UC-104 | OUTSIDE THE THREE TESTS AS RECORDED | 5 | UC-105 | OUTSIDE THE THREE TESTS AS RECORDED | 6 | UC-106 | NOT A HIRING OR PROMOTION USE AS RECORDED | 7 | UC-107 | NOT USED IN THE CITY AS RECORDED | FOR LEGAL TO CHECK: 1; NOT A HIRING OR PROMOTION USE AS RECORDED: 2; NOT USED IN THE CITY AS RECORDED: 1; OUTSIDE THE THREE TESTS AS RECORDED: 3 This sorts uses by what the register says. It does not say a use is or is not covered: Legal decides.
By sex, 72 of the 2,000 people are in an unknown category and are left out; 38 of them declined to self-identify. The impact ratio of the Female category is 0.896, which is its selection rate divided by the Male rate. The second table has the 2% option at work.
-- sex: applicants in the calculation 1928; in an unknown category (left out, as 5-301(b)(5) asks you to count): 72 (of them, declined to self-identify: 38) category applicants selected rate share impact ratio if <2% left out Female 926 265 28.6% 48.0% 0.896 0.896 Male 1002 320 31.9% 52.0% 1.000 1.000 -- race/ethnicity: applicants in the calculation 1943; in an unknown category (left out, as 5-301(b)(5) asks you to count): 57 (of them, declined to self-identify: 27) category applicants selected rate share impact ratio if <2% left out Asian (Not Hispanic or Latino) 366 110 30.1% 18.8% 0.488 0.872 Black or African American (Not Hispanic or Latino) 368 94 25.5% 18.9% 0.415 0.741 Hispanic or Latino 333 87 26.1% 17.1% 0.425 0.758 Native American or Alaska Native (Not Hispanic or Latino) 26 16 61.5% 1.3% 1.000 excluded Native Hawaiian or Pacific Islander (Not Hispanic or Latino) 42 8 19.0% 2.2% 0.310 0.553 Two or More Races (Not Hispanic or Latino) 39 9 23.1% 2.0% 0.375 0.670 White (Not Hispanic or Latino) 769 265 34.5% 39.6% 0.560 1.000 under 2% of the data, so an independent auditor may leave them out of the impact ratio; the summary must still give their applicants and rate: Native American or Alaska Native (Not Hispanic or Latino) 26 applicants, rate 61.5%
The category Native American or Alaska Native has 26 applicants, 1.3% of the data, and the highest rate in the table, 61.5%. With every category counted, White has an impact ratio of 0.560. With that small category left out, White is 1.000 and Asian moves from 0.488 to 0.872. The script prints both and does not choose between them. These numbers come from invented records and say nothing about any real tool.
Run 2: planted faults, and the script finds them
The same script with --plant. It adds seven faults: a duplicated register id, a register row with no id, an answer (overrides) in no list, a duplicated candidate, a candidate with no id, a selected value (maybe) and a race answer (Hisp.). It also adds register rows with no answer on output use, with the answer written in another case with spaces, and with no New York City answer. Each fault trips its own check, both lists are marked INCOMPLETE, and seven uses are for Legal.
== Input checks (register rows: 13; selection rows: 2002) [ok ] City rule that is not the pinned file: 0 [ok ] the City's worked example recomputed to different figures: 0 [FAIL] register ids listed more than once: 1 (trips on: a duplicated register row) [FAIL] register rows with no use case id: 1 (trips on: a register row with no id) [FAIL] register answers in no list (decision, output use, New York City): 1 (trips on: an answer such as 'overrides' that is not one of your words) [FAIL] selection candidates listed more than once: 1 (trips on: a duplicated candidate row) [FAIL] selection rows with no candidate id: 1 (trips on: a row with no candidate id) [FAIL] selected values that are not in SELECTED_WORDS: 1 (trips on: a selected value such as 'maybe') [FAIL] sex and race/ethnicity answers in no list: 1 (trips on: an answer such as 'Hisp.' that is not one of your words) [info] output_use answers in your word list that no row uses: overrules other factors == 6 RCNY 5-300: the three tests: use_case_sort ** INCOMPLETE: see input checks ** filters : every register row, none dropped; first applicable status; an empty or unrecognized answer is FOR LEGAL TO CHECK, never outside rows : 13 sha256 : 15901cc3782b76e1629969e7b943a2c0f6b6f49a773d73ffedf49b8434af4ef3 1 | UC-101 | OUTSIDE THE THREE TESTS AS RECORDED | 2 | UC-102 | FOR LEGAL TO CHECK | may meet test ii 3 | UC-103 | NOT A HIRING OR PROMOTION USE AS RECORDED | 4 | UC-104 | OUTSIDE THE THREE TESTS AS RECORDED | 5 | UC-105 | OUTSIDE THE THREE TESTS AS RECORDED | 6 | UC-106 | NOT A HIRING OR PROMOTION USE AS RECORDED | 7 | UC-107 | NOT USED IN THE CITY AS RECORDED | 8 | UC-108 | FOR LEGAL TO CHECK | output use not recorded or not recognized 9 | uc-109 | FOR LEGAL TO CHECK | may meet test ii 10 | UC-110 | FOR LEGAL TO CHECK | output use not recorded or not recognized 11 | UC-111 | FOR LEGAL TO CHECK | in the city not recorded or not recognized 12 | UC-102 | FOR LEGAL TO CHECK | may meet test ii 13 | - | FOR LEGAL TO CHECK | decision not recorded or not recognized FOR LEGAL TO CHECK: 7; NOT A HIRING OR PROMOTION USE AS RECORDED: 2; NOT USED IN THE CITY AS RECORDED: 1; OUTSIDE THE THREE TESTS AS RECORDED: 3 This sorts uses by what the register says. It does not say a use is or is not covered: Legal decides.
Row 9 shows the case rule: Weighted Most is read as weighted most, so the use may meet test ii. Rows 8, 10, 11 and 13 are for Legal because an answer is empty or not recognized; none is called outside.
Run 3: the check on the formula
With --wrong-denominator the script divides each rate by the lowest rate instead of the highest. The recomputed worked example no longer matches the City's, and the input check fails: the 9 at the end of the line is the number of categories in the City's example (two by sex, seven by race/ethnicity) with a figure that differs, and the message names the first three. The flag exists only to show the check can fail.
[FAIL] the City's worked example recomputed to different figures: sex: Male, sex: Female, race/ethnicity: Hispanic or Latino: 9 (trips on: an impact ratio divided by anything but the most selected category's rate (run with --wrong-denominator))
How to read the results
- A status is a prompt for Legal, not a finding.
FOR LEGAL TO CHECKmeans the register says the use may meet a test, or the register is silent or not recognized.OUTSIDE THE THREE TESTS AS RECORDEDis only as good as the register's answer. - An impact ratio is a number, not a verdict. The law sets no pass mark for it and the script applies none.
- Both columns of the 2% option matter. Which one an auditor uses, and why, goes in the published summary.
- The unknown count is part of the result. Report it with the tables.
Run it on your own data
To run it on your own data, change only the settings between the lines marked CHANGE THESE FOR YOUR OWN DATA. Set REGISTER_FILE and SELECTIONS_FILE to the paths of your two CSV files. In REG_COLS and SEL_COLS, put the name of the column in your files that holds each field. In DECISION_WORDS, OUTPUT_USE_WORDS, NYC_WORDS, SEX_WORDS, RACE_WORDS and SELECTED_WORDS, put the answers your files use, in lower case, each with the meaning the script gives it. An answer that is in none of those lists is counted and fails an input check, so the run tells you which one to add.
We also ran the script on a second set of generated files with other file names, other column names, other answer words, upper-case answers, a blank New York City answer and a fault planted for each of the seven checks that read your files. It read MOST WEIGHT as the second test, sent the blank New York City answer to Legal, counted two "prefer not to say" answers as unknown without failing a check, printed the unknown count for sex, and failed the input checks for the repeated id, the row with no id, the answer in no list, the candidate listed twice in another case, the row with no candidate id, the selected value ? and the ethnicity answer in no list. We have not run it on a real company's records.
Download nyc_ll144_claude_check.py
Download the full guide (PDF)
Method
- Input: generated records, written by the script with a fixed seed (225): a register of seven invented Claude uses and 2,000 invented applicants with a sex, a race/ethnicity category and a selected value. The category weights and selection rates are invented for the demonstration.
- The City's text: definitions, the three tests, the formulas, the 2% option and the worked example are read from the adopted rule text (read 2026-10-31), which the script pins by size (462675 bytes) and SHA-256. The statute is read from NYC Council: Local Law 144 of 2021 (read 2026-10-31), and the FAQ points from DCWP: AEDT frequently asked questions (read 2026-10-31). None of the web pages linked on this page is pinned.
- Selection rate and impact ratio: as the rule defines them, computed from counts. The most selected category in each table is the reference. Percentages are rounded to one decimal and ratios to three.
- The 2% option: a category's share is its applicants divided by the applicants with a known category in that table.
- Settings you can disagree with are at the top of the script: the answer lists, the column names and the mapping to the three tests.
- Every problem row is counted. An unrecognized answer is counted and never guessed. A failed input check marks the lists INCOMPLETE.
Limits
- This is not a client result. The records are generated. The runs show the script finds planted faults and reproduces the sex and race/ethnicity tables in the City's example. They say nothing about any real tool or company.
- This is not a bias audit, and Data Meaning is not an auditor. The law requires an independent auditor and a published summary.
- The script does not decide whether Local Law 144 covers a use. It sorts uses by what your register says. Legal decides.
- It calculates selection rates only. For a tool that scores candidates, the rule uses a scoring rate based on the sample's median score (section 5-301(c)). The script does not calculate it.
- The three tests need a simplified output. A register answer that says there is none puts the use outside the tests as recorded. If the answer is wrong, so is the status.
- Sex and race/ethnicity are sensitive data. Not every applicant gives their sex or race/ethnicity, and the unknown category counts those people.
- Our reading is not the City's. The mapping from register answers to the tests, treating a declined answer as unknown, and the 2% share, are ours. We have not asked DCWP to review them. The rule may be amended: check the City's pages for the current text.
Tested on
Tested on
- Platform
- DuckDB 1.5.6, Python 3.12, run from Python with the
duckdbmodule - Edition
- Open-source DuckDB, in-memory
- Date
- 2026-10-31
- Dataset
- Generated records, seed 225 (not a real organization). Definitions, formulas and worked example from the City's adopted rule (read 2026-10-31)
- Code
- nyc_ll144_claude_check.py
When this is not enough
- You need the bias audit itself. Only an independent auditor can do it, and the summary must be published before use.
- You need an answer on coverage. Take the
FOR LEGAL TO CHECKrows, and the rows whose answers you doubt, to your counsel. - Your tools produce a score. You need the scoring rate as well, and your auditor's method.
- You do not yet keep a register. A register of Claude uses has to exist before this script has anything to sort.
Where Control Tower for Claude runs
Control Tower for Claude runs in your own Azure subscription. See how it is set up on the solution page.