NYC Local Law 144 AI Compliance Checklist: 11 Items With an Objective Completion Test

Local Law 144 took effect on 1 January 2023, and the New York City Department of Consumer and Worker Protection has enforced it since 5 July 2023. In December 2025 the New York State Comptroller published an audit of that enforcement covering July 2023 through June 2025 and called it ineffective. DCWP had reviewed 32 published bias audits and identified one non-compliance issue; the Comptroller re-read the same 32 and identified at least 17 potential issues.

DCWP agreed to recommendations covering complaint handling, staff training, written policies, and enforcement through tool demonstrations and interviews. A summary that cleared a light review will meet a heavier one on the next cycle.

These eleven items run in dependency order. Each states the test somebody outside your HR team could run to confirm it is done.

1. Determine whether each tool meets the AEDT definition

The law reaches computational processes derived from machine learning, statistical modelling, data analytics, or artificial intelligence that issue simplified output used to substantially assist or replace discretionary decision-making for employment decisions. Product category is beside the point; function decides. A general-purpose model prompted to rank resumes falls to be assessed on what it does.

Done when: a dated written determination exists for every tool touching hiring or promotion, naming who made the call and on what basis.

2. Establish which candidates reside in New York City

The obligation attaches to candidates and employees who reside in the city, which is a data question your applicant tracking system may or may not answer reliably.

Done when: the residency field is populated for the full candidate population over the last audit period, with a documented method for the records where it was absent.

3. Record every AEDT invocation on the request path

This is the item the other eight depend on and the one almost nobody has. Where the tool is reached over HTTP, whether a hosted model or a vendor scoring API, each call is a request leaving your boundary carrying a candidate reference.

An applicant tracking system stores the outcome a recruiter entered. It rarely stores which tool version ran, when it ran, or whether it ran at all for a given candidate.

Done when: for a sampled candidate, a record exists naming the tool and version, the timestamp, and the request that carried their data.

4. Give notice before the tool runs, not after

Candidates must be notified before the AEDT is used, with the job qualifications and characteristics it assesses, and told of their right to request an alternative process.

The obligation is time-ordered, so evidencing it means holding two timestamps that sit in the right sequence.

Done when: for a sampled candidate, the notice timestamp precedes the first AEDT invocation timestamp for that same candidate.

5. Publish the alternative-process route and staff it

A stated right that routes to an unmonitored inbox creates the exposure it was meant to close.

Done when: a named owner exists, and requests from the last quarter show a recorded outcome each.

6. Commission the bias audit from a genuinely independent auditor

Independence is a procurement question and it is the one the Comptroller's re-read suggests gets treated casually. An auditor with a financial interest in the tool or in the outcome fails the requirement regardless of the quality of the arithmetic.

Done when: an engagement letter exists documenting the auditor's independence from both your organisation's interest in the result and the tool vendor.

7. Give the auditor tool-behaviour data, not recruiter-decision data

The audit computes selection or scoring rates by sex, race, and ethnicity from tool outputs joined to demographic data. A spreadsheet exported from an applicant tracking system describes what recruiters recorded, which diverges from tool behaviour whenever a score was overridden or an integration failed quietly.

Done when: the dataset handed to the auditor reconciles, row for row, against the invocation records from item 3.

8. Run the audit annually and diary the next one

Each day a tool is used without a valid bias audit counts as a separate violation, which turns a lapsed audit into a compounding exposure rather than a single finding.

Done when: the audit date is within the last 12 months and the next engagement is booked with a named auditor.

9. Post the summary clearly and conspicuously, and keep it up

Publication on the employer's website is the transparency obligation, and "clearly and conspicuously" has been read to exclude a link buried three levels into a careers footer.

Done when: the summary loads from a URL reachable in two clicks from the careers page, and a dated screenshot is retained for each posting period.

10. Check the summary against what the law asks it to contain

The Comptroller identified at least 17 potential issues across 32 posted summaries that DCWP had largely cleared. Read your own summary as an adversarial reviewer would, against the source requirement rather than against whatever the auditor's template produced.

Done when: a dated line-by-line review exists comparing the posted summary against the requirement, with any gap either fixed or explained in writing.

11. Retain the underlying records for the full exposure period

Penalties reach $1,500 per violation with daily accrual, and defending a historical period requires the records from that period rather than a current snapshot.

Done when: invocation records, notice timestamps, audit reports, and posted-summary screenshots are retained across the full period of tool use and verify against their stored form.

What a gateway contributes and what it does not

Items 3, 4, 7 and 11 are record problems that a component on the request path solves. The rest are governance, procurement, statistics, and web publishing, and treating them as an architecture problem would leave a team underprepared.

The disparate-impact calculation is statistical work performed by an independent auditor on demographic data, and no proxy performs it. Auditor independence is settled at procurement. The published summary is a web-publishing obligation. The notice text, the alternative process, and the recruiting workflow around them belong to HR and legal. What sits on the HTTP path is the invocation record underneath items 3, 4, 7 and 11, and that record happens to be the input everything else is computed from.

My candid view: the enforcement audit will produce a market for better-looking bias audit summaries, and the summary was never the weak point. The weak point is that most employers reconstruct tool behaviour after the fact from an applicant tracking system, then have an independent auditor compute statistics on the reconstruction. The full requirement walkthrough sits in the Local Law 144 bias audit piece, and the evidence question in the audit evidence walkthrough.

DeepInspect

This is the record layer items 3, 4, 7 and 11 need. DeepInspect sits inline between your systems and the model APIs they call, as a stateless proxy the calling application has no custody over. It evaluates identity, request classification, and destination on every call, enforces per-role and per-route policy with a fail-closed default, and writes a signed, tamper-evident per-decision record.

Where an AEDT is reached over HTTP, that record establishes which candidate reference went to which tool version at which moment, produced outside both the tool vendor's systems and your applicant tracking system. The bias audit still belongs to an independent auditor and the notice still belongs to your recruiting workflow, with the data underneath both stopping being a reconstruction. Book a technical deep dive at deepinspect.ai.