What did the New York State Comptroller find about Local Law 144 enforcement?

The audit, published in December 2025 and covering July 2023 through June 2025, concluded that DCWP's enforcement was ineffective. It found that 75% of test calls to the 311 hotline regarding AEDT issues were improperly routed and never reached DCWP, that DCWP reviewed 32 posted bias audits and identified one non-compliance issue where the Comptroller's own review of the same 32 identified at least 17, and that DCWP had not used its formal Enforcement Workbook or consulted the Office of Technology and Innovation.

Does a weak enforcement record reduce an employer's risk?

It reduces the chance of being caught rather than the obligation. DCWP agreed to the Comptroller's recommendations on complaint handling, staff training, written policies, and enforcement through tool demonstrations and interviews, which points toward more scrutiny rather than less. Penalties reach $1,500 per violation with each day of use without a valid audit counting separately.

What data does a Local Law 144 bias audit need?

Tool outputs joined to candidate demographic data across the audit period, sufficient for an independent auditor to compute selection or scoring rates by sex, race, and ethnicity and compare them. The quality question is where the tool-output side comes from, since an applicant tracking system records the decision a recruiter entered rather than what the tool returned and when.

Can our AEDT vendor's reports serve as the evidence?

They can contribute, and they carry the weakness of being produced by the party whose tool is under examination. A vendor report that undercounts invocations understates the population the bias audit should cover, and the employer holds the obligation either way. A record produced on the request path, outside the vendor's systems, is the version that does not depend on the vendor's own accounting.

How do we evidence the candidate notice requirement?

By showing that notice preceded use for each candidate, which requires a timestamp on the notice and a timestamp on the first AEDT invocation for the same candidate. Most deployments can produce the first and estimate the second. Joining the two into one sequence is what converts a policy statement into evidence.

Does Local Law 144 apply if the tool is a general-purpose LLM?

The law addresses computational processes used to substantially assist or replace discretionary decision-making in hiring or promotion for candidates residing in New York City, rather than a named product category. A general-purpose model used to score or rank candidates falls to be assessed on that function. Where the model is reached over HTTP, every scoring call is a request that can be recorded on the path.

NYC Local Law 144 AI Audit Evidence: What the Comptroller Found When It Re-Read 32 Bias Audits

In December 2025 the New York State Comptroller published an audit of the Department of Consumer and Worker Protection's enforcement of Local Law 144, covering July 2023 through June 2025. The finding that should reach anyone running an automated employment decision tool sits in the compliance-review section. DCWP reviewed 32 publicly posted bias audits and identified one non-compliance issue. The Comptroller's own review of the same 32 audits identified at least 17 potential issues.

The Comptroller called the enforcement "ineffective." It also found that 75% of test calls made to the 311 hotline about AEDT issues were misrouted and never reached DCWP, and that the agency had not used its own Enforcement Workbook when determining potential violations. DCWP agreed to the recommendations.

Read the enforcement history as a floor rather than a ceiling. An employer whose posted audit summary was cleared by a review that a state auditor then found seventeen problems in has not been found compliant, and the reviewer's attention level is the variable that changed.

What the law requires

Local Law 144 took effect on 1 January 2023 and DCWP began enforcement on 5 July 2023. It applies to employers and employment agencies using an automated employment decision tool to evaluate candidates or employees who reside in New York City, and it imposes three obligations.

An annual bias audit conducted by an independent auditor, evaluating potential disparate impact by sex, race, and ethnicity. Publication of a summary of the results clearly and conspicuously on the employer's website. Notice to candidates before the tool is used, identifying the job qualifications and characteristics the tool assesses and informing them of their right to request an alternative process. Civil penalties reach $1,500 per violation, and each day a tool runs without a valid audit counts separately.

I covered the requirement in detail in the Local Law 144 bias audit walkthrough. This piece takes the narrower question the Comptroller's finding raises: which artifacts does an employer actually need to hold.

The evidence the independent auditor consumes

The bias audit is arithmetic performed on records. An auditor computes selection or scoring rates by demographic category and compares them, which requires a dataset of tool outputs joined to candidate demographic data across the audit period.

Where that dataset comes from is the question most deployments answer badly. An HR system holds the outcome that a recruiter recorded. It rarely holds which tool version produced the score, when the tool ran, what input the tool received, or whether the tool ran at all for a given candidate. An auditor handed a spreadsheet exported from an applicant tracking system is auditing the recruiter's records of the decision rather than the tool's behaviour, and the two diverge whenever a recruiter overrode a score or the integration failed silently for a week.

Where an AI gateway sits, and where it does not

Most of Local Law 144 sits outside anything a proxy on HTTP traffic touches, and it is worth being blunt about that before claiming the part that does.

The disparate-impact calculation is statistical work performed by an independent auditor on demographic data, and no gateway performs it or improves it. Auditor independence is a procurement and governance question. The published summary is a web-publishing obligation. The candidate notice is a recruiting-workflow change touching job postings, application flows, and the alternative-process path. None of that is an architecture problem.

The in-scope slice is narrow. Where the AEDT is a model reached over HTTP, whether that is a hosted LLM scoring a resume or a vendor scoring API, every invocation is a request leaving your boundary. A record produced on that path establishes which candidate record was sent, to which tool and version, at what moment, under which policy, with what returned. That record is the substrate the bias audit consumes, and it is the only version of it that the tool's own vendor did not author.

The four questions an investigator asks

That fourth row is the one that catches people. The notice obligation is time-ordered, and evidencing it requires knowing when the tool actually ran for that candidate. An HR system records when a notice template was sent. Joining that to the first AEDT invocation for the same candidate is what turns two separate claims into one demonstrable sequence.

Why the record's author matters here

A vendor supplying an AEDT will offer usage reports, and those reports are worth having. They are also produced by the party whose tool is under examination.

The same structural problem applies as anywhere else: a record written by the system under review can omit the unsuccessful path, be modified by whatever failed, and disappear when a process dies mid-transaction. Under Local Law 144 the specific exposure is a vendor report that undercounts invocations, which understates the population your bias audit should have covered and does so in the direction that flatters everyone. The general argument sits in You Own the AI Liability, Not the Vendor, and the liability point transfers cleanly: the employer carries the obligation, not the tool vendor.

My candid view: the Comptroller's 17-versus-1 finding will produce a wave of consultants selling better bias-audit reports, and the report was never the weak link. The weak link is the dataset the report is computed from, which in most deployments is assembled after the fact from an applicant tracking system by someone reconstructing what the tool probably did. Fix the record and the audit improves as a side effect.

DeepInspect

This is the record layer an AEDT deployment lacks. DeepInspect sits inline between your systems and the model APIs they call, as a stateless proxy the calling application has no custody over. It evaluates identity, request classification, and destination on every call, enforces per-role and per-route policy with a fail-closed default, and writes a signed, tamper-evident per-decision record.

For a Local Law 144 deployment where the tool is reached over HTTP, that record establishes which candidate reference went to which tool version at which moment, independent of the vendor supplying the tool and independent of the applicant tracking system. The bias audit still belongs to an independent auditor, and the notice still belongs to your recruiting workflow. The evidence underneath both stops being a reconstruction. Book a technical deep dive at deepinspect.ai.