In brief
Mercor tested 12 licensed CPAs on four simplified month-end close scenarios; frontier models scored perfectly.
In Digits' transaction test, the top one-shot model scored 79.5%, versus a 79.1% human baseline.
Client context, unequal access to historical records and vendor-run testing limit broader conclusions.
Two tests answer different questions
Two new accounting benchmarks offer a reason to test AI more seriously, and a reason to read the methodology before making staffing decisions. One found a large performance gap on tightly defined close tasks. The other found general-purpose models roughly level with accountants who were unfamiliar with the client's books.
Those are not contradictory findings. Completing a structured file-based exercise and deciding how a particular business categorizes a transaction demand different information. Neither result establishes that AI is better at the whole job of accounting.
Mercor tested a narrow slice of the close
Mercor hired 12 licensed CPAs with roughly five and a half years of accounting experience to complete four month-end close scenarios. Participants had to search company working files, perform calculations and deliver a table of results. Frontier models scored perfectly on the tasks.
The important qualification comes from Mercor itself: the exercises emphasized detail-oriented file searches and close adherence to instructions, areas in which models perform well. Participants did not have accumulated client knowledge or coworkers to consult.
The tasks also included hard-to-spot requirements, and one missed number could substantially reduce a score. This is evidence about performance under those conditions, not a measurement of everything an accountant contributes to a live engagement. Client communication and the ability to clarify an ambiguous request were outside the test.
Digits found that business context changes the answer
Digits tested 2,000 real transactions from four small businesses, using accountant-reviewed classifications as the reference answers. Its best general-purpose one-shot model reached 79.5% accuracy, compared with a 79.1% human baseline. That 0.4-percentage-point difference is too small to support a sweeping claim of superiority.
When models could retrieve historical bookings, the strongest agent reached 88.9%. But those agents had access to information the outsourced human baseline did not, so the comparison is not simply software versus people with identical resources.
Digits reported 97.8% accuracy for its own ledger-native system. That is a vendor-reported result on the company's benchmark, not an independently established accuracy rate for every firm or client. The company sells the specialized accounting technology being evaluated.
Test the workflow, not the headline
The useful takeaway for firms is to identify a bounded task and measure it against their own reviewed work. Give the tool the context it needs, retain an exception process and track the time required to correct its output, not just the time required to generate it.
A model that can follow a detailed file trail may be valuable in close preparation. A categorization system may improve when it understands a client's historical conventions. Neither finding removes the need to decide whether those conventions are appropriate or whether the final financial statements make sense.
Treat these studies as hypotheses for a controlled pilot. The purchasing decision should rest on material errors, review effort and reliable completion in the firm's actual workflow, not a benchmark winner label.
