How to Verify an AI Productivity Claim

A UAE service company says its AI assistant cuts processing time by 30%. The figure might mean that a draft appears faster, that an employee spends less time on a case or that customers get a correct result sooner. Those are different claims. If the company timed the first draft and a supervisor still has to check it, a buyer cannot use the percentage to estimate staffing or cost savings. A reader needs to know what was counted and how the comparison was made.

Start by rewriting the headline as a sentence someone else could verify. For example, “The Dubai support team completed more eligible cases per paid team hour during an eight-week comparison, with the same quality checks.” Each detail points to evidence you can ask for. If the claimant cannot specify the team, task, period and unit, the most defensible conclusion may be that a narrow improvement was observed and its effect on wider work remains open.

Find out what the number actually measures

Ask for the numerator and denominator in plain language. Output could mean drafts, closed tickets, correct decisions or invoices accepted by the finance system. Hours could mean application time, active employee effort or all paid hours for the team. Establish whether review, escalations and repeat work are included. A claim based on the time a model takes to respond has little bearing on how many cases a shift can complete.

Also check whether the figure describes fewer minutes per case or more cases per hour. In a simple example with identical work and a fully occupied employee, a task that falls from ten minutes to seven takes 30% less time. The implied rate moves from six cases per hour to roughly 8.57, about 43% higher. That second figure assumes the employee can fill the entire hour with the same task and that quality holds. Do not carry the implied rate into a staffing forecast without testing those assumptions.

For an external claim, find the earliest available study or case note. Check the publication date, the version of the tool and whether the figure comes from a vendor, a customer or an independent evaluator. If a release quotes a study, use the figure and wording in the study. Label a self-reported estimate as an estimate. Where the underlying data are private, say which parts of the claim are documented and which the reader cannot independently reproduce.

Check what would have happened without the tool

A before-and-after comparison can establish that the number changed. Several things may have changed at the same time. The company might have hired experienced staff, moved routine work to a central queue or changed which customers it serves. Ask for a comparison that could separate the tool from those changes. The NIST AI Risk Management Framework calls for human baseline measures where AI is intended to augment or replace human activity. That principle is useful when examining a productivity claim about people working with an AI service. NIST on framing AI risk and human baselines

A contemporaneous comparison is often stronger than comparing two different seasons. Ask how cases reached the assisted and unassisted teams. Random assignment can help where it is practical and does not compromise service. A staged rollout across similar branches may provide another comparison if the branches are measured consistently and differences are accounted for. When employees choose whether to use the tool, compare their case mix and experience. Staff may select AI for simple requests and retain the hard ones, or do the reverse.

Ask whether the analysis includes every eligible case or only cases in which the tool was used successfully. Report assignment and actual use separately where possible. If the AI service is available on a shift but agents bypass it on ambiguous requests, the result for the whole shift captures a different question from the result for completed AI sessions. Both can be useful when described accurately.

Digital Dubai's survey participation carousel illustrates the collection stage of evidence. In a separate announcement for its 2026 Labour Force Survey, Digital Dubai lists areas such as employment, occupations and economic activity. Those categories describe the labour market. Verifying a workplace AI claim requires a defined workflow and a comparison that measures what changed in that workflow. Keep the level of the claim matched to the level of the data. Digital Dubai Labour Force Survey announcement

Recalculate the result with the full workload

Consider an illustrative UAE claims team. The baseline and assisted groups each receive 500 comparable cases. Drafting time falls from ten to seven minutes per case, a 30% reduction. The manual workflow takes 15 total minutes per case after review and correction. The assisted workflow takes 14. Across the 500 cases, the difference is 500 minutes, or eight hours and 20 minutes. The improvement in total active handling time is about 6.7%.

Now count correct cases after a fixed follow-up window. Suppose 480 of the manual cases and 450 of the assisted cases meet the same standard. Active handling effort totals 125 hours for the manual workflow and approximately 116.67 hours for the assisted workflow. The manual group produces 3.84 correct cases per active hour. The assisted group produces about 3.86. A reader can report the drop in drafting time. The broader claim about correct output per hour is much smaller in this example, and the lower correct completion rate needs investigation before anyone relies on it. The completion rates differ by six percentage points, which is a separate finding from the relative change in time.

Check whether the same people did the checking and whether they saw which tool produced the work. An employee may correct a polished answer less carefully than a rough draft. Sample accepted cases and have qualified reviewers apply a common standard to both groups. Agree whether a case reopened during the follow-up window loses its earlier status. If the quality audit covers a sample, report that sample and how it was drawn alongside the full production counts.

The aggregate may also conceal who benefited. Ask for results by task type, experience, channel and language where volumes allow. A UAE service team may see different patterns for formal Arabic messages, short bilingual chats and English email. Describe sample sizes and uncertain results. A slice with five cases can help identify an issue worth testing. It supplies a weak basis for extrapolating a stable percentage across all customers.

Separate freed capacity from realized savings

If time was saved, ask where it went. A team may handle a larger queue without overtime, improve the speed of the service or let experienced staff handle escalations. Those outcomes have value if the company can document them. A calculation that multiplies minutes by a loaded staff rate estimates capacity value. To claim a reduction in cash spending, identify the spending that fell and account for the costs of the tool, implementation and support.

Suppose the illustrative team values eight hours and 20 minutes of freed time at AED 120 per hour, or AED 1,000. If monthly usage and extra support cost AED 1,400 for that workload, the modeled capacity value is AED 400 below the incremental cost, before setup. The lower correct completion rate also needs to be addressed. If the time estimate included routine human checks, keep those checks in the workflow calculation and avoid charging them again as an incremental expense. Include any downstream rework omitted from the measured time.

FinOps Foundation guidance connects technical unit costs to business outcomes, including measures such as cost per case. For a management review, ask for the amount spent per correct output and a separate account of where cash changed. The first helps compare operating efficiency. The second supports a claim about the company's expenditure. FinOps Foundation on unit economics

Write a conclusion that matches the evidence

When you publish or brief the claim, name the organization or describe the scope if it cannot be disclosed, identify who measured the result and state the exact unit. Include the period, case count and comparison. If the result comes from one department, say so. If the tool changed during the trial, describe which configuration the result covers. Keep the uncertainty close to the number so the reader does not have to discover it in a footnote.

Request the source table behind any weighted average. Two teams can each improve while the overall result falls if a greater share of work shifts to the slower team. Conversely, a favorable case mix can lift the headline even if performance within each group is flat. When the proportions differ, ask for a comparison under a common mix or report the groups separately. Reconcile the totals with the source records before using a summary number in a decision paper.

Check which statistic the headline uses. A median can improve while the longest cases get worse, and the average can be moved by a few unusually costly failures. Ask for a measure of the long wait, such as the time below which 90% of cases finish, and count cases still open at the end of the period. For a support queue, a fast result for most cases may coexist with a growing backlog of the cases customers care most about.

A statement for the hypothetical claims team could say that drafting took 30% less time across 500 assisted cases than across 500 comparable manual cases, while active effort fell by about 6.7% and correct completion dropped from 96% to 90%. It can then say what remains unresolved, including whether the case comparison accounts for differences between groups and whether the quality gap persists after changes. A stronger claim about productivity or savings would require better evidence about completed work, costs and the alternative process.

Decide what action the evidence supports. A precise improvement in a drafting stage may justify another trial. A credible increase in correct output across comparable shifts may support a limited expansion. A claim about the economics of the whole company requires coverage of more than one team and evidence that downstream work and costs moved as reported. The wording should preserve those boundaries so the next reader can reconstruct the decision.