Why Careem Fixed Its Data Definitions Before Scaling AI Analytics

Dubai-based data platform leader Niyazulla Khan Pathan and his colleagues describe what it takes to turn a business question into a number that an analyst can reproduce.

In early 2024, Careem’s data team asked a language model a question about a food promotion. It wanted to know, by city, how many customers who placed their first food order during the promotion returned for another order within 14 days. The team supplied a table schema, descriptions and example queries. The model wrote plausible SQL, the language used to query the database. It chose the wrong date field, filtered on a status that did not exist and joined the data in a way that double-counted cities. Careem’s later engineering account describes an experiment whose output could have looked credible to someone seeing only the result. Careem Engineering’s account of the analytics programme

Each mistake points to a question a person should be able to answer before delegating the work. What makes an order count as someone’s first? Which event starts the 14-day clock? Which city belongs to the customer or order? The team’s account shows how those questions took it from cleaning up the warehouse to governing the calculations used by dashboards and assistants.

At a 2026 dbt Summit session, Careem data platform engineering manager Niyazulla Khan Pathan and lead software engineer Akash Srivastava described a related workflow for turning analyst requests into tested changes to data models. Pathan lists Dubai as his base. The session gives a named practitioner’s account of one part of the work. Careem’s separate engineering article describes the broader analytics programme, and does not identify Pathan as the author or sole owner of those decisions. Pathan’s professional profile, dbt Summit session record

Put the business meaning where people can find it

Careem describes reorganising its warehouse into data marts, which are groups of curated tables designed for particular business uses. Its grocery mart, for example, divided the work into orders, customers, supply chain and catalogue. The team adopted a naming convention that describes what a table contains and how it represents data. That helps an analyst tell whether a table records individual events or already summarises them. The distinction matters because a join between two event tables can multiply rows even when both tables are correct.

Careem UAE’s July 2024 Quik Electronics reel gives a customer-facing example of one kind of order within the app.

A shopper recognises a product and orders it. Behind that interaction, a team might need separate records for what was in the catalogue, when the order was placed, whether the item was available and whether the delivery was completed. If an operations manager asks whether a promotion led to more completed orders, counting catalogue visits or orders submitted answers a different question. The reel shows a historical promotion. Careem’s engineering account supplies the evidence about how the data team organised the warehouse.

The company says it also introduced a weighted scorecard with named owners. The sequence it describes included agreeing priority metrics with stakeholders, getting sign-off from the person responsible for the data and then adding quality tests in priority order. Deleting obsolete tables counted toward progress. Careem reports that its finance domain went from roughly 2,700 legacy tables to fewer than 800. That is the company’s account of the size of a cleanup. It does not show how much staff time the change saved or how accurate the assistant became.

A smaller business can use the same reasoning without a large warehouse project. Pick one decision people regularly make and trace the numbers that inform it. Identify the source that owns each event and the people who can explain the exceptions. If two reports use different definitions, keep enough detail to establish whether they answer different questions or whether one of them should be retired. Retiring a report also means finding the spreadsheet, dashboard or meeting that still depends on it. Otherwise the old number can stay in circulation long after the team agrees on a replacement.

Move shared definitions out of individual dashboards

Careem’s engineering account describes a problem that persisted after the warehouse cleanup. Analysts still embedded metric calculations in individual dashboards. Even if two people started with the same curated tables, they could apply different filters, joins or rules. Careem used the term federal metrics for measures reused across businesses. Agreeing what those measures meant was one step. Making dashboards use the same implementation took separate work.

The team describes investing in an open semantic layer using Cube. In this context, a semantic layer exposes approved metrics, dimensions and relationships so that a reporting tool or assistant can request a measure without recreating its calculation from raw tables. A common definition can survive a change of dashboard software or AI interface, provided those tools actually use it. Careem reports that integrating the layer required engineering work, including fixes to its business intelligence integration and contributions to Cube. That is a useful part of the sourcing story. Moving a calculation out of a dashboard creates a shared dependency that still has to be maintained. Cube’s technical explanation of governed AI analytics

For a business owner, the decision is which numbers deserve that treatment. A measure that determines staffing, a partner payment or a decision to renew a promotion needs an owner and a stable definition. A one-off analysis can still have value if the reader knows what it measures and when it should expire. Requiring the same approval for every experimental query can send analysts back to their own copies. Leaving every measure local creates a different problem when an experimental number becomes the one executives use each month.

A usable definition records the event being counted, the level at which one row is unique, the time and location rules, any exclusions and the owner. It should also contain an example that exposes a common disagreement. For a repeat-order measure, the example might show a customer whose first completed order was followed by a cancelled order. The business owner can decide whether that second attempt counts toward the measure. Engineers can then test the definition against the same case instead of trying to resolve the question after a dashboard changes.

Give the AI a specification before it writes code

The project Pathan and Srivastava presented addresses a related problem. It uses an agent to help prepare proposed changes to data models. The summit record describes a workflow that checks whether generated SQL compiles, tests data properties such as uniqueness and relationships, and validates orchestration and lineage before a human reviews the change. In the organiser’s recap, the presenters describe a step that requires clarification about what each row represents, which metric is intended and which filters apply before the agent starts writing. dbt Labs’ recap of the Careem team’s presentation

That is a different job from answering a manager’s question in a chat window. A change to a data model can affect reports that depend on it later. Before the team merges such a change, it needs to know that the code behaves as specified and that a human understands the proposed business logic. In a chat interface, the comparable question is whether the answer is grounded in an approved measure and whether it can show enough about its scope for someone to check it.

The presenter account also distinguishes between governed metrics and temporary outputs for one-off requests. The recap says temporary work can be promoted into a proper mart or cleaned up after 30 days. That separation allows analysts to explore a new question without automatically giving the result the status of a company-wide measure. A data leader still needs someone to decide when an experiment has become important enough to maintain.

Compare tools on the same question

Before widening access to an AI analytics tool, ask two people from different teams to answer the same business question using the existing system. Look at their definitions and filters as well as their results. If the answers differ, identify who can decide which version is appropriate and how the distinction should be shown to users. A procurement demo can then use the approved measure and a set of cases with known expected answers.

A useful comparison includes a question with missing context, a request for a breakdown the user is authorised to see and one for data outside their access. Evaluate whether the assistant asks for clarification, respects access restrictions and returns a result that can be reproduced. Include a source update in the test. The team should be able to show when a revised definition takes effect and whether it changes only new results or also changes comparisons with prior periods. Access and audit evidence can be evaluated separately from how fluent the answers sound.

Keep the test questions and expected answers with the definitions they exercise. When someone changes the treatment of refunds, reviewers should be able to see which answers are supposed to change. That makes a test failure actionable. If the new number is correct under the revised definition, the work may be to update a report label or explain a break in the historical series. If the expected number is unchanged, the team can investigate whether a join, filter or tool configuration introduced the discrepancy. This record also gives the next vendor the same test as the current one.

For example, a Dubai delivery team assessing a promotion could set aside a small dataset in which some people made multiple purchases, some orders were cancelled and one person switched delivery cities. Two tools should get the same instructions and face the same edge cases. Compare their results with the answer an analyst derived from the agreed definition. If one tool returns an unexpected number, check whether it misunderstood the request, called the wrong metric or got a flawed result from the data layer. Each failure belongs to a different owner and calls for a different fix.

Careem’s 2024 experiment mattered because it exposed mistakes the team could identify before the result reached a decision-maker. Pathan and Srivastava’s later account shows one way to carry that discipline into changes that affect real reporting. The operational gain a data leader can look for is simple to verify. The person acting on a number should be able to establish what was counted, who owns the definition and whether a second person can reproduce it.