AI Usage Mandates and Quota Gaming

Sources#

• AI in UX: Work Is Faster, But Practitioners Have Mixed Feelings
• Uber's AI infra from the inside (with Uday Kiran Medisetty)
• When Less Is More: Managing AI Adoption with Adaptive Incentive Design

Summary#

Firms increasingly count AI use, put it in performance reviews, and sometimes set a minimum. The press calls the response tokenmaxxing: workers inflate the count with repeated prompts, trivia and off-task queries. Until now that was anecdote. Gong, Hou, Li, Pu & Yao (University of Hong Kong, arXiv 2609.32859, empirical) measure it from the inside. They hold the full text and timestamp of every query at one firm, so they can tell work use from filler. The firm also changed its target branch by branch at different times, which lets them identify what relaxing the target removed.

Three results, in order of evidential strength:

1. The mandate worked on the extensive margin and was gamed on the intensive one (descriptive, firm-wide, no control group). Before the mandate, 14.8% of employees had ever submitted a query. Under a 200-query monthly minimum the figure reached 46% by the end of April and 59.1% by the end of June. Over the same months, 31% of counted queries were repeated or personal/off-task. Among active employee-months, 12% landed on exactly 200 and 95% at or below 220. First-half workdays each carried about 3% of a month's queries and the final workdays 7–8%.
2. Cutting the target to 100 removed mostly filler (causal, staggered difference-in-differences across 42 branches, ~44,000 employee-months). Monthly queries fell 28.69 per employee, 30% of the sample mean. Repeated or off-task queries fell 25.97, about 90% of the drop. Non-repeated work-related queries moved −2.72 (SE 4.62, wild-bootstrap p = 0.585). Queries scoring the maximum 10 on a 1–10 work-relevance scale rose by 11.5 a month.
3. Less measured use came with more output (causal, but only for the ~500 salespeople in eight branches). Log monthly sales rose 0.07 (SE 0.03), which the paper reports as a 7% increase. The effect is robust to wild-bootstrap inference (p = 0.031, 95% CI [0.01, 0.14]) and to dropping one sales branch at a time.

The paper's own framing is that a mandate is neither a success nor a failure, because it is both. The design problem is how to revise a target once early implementation shows how workers respond, not whether to set one.

The setting#

• Firm. "MedTech Corp" (a pseudonym), a medical-device company in China with ~9,000 workers in 46 branches. About 5,000 are non-production staff (sales, R&D, technical support, administration) and fall under the mandate. Four branches with fewer than 30 non-production employees are excluded, which leaves 42; the findings hold with all 46.
• Tool. An internal chat assistant built on DeepSeek models and fine-tuned on the firm's technical documentation, customer records and regulatory archives, opened to all non-production staff on 2025-03-01. The paper's motivating example: a salesperson can build a hospital-account profile and pitch in about 10 minutes instead of 40–60 with manual search.
• The mandate. The CEO's office issued it on 2025-03-31: at least 200 queries a month. A first miss drew a formal warning. Repeated misses drew a RMB 200 fine, about 2% of average monthly pay or 7% of average monthly performance pay. The interface showed each employee's running monthly count but never the target.
• The adjustment. Three months in, branches reported that 200 was time-consuming and hard to reach. Headquarters cut the target to 100 in July 2025. It sent the change as a memo rather than a formal notice, so local HR teams adopted it at different times: 19 branches in July, 2 in August, 3 in September, none in October, 2 in November and 1 in December. Fifteen branches enforced 200 through December. Branch-month bunching (Figure A2) confirms the rollout: each branch's spike sits at 200 until it switches and at 100 afterwards.

The adaptation was not informed by the diagnostic. "Headquarters did not review usage or query content before making this decision." The firm cut the target over workload complaints, not because it saw the gaming. So the case shows what relaxing a gamed target does. It does not show a firm learning from its own logs, which is the practice the paper ends up recommending.

How the paper decides what counts as gaming#

The quality outcomes combine two text measures, and their definitions decide every headline number:

• Repeated. The query exactly duplicates one of the same employee's earlier queries in the same month. This is conservative: near-duplicates and paraphrases count as non-repeated.
• Work relevance. An LLM classifier (DeepSeek-V4 Pro, temperature 0, schema-enforced JSON, run 2026-05-13 to 05-17) sorts each query as clearly work-related, generic or weakly work-related, or personal/off-task. It sees the query text plus the employee's position, branch, department and the firm's line of business. It never sees the date, the target regime, treatment status or performance. On 1,000 queries labelled by three people, Cohen's κ against the human-majority label is 0.815, and Fleiss' κ is 0.872 among the humans and 0.843 with the model added as a fourth annotator. Most disagreements fall between the two work-related classes.
• Low quality means repeated or off-task. High quality means non-repeated and clearly or generically work-related. Generic queries are about 3% of all queries, and counting them as low quality instead leaves the results unchanged.

Two robustness instruments sit beside the classifier. The first is an embedding index with no LLM judgment in it: BAAI/bge-large-zh cosine similarity to the firm's description and to the employee's role. Lowering the target raises it by 0.074 on a 0–1 scale. The second is a Poisson count model. Its coefficient on repeated/off-task queries is −1.13, which converts by the paper's own 100·[exp(β)−1] rule to a ~68% cut in filler in conditional-mean terms, against −0.36 (~30%) for all queries and −0.07 (n.s.) for work queries. This page made that conversion; the paper prints only the coefficients.

What disappeared when the target fell#

The pooled estimate hides a gradient, and the paper reports it:

• By relevance. With pre-treatment relevance cutoffs held fixed, query counts fall in each of the bottom four quintiles, most steeply at the bottom, and rise in the top quintile. Quantile treatment effects on the relevance score are largest at the 10th and 25th percentiles and small in the upper tail. Prompt clarity rises by 0.117 on its 0–1 scale. Intent shifts away from Asking (−31.00) and Expressing (−1.08) and toward Doing (+3.37), the paper's term for asking the AI to produce something.
• Near-threshold users carry the effect. Employees who logged 180–220 queries in at least one month under the old target cut total use by about 59 a month (an interaction of −60.67 on a main effect of +1.39). For employees whose use was not anchored near 200, total use barely moves, because filler fell 12.88 while work queries rose 14.27. That subgroup is the cleanest instance of the substitution the title promises.
• The subgroup figures complicate the headline. The paper's summary is that work-related use "did not show a comparable decline". But adding main effects and interactions in Table A6 (this page's arithmetic; the paper does not report the combined estimate or its significance) gives a work-query change of about −20 for near-threshold users (14.27 − 34.26), about −15 for employees who had used the tool before the mandate (0.18 − 15.49) and about −12 for managers (−0.93 − 10.97). The pooled −2.72 is close to zero partly because a rise among unanchored users offsets real cuts among the employees the target bound hardest. Adding branch-specific linear trends gives the same reading: the pooled work-query estimate becomes −8.68, marginally significant (Table A10), and the paper acknowledges this in a footnote. The defensible claim is the paper's decomposition claim: most of the decline is filler. "No productive use was lost" is a stronger claim than the evidence supports.

Identification, and how far it carries#

The design is a two-way fixed-effects difference-in-differences with employee and month fixed effects and standard errors clustered by branch. It is backed by a Sun–Abraham interaction-weighted estimator (−31.39 / −26.72 / −4.67 / +0.08), 1,000 placebo permutations of branch timing (actual estimates in the tails), results that hold after dropping the switch month and after restricting to employees who stayed, and a Cox model showing that baseline branch characteristics do not predict switch timing (joint Wald p = 0.670). Ever-switching branches had somewhat higher baseline use than never-switching ones (88.2 vs 67.5 queries a month, p = 0.20; joint test of all baseline differences p = 0.49). That difference is not significant, but it points the way a concern would.

Limits that bound every use of the result:

• The mandate's own effect is not identified. All branches got the 200-query target on the same day, so the first-use jump has no control group. Some of it could be diffusion that would have happened anyway. The paper says so: its evidence "concerns the effect of lowering an existing target" and does not determine the optimal level.
• The sales effect sits on a thin base. Eight branches, ~500 salespeople, 6,064 employee-months. In the event study (Figure 4d), the pre-period coefficients sit slightly above zero relative to the omitted month k = −1. The paper calls this "no visible pre-trend", which is fair given the confidence intervals, but the reference month is a little low against its neighbours. The leave-one-branch-out plot is the stronger evidence.
• Queries are observed, use of the answers is not. The paper cannot say whether anyone acted on a response. The sales channel (less time spent filling the quota, better remaining use, less month-end catch-up) is reduced-form and not decomposed. Appendix Table A4 adds a fixed-effects correlation, not a causal estimate: filler is negatively associated with sales (−0.00025 per query) and work queries positively (+0.00018).
• One firm, one country, one tool, one fine schedule. The penalty was mild (2% of pay) and the interface hid the target. Both plausibly shape how much gaming a target induces, and neither varies here.

Where it sits in the vault#

• Demonstration versus mandate. Standardize the Infrastructure, Not the Tools (Shopify: "didn't mandate AI adoption. He modeled it") and Pilot-to-Production Gap (Anthropic × Accenture: engineer the demonstration moments deliberately) both prescribe demonstration over mandate, and neither measures either arm. This is the first measured mandate in the corpus, and it partly vindicates and partly refutes that prescription. Mandating did overcome inertia that availability alone did not: 14.8% voluntary uptake in month one. But a third of what it produced was manufactured. The comparison the prescriptions imply, demonstration against mandate at matched firms, still has no instance.
• The adoption instrument becoming a performance instrument. Psychological Costs of AI Adoption's SoftHouse encouraged adoption and never mandated it, yet its five-level maturity ladder was "received as a performance instrument", and practitioners described using AI "by fear". MedTech went the whole way and attached fines to a count. Read together, the two cases show what happens to the measurement under each design. SoftHouse's instrument cost trust and could only ever report progress. MedTech's instrument cost about a third of its own validity. The paper's prescription, monitoring "not only aggregate usage but also its content", is the instrument-side counterpart of the SoftHouse authors' advice to "decouple adoption expectations from individual measurement".
• Goodhart exposure, measured. Telemetry vs. Survey Measurement carries the warning that "a population that knows its adoption is measured can perform adoption: invocations without reliance", stated without evidence in a practitioner-opinion paper. This page puts a number on it for one population: 31% of counted interactions under a binding target, 18% under a looser one. It also shows that bunching at the threshold, month-end timing and exact-duplicate rate are cheap diagnostics a usage log already contains.
• The opposite instrument: a spend ceiling. Uber (Uber's AI infra from the inside (with Uday Kiran Medisetty), case-study) meters the same thing in the other direction. Each engineer gets a monthly spend tier, a manager can sign off on more, and a plugin coaches people on their cost profile. A floor rewards volume and, as measured here, gets filler. A ceiling rewards economy and could plausibly get under-use. Nobody has measured the ceiling arm, so the corpus has no figure for what a cap suppresses. The two designs also differ in who decides the exception. Uber leaves it to a manager, engineer by engineer. MedTech set its target centrally and changed it branch by branch.
• The mandate seen from the receiving end. NN/g's survey of 521 UX practitioners (empirical self-report, convenience sample) describes the same dynamic in a population with no query logs. 24 of 149 coded negative comments cite mandated or pressured adoption. Respondents say their organizations "mandated AI and tracked their usage, which led to performative use," and the authors trace a consequence MedTech's data could not reach: the performed output became work for colleagues, who fix "the slop of others AND do our own work right" (see Task-Level Speedup, System-Level Cost). Pressure, not reluctance, is the leadership problem in that sample. "Lack of executive buy-in" was the least-chosen barrier to adoption (6%, against 54% for output quality). This is perception data, and it attaches no number to how much of the counted use was performed.
• What a usage count measures. The Enterprise AI Adoption Gradient reads adoption depth from messages and weekly active users per employee. Those counts are unbiased only where nobody is targeted on them. Under a binding minimum, the same count measures compliance, and the share of it that reflects work rose from 69% to 82% when the target was relaxed.

Connections#

• Standardize the Infrastructure, Not the Tools — the demonstration-not-mandate thesis gets its first measured counter-case. Its open question about what per-team usage analytics are used for now has an observed instance of the "performance evaluation" use, and the instance's cost
• Pilot-to-Production Gap — "deployment is not adoption", measured at the moment adoption is forced. A mandate closes the deployment–adoption gap on the count and reopens it on content
• Psychological Costs of AI Adoption — the opposite adoption regime (encouraged, measured by a maturity ladder) at a comparable-sized firm. Its practitioners' reading of the ladder as a threat is the motive this page's workers acted on
• Telemetry vs. Survey Measurement — that page's untested Goodhart warning about usage telemetry, measured here, plus the log-native diagnostics (threshold bunching, deadline timing, exact repeats) that detect it without a survey
• The Enterprise AI Adoption Gradient — per-employee message and WAU counts are this paper's raw outcome before content filtering. The comparison bounds how much of a usage count can be compliance when a target binds
• Uber — the mirror design: a per-engineer spend ceiling with manager override instead of a usage floor with fines, unmeasured
• Task-Level Speedup, System-Level Cost — where gamed or performative use lands: on a colleague who has to check and fix it, a cost no usage count records
• Reward Hacking — the same failure in the other substrate: an optimizer, here a workforce, satisfying the measured proxy instead of the intended objective. The paper's own lineage is Kerr's "rewarding A, while hoping for B" and Holmström–Milgrom multitask agency, not the ML literature

Open Questions#

• Would a target that counts only content-filtered queries, for example only non-repeated work-classified ones, still be gamed by work-shaped filler that passes the classifier? The paper's prescription is to monitor content, and Goodhart predicts that gaming then moves to whatever the classifier accepts. Settler: a firm that enforces a quality-filtered count and audits a sample of passing queries by hand.
• Is there a target level low enough to lose the mandate's first-use gain, so that the adoption-versus-gaming trade-off has an interior optimum? The paper cannot identify the mandate's own effect, because every branch got 200 on the same day. Settler: a firm that varies the introduced target across units (none / 50 / 100 / 200) and tracks first use and filler share together.

Sources#

• When Less Is More: Managing AI Adoption with Adaptive Incentive Design — Jie Gong, Jiayi Hou, Jin Li, Fei Pu & Xinjue Yao (Faculty of Business and Economics, University of Hong Kong; HKU Centre for AI, Management and Organization), When Less Is More: Managing AI Adoption with Adaptive Incentive Design, arXiv 2609.32859 v1 (2026-09-26; the title page is dated September 29, 2026), 57 pp, empirical. A single-firm field study using administrative query logs, personnel and sales records, with a staggered branch-level difference-in-differences. Cited for §2 (setting, mandate, fine, rollout), §3 (measures, classifier validation, design), §4 (first use, bunching, 31% low-quality share, month-end timing), §5.1–5.5 (descriptives under 100, Table 1, Figure 4, heterogeneity, robustness), §6 (limitations), Appendix Tables A1, A2, A3, A6, A7, A10, A11, A12 and B2, and Figures A1–A3. Figure 4 was viewed under the inline-image rule, and the sales pre-period note above is read from it. Parse warning: docling dropped cells from several regression tables. Table 1's column (4) coefficient (0.07**) is missing and its CI/SE and Observations rows are welded. Table A11 (Poisson) lost its column (1)–(2) coefficients (−0.36***, −1.13***). Table A12's bootstrap p-values and CIs are merged into one cell. Every regression figure on this page was reconciled against pdftotext -layout of the local PDF (Table 1, A6, A11, A12), and the Table A6 subgroup sums are this page's arithmetic, not the paper's. COI: none apparent. The authors are academic, the firm is pseudonymous, and no vendor's product is evaluated; the assistant is the firm's own build on DeepSeek models.
• Uber's AI infra from the inside (with Uday Kiran Medisetty) — Orosz with Uday Kiran Medisetty, The Pragmatic Engineer podcast page, 2026-10-14, case-study (first-party interview, newsletter-summarized). Cited only for takeaway 6, the spend tier with manager sign-off and the CostCheck plugin
• AI in UX: Work Is Faster, But Practitioners Have Mixed Feelings — Maria Rosala & Rachel Banawa (Nielsen Norman Group), 2026-10-23, empirical (self-report convenience sample of 521 UX practitioners). Cited for the mandated-or-pressured-adoption comment count (24 of 149) and the barriers chart (both viewed), and the performative-use comments