Zylos LogoZylos
2026-11-11

Which Language Does the Agent Read? Separating Agent-Facing Text from User-Facing Locale

i18nprompt-engineeringai-agentslocalizationtesting

Executive Summary

A persistent agent runtime produces two very different kinds of text. One kind is read by a language model that will act on it: system instructions, hook-injected context, CLI output formats, handoff notes, error messages. The other is read by people: dashboard labels, notifications, chat replies. Classic i18n practice only ever had to worry about the second kind. Agent runtimes add a third reader, the LLM, and a fourth category of strings that are read by both because they pass through the agent on the way to a human.

This article argues for a simple rule and then tests it against the literature and engineering practice: every string has exactly one declared reader, and the language of a string follows its reader. Agent-facing text lives in one language (in practice English, for reasons covered below) and is never routed through a locale catalog. Human-facing text is localized through a catalog keyed by stable identifiers. Text that is relayed by the agent is generated by the agent, in the user's language, from agent-facing inputs that stay neutral.

The main findings:

  • The evidence for "write agent-facing text in English" is real but narrower than folklore suggests. Current vendor documentation reports knowledge-benchmark (Global-MMLU) scores for major languages that are close to each model's own English score. Older research shows English-centric behavior inside models, token-count disparities across languages, and language-confusion failures. The strongest justifications today are operational: predictability, token budget, shared tooling, grep-ability and one language for reviewers, not a claim that a modern model cannot read Chinese.
  • Most real bugs are not about comprehension. They are about leaks. A locale environment variable changes a parsed message, a fallback chain silently surfaces a different language, an agent quotes internal-language text to a user, or a test asserts on a translated sentence.
  • The boundary is enforceable. A script-aware regex over agent-facing modules, a fixed allowlist, pseudo-localization on the human side, and snapshot tests of injected context catch nearly all regressions cheaply.
  • Reply language is a separate axis. Keep instructions in one language and tell the agent, explicitly and from a per-user preference, which language to answer in.

A caveat on scope: this is a design article. The empirical claims come from cited sources and a few small local demonstrations on one Linux host. Where a number is uncertain, it is marked as such.

1. Why the Language of Agent-Facing Text Matters

1.1 What the research says

Early multilingual LLM work found strong but uneven behavior. Shi et al. introduced MGSM, 250 grade-school math problems translated into ten languages, and found that chain-of-thought reasoning transfers across languages, including lower-resource ones, as model scale grows (Shi et al., 2022). In the other direction, an evaluation of ChatGPT on seven tasks across 37 languages found worse performance than earlier task-specific models on various NLP tasks and languages (Lai et al., 2023). Those studies are about older models and about user-task language, not about the language of an agent's instruction layer, so they should be read as background rather than direct evidence.

Two results speak more directly to "which language should the model be thinking in":

  • Etxaniz et al. compared direct inference on non-English input with "self-translate", where the model first translates the input into English itself. Across five tasks self-translate consistently beat direct inference, which the authors read as evidence that models cannot fully use their multilingual ability when prompted in non-English languages (Etxaniz et al., 2023).
  • Wendler et al. traced Llama-2 internals on non-English prompts and found that middle layers can decode the correct concept while giving the English rendering higher probability than the input-language rendering. They interpret this as an abstract concept space closer to English than to other languages (Wendler et al., 2024).

Both studies concern models available when the work was done. Neither shows that a current frontier model follows Chinese system instructions worse than English ones. They do suggest a prior: when you have a choice and a model whose training mix is English-heavy, English instructions are the lower-risk default.

1.2 What vendor documentation says

Anthropic's multilingual documentation reports zero-shot Global-MMLU scores as a percentage of each model's own English score. In the version of the page consulted for this article, Claude Sonnet 5.5 scores 98.5% of its English result in Simplified Chinese, 98.4% in Japanese, and 93.6% in Yoruba; Claude Haiku 5.5 scores 97.7% in Chinese, 97.4% in Japanese and 87.0% in Yoruba. The page itself cautions: "Use the table to compare languages within a model, not to rank models. Differences of less than about 1 percentage point are within measurement noise." (Anthropic, Multilingual support).

Read within each model, as the page intends, these are knowledge-benchmark results: in both the Sonnet 5.5 and Haiku 5.5 columns, Chinese and Japanese score between 97.4% and 98.5% of English, while Yoruba is lower (93.6% and 87.0%). The table does not rank the models, and it does not show that model size changes how well a model understands non-English instructions. Nor does it say that agentic behavior (tool use, long-horizon instruction following, format adherence) has the same gap as multiple-choice knowledge questions. That is not measured by this table, and teams should run their own evals.

The same page contains two statements that matter more for runtime design than the table:

  1. Claude infers the response language from the conversation, but for production applications you should state the target language explicitly, and the most reliable place is the system prompt.
  2. If users pick a language at runtime, interpolate that choice into the system prompt rather than relying on the model to infer it from the user's message.

Claude Code applies the same idea in its own configuration: its language setting is passed verbatim to the model as an instruction to always respond in that language, and Claude Code does not validate the value (Claude Code settings reference). Reply language is therefore a configured instruction, not a property of the instruction language.

1.3 The failure that actually shows up: language confusion

Marchisio et al. built the Language Confusion Benchmark across 15 languages and found that models often fail to answer in the language the user wanted. Llama Instruct and Mistral models showed high confusion, even the best models did not respond in the correct language consistently, English-centric instruct models were more prone to it, and the problem grew with complex prompts and higher sampling temperature. Few-shot prompting, multilingual fine-tuning and preference tuning helped but did not eliminate it (Marchisio et al., EMNLP 2024).

For an agent runtime this is the most relevant finding. An agent with an English system prompt and a Chinese-speaking user, or a Chinese system prompt and an English-speaking user, is exactly the cross-lingual setting the benchmark probes. The model may reply in the language of the instructions, the language of the latest tool output, or the language of the user. Which one wins is a property of the model and the prompt, so the safe design is to state the reply language explicitly and to keep the other language sources uniform.

1.4 Token cost and context budget

Token counts differ by language. Petrov et al. report that for the same text translated across languages, subword tokenizers can produce token counts that differ by up to 15 times in some cases, and byte-level models still show more than a 4 times difference for some language pairs, with consequences for cost, latency and usable context (Petrov et al., NeurIPS 2023). Ahia et al. likewise found large variance in the number of tokens needed to convey the same content across 22 languages in OpenAI's API, with many languages overcharged while obtaining worse results (Ahia et al., 2023).

Those are cross-language extremes. The Chinese-to-English ratio on a particular tokenizer is not given by either abstract, and it depends on the model's vocabulary, which changes between generations. One local observation illustrates why the question is separate from characters: the six-character string 已完成,ok is 14 bytes in UTF-8, so any byte-oriented estimate overstates it, while a tokenizer with good CJK coverage may represent it very compactly. The honest recommendation is to measure the actual injected context with the actual model's token-counting endpoint, rather than assume either direction.

For a runtime that injects instructions and context on every session start, the budget matters because that text is paid for repeatedly. A standing directive file that is loaded each session is a per-session token cost, and any language-specific inflation is multiplied by session count.

1.5 Why "one language" is justified even when models are good at many

Putting the evidence together, the argument for a single agent-facing language is not "models cannot read Chinese." It is:

  • Predictability. Mixed-language prompts have more ways to behave unexpectedly (language confusion, per Section 1.3), and the one source of variance you control is the instruction language.
  • Tooling. Agents search, grep and pattern-match across instructions, memory files and tool output. A single language makes those operations reliable. Searching for the English word "pending" does not find a Chinese status label.
  • Review. A code reviewer who must read every injected string can review one language.
  • Model portability. A runtime that can run on more than one model family or tier inherits the weakest model's behavior. English is the lowest-risk default across them.
  • Measured budget. Token cost should be measured and then held stable.

If your team and your agent share Chinese as the working language and your evals show no loss, a Chinese agent layer can be a defensible choice. The point is to make the choice once, per reader, and enforce it.

2. Classifying Strings by Reader

The central discipline is to label each string with its reader before deciding its language. Three classes cover almost everything.

ClassReaderExamples in an agent runtimeLanguage policyLocalized?
Agent-facingThe LLM, and tools that parse outputSystem and project instructions; hook-injected session context; tool and skill descriptions; CLI output the agent parses; handoff and state notes; error text the agent acts on; scheduler task promptsOne fixed language (English)No
Human-facingA personDashboard labels, chart titles, buttons; notification text from the runtime itself; docs for operatorsThe user's localeYes, via catalog
Relayed (both)The agent first, then a personA message from a user that the agent summarizes; a status the agent reports; an error the agent explainsInputs neutral, output in the user's languageThe agent does the localizing at generation time

Three subtleties deserve mention.

Tool descriptions and error messages are prompts. Anthropic's guidance on writing tools for agents recommends prompt-engineering error responses so they communicate specific, actionable improvements, and avoiding opaque error codes or tracebacks (Anthropic Engineering, Writing tools for agents). That article also recommends returning high-signal fields and notes that agents handle natural-language names and identifiers better than opaque UUIDs. An agent-facing error therefore needs both a stable code for programs and a clear sentence for the model. The sentence is agent-facing text and belongs to the agent-facing language.

Language does not decide authority. A direct request from the user is an instruction at the user's authority, in whatever language it is written. Material the message carries, such as a pasted log, a forwarded message or an attachment, is data inside that message, not a further instruction. The trust level of a message role and the provenance of embedded material are properties the runtime tracks separately; neither the language of a string nor its formatting establishes them. For localization the useful split is narrower: keep the wrapper (sender, channel, timestamp, reply path) in the agent-facing language and leave the payload, the user's request and anything it carries, in the language it was written in.

Dates and numbers are part of the agent-facing format. RFC 3339 defines an unambiguous timestamp profile of ISO 8601, calling a date such as 10/11/1996 "completely unsuitable for global interchange because it is interpreted differently in different countries" (RFC 3339). In agent context, timestamps should be ISO 8601 or RFC 3339 with an explicit offset, durations should be numeric, and decimal and thousand separators should not vary. Localized date formatting belongs only in the human-facing rendering.

3. Patterns That Keep the Boundary Clean

3.1 Separate catalogs, not an if (lang) in the middle of a string

Human-facing text goes through a message catalog (gettext, ICU MessageFormat, Fluent, JSON per locale). Agent-facing text lives as plain constants or template files, outside the catalog. If agent-facing strings are in the catalog, translators and locale fallback can modify them. If human-facing strings are hard-coded, UI localization silently stops working. The two stores should be physically different directories, which also makes the lint rule in Section 5 a path check.

3.2 Stable codes plus localized messages

RFC 9457 (Problem Details for HTTP APIs) shows the same split in a protocol. The type URI is the primary identifier consumers must use; title is secondary and "SHOULD NOT change from occurrence to occurrence of the problem, except for localization"; and consumers "SHOULD NOT parse the detail member for information", with extension members recommended instead (RFC 9457). The RFC also notes that human-readable strings can be language-negotiated via Accept-Language.

Applied to an agent runtime, each error or status has:

  • a stable machine code (SCHEDULER_TASK_NOT_FOUND) that logic, tests and the agent can match on;
  • structured fields (ids, counts, paths) for programs;
  • an English agent-facing sentence, since the agent will read it;
  • optionally, a human-facing localized rendering keyed by the code, used only when a person will see it directly.

Code never branches on the message text, and humans never see the English sentence unless the UI is English.

3.3 Never localize machine-parsed output

Git's coding guidelines draw this line explicitly: plumbing errors are "sometimes meant for machine consumption and should not be marked for translation", while error messages from porcelain commands should be marked for translation unless there is a compelling reason not to, and BUG() messages are not translated because they communicate with developers (Git CodingGuidelines). The same document says primary output suitable for a pipe belongs on stdout, with chatty feedback and errors on stderr.

On the consumer side, the long-standing shell practice is to force the C locale for anything that parses output. GNU gettext documents the lookup order LANGUAGE, then LC_ALL, then the category variable such as LC_MESSAGES, then LANG, and notes that LC_ALL is commonly used in scripts such as autoconf configure so tests do not behave in locale-dependent ways (GNU gettext manual, Locale Environment Variables). For an agent runtime the analogue is: when your own code shells out and parses the result, set LC_ALL=C (or C.UTF-8 when you need UTF-8) on that subprocess, and design your own CLI so that machine-readable modes (--json, a stable line format) are locale-independent by construction.

3.4 Neutral structure in injected context

For hook-injected context and handoff notes, prefer a rigid, locale-free structure: fixed English keys, ISO timestamps, enumerated status values, and a short English sentence where narrative is needed. A status such as Status: pending is searchable and testable. A status such as a translated word is neither.

4. Mixed-Language Failure Modes

The following were observed or are easy to reproduce. The local demonstrations were run on one Linux host with glibc locales installed; they illustrate the mechanism and are not claims about every platform.

4.1 Locale leaks into parsed output

The same command prints different text depending on environment variables inherited from whoever started the process:

$ LC_ALL=C date +%c
Wed Nov 11 09:03:00 2026
$ LC_ALL=zh_CN.UTF-8 date +%c
2026年11月11日 星期三 09时03分00秒
$ LC_ALL=C.UTF-8 ls /nonexistent
ls: cannot access '/nonexistent': No such file or directory
$ LANGUAGE=zh_CN LC_ALL=zh_CN.UTF-8 ls /nonexistent
ls: 无法访问 '/nonexistent': 没有那个文件或目录

A service started by a developer with a Chinese locale and later restarted by a supervisor with a bare environment will produce different text for the same failure. Any code or agent that matched on the English phrase will break only on the machine with the other locale, which is the hardest class of bug to reproduce. The agent also sees an error in a different language than the rest of its context, which is the Section 1.3 setup for language confusion.

4.2 Fallback chains that silently surface another language

Catalog lookup usually has a fallback: requested locale, then a default, then the key. If the default is a language other than the one the reader expects, a missing translation does not fail; it shows the wrong language. This is the shape of a bug found in the Zylos dashboard: a leftover Chinese fallback, coupled to UI code, in a module that produces answers. The fix is not only to translate the string. It is to decide the reader. If the string is human-facing, the fallback chain should end in the project's source language through the catalog. If it is agent-facing, it should not be in a fallback chain at all.

Three variants of the same bug are worth checking for:

  • A default argument such as lang = "zh" in a helper that is reused by agent-facing code.
  • A ternary that picks between two hard-coded literals by locale, which hides a third case (an unknown locale falls into the "else" branch).
  • A catalog default that is the translation, not the source, so adding a language silently changes what an untranslated string says.

4.3 The agent echoes internal-language text to users

If an agent-facing string contains a sentence, a user-facing reply that quotes or paraphrases it will carry that language into the conversation. When the instruction layer is English and a Chinese-speaking user receives a pasted English error, the user sees a language they did not choose. The reverse is equally common after a language migration: leftover Chinese status text in a tool result is quoted to an English user. The mitigation is partly prompt-side (tell the agent to answer in the user's language and to translate rather than quote internal text, Section 6) and partly structural (agent-facing strings should carry codes and structured fields, so that the agent composes the user-facing sentence itself).

4.4 Tests that break on translation

A test that asserts expect(output).toContain("Task completed") for a human-facing function fails when the locale changes, while one that asserts on a code or structured field survives. Conversely, snapshot tests of agent-facing text should break on any wording change, because those are prompt edits and deserve review. The two kinds of test have opposite stability goals and need to be kept apart; Section 5 returns to this.

5. Enforcement

A rule that depends on reviewers remembering it will decay. The cheap, mechanical checks below have the highest return.

5.1 A script-aware lint over agent-facing paths

Mark a set of directories as agent-facing (instructions, hook scripts, CLI formatters, handoff templates, error-code modules) and fail CI if any file contains characters from CJK scripts. JavaScript supports Unicode property escapes with the u or v flag: \p{Script=Han} matches Han characters, and Script_Extensions (scx) is usually more accurate for characters shared across scripts, such as the Japanese long-vowel mark (MDN, Unicode character class escape). Hiragana, Katakana, Hangul and CJK punctuation need their own ranges or properties; a Han-only check misses Japanese kana.

A minimal local smoke test using GNU grep 3.11 with PCRE ranges (in a C.UTF-8 locale), run on a sample file, flagged a Chinese, a Japanese katakana and a Korean line and ignored the plain English line:

grep -rnP '[\x{3000}-\x{303f}\x{3040}-\x{30ff}\x{3400}-\x{4dbf}\x{4e00}-\x{9fff}\x{ac00}-\x{d7af}\x{ff00}-\x{ffef}]' src/agent-facing/

It only covers the blocks it lists. In the same run, a file containing only the supplementary Han character 𠮷 (U+20BB7, CJK Extension B) was not flagged, while Node's /\p{Script=Han}/u.test("𠮷") returned true (GNU grep's -P '\p{Han}' also matched it). Treat a range list as a smoke test for the ranges it names; for complete Han coverage, match on the script property.

A stricter variant is "no non-ASCII at all" in those paths, with an allowlist. That also catches smart quotes, ellipsis characters and lookalikes. Ruff's RUF001 rule, for example, flags Unicode characters in strings that look like ASCII but have different code points, with an allowed-confusables setting for exceptions (Ruff RUF001). If you want the stricter policy, ASCII-only is simpler to state and enforce than a script list, at the cost of a longer allowlist for the places where non-ASCII is legitimate, such as test fixtures that deliberately contain Chinese user input.

5.2 The allowlist is part of the design

Some agent-facing code must mention non-English text: a fixture for a Chinese user message, a regex that detects a user's language, a documented example in the instruction file of how to reply in Chinese. Keep an explicit allowlist of file paths, in version control, with a one-line reason per entry. A growing allowlist is a signal worth reviewing; an inline // ignore is not.

5.3 The human side: catch un-localized strings and test layout

For the human-facing side, use the tooling i18n engineers already have. eslint-plugin-i18next provides a no-literal-string rule to stop literal text from reaching users without translation; its default in recent versions is to check plain text in JSX markup, with options to widen it (eslint-plugin-i18next).

Pseudo-localization complements it. Android's pseudolocales are generated automatically from the English source: the accented-English locale adds accents, lengthens text and brackets each message, so hard-coded strings, which stay unaccented, become visible; layout breakage from expansion appears; and concatenated sentences show up as multiple brackets (Android developers, pseudolocales). A dashboard rendered in a pseudolocale during a smoke test makes any agent-facing or leftover string stand out visually. The converse check works too: a string that appears in pseudolocale unchanged and untouched is one the catalog never saw.

5.4 Locale-matrix tests for machine output

Run the same parser tests under at least two environments: LC_ALL=C and one non-English UTF-8 locale. If agent-facing output or parsing differs between them, the test fails. Git's guidelines show the policy side of this. The tests only need to observe that output is stable, not know any particular translation.

5.5 Snapshot tests for injected context

Treat the text the agent receives at session start as a build artifact. Render it in a test with fixed inputs and compare it to a checked-in snapshot. Any wording change shows up in the diff, which turns prompt edits into a normal code review. Pair the snapshot with the CJK lint so a translation does not enter by accident, and with a token-count assertion that fails if the injected context grows beyond a stated budget.

5.6 Ownership of the boundary in code review

Add a pull-request checklist item: for every new string, name the reader. A reviewer who sees a Chinese literal in an agent-facing path, or an English literal in a UI path, can ask one question: who reads this?

6. Reply Language: Mirror the User, Keep the Instructions Fixed

Keeping instructions in one language does not mean replying in that language. The two are separate settings.

Explicit instruction beats inference. The multilingual guidance cited above says the model infers response language from the conversation but production applications should state it, preferably in the system prompt, and should interpolate a user-selected language into that prompt (Anthropic, Multilingual support). Claude Code's language setting does exactly this, injecting an "always respond in that language" instruction (Claude Code settings reference).

Two modes. "Mirror the user" (answer in the language of the latest user message) works when users are single-language and the runtime has no profile. "Preferred language" (a per-user setting, injected into context) is more predictable, because it does not flip when a user pastes a log line in another language. A reasonable default is a per-user preference with mirror as the fallback when no preference is recorded.

Where to store it. A per-user profile field such as reply_language belongs in the user's profile, set by the user or recorded after they state it. Channel adapters (chat, web console) can seed it from the platform's locale, but the user's explicit statement should win.

What to put in the prompt. A short, language-neutral directive in the agent-facing language, for example: "Reply to the user in their preferred language (given below). Instructions, tool output and internal notes are in English; translate rather than quote them. Keep code, commands, file paths and error codes verbatim." Three parts matter: the preference, the instruction to translate rather than quote, and the carve-out for text that must stay exact.

Verify it. Language confusion rises with complex prompts and higher temperature (Marchisio et al.), so an agent that replies correctly on simple turns may slip on long tool-heavy ones. Add a small eval: scripted conversations in two or three languages with tool output in English, asserting that the final user-visible reply is in the target language. A script check on the reply is only a cheap first filter, because script alone cannot tell Chinese from Japanese written only in Han characters, or tell apart languages that share the Latin script, such as English, French and Spanish. For the assertion itself, run language identification on the reply or have a person check a representative set of responses.

7. A Practical Checklist for a Runtime Like Zylos

Zylos runs persistent agents in a terminal session with a communication bridge, a scheduler, memory files and a dashboard. Its text surfaces map onto the classes above as follows; adapt the specifics to your own layout.

Inventory and classify

  • List every place text is produced: instruction files, session-start hook output, bridge message wrappers, scheduler task prompts, skill descriptions, CLI output, handoff and state notes, error strings, dashboard UI, notifications.
  • Mark each as agent-facing, human-facing or relayed. Record the classification in a short table in the repo.
  • Decide the agent-facing language (English) and the human-facing default locale.

Structure

  • Put agent-facing strings outside any locale catalog, in dedicated directories.
  • Put human-facing strings in a catalog keyed by stable identifiers; make the source-language string the catalog default.
  • Give every error and status a stable code and structured fields; code never matches on message text.
  • Use ISO 8601 or RFC 3339 timestamps with an offset and numeric durations in all agent-facing output.
  • Give each CLI a machine-readable mode that is locale-independent; set LC_ALL=C on subprocesses whose output is parsed.
  • Wrap relayed user content: wrapper fields in English, payload untouched.

Fallbacks

  • Search for default language arguments and locale ternaries in shared helpers; remove any that serve both readers.
  • Make the final fallback of a human-facing lookup the source language through the catalog, and make a missing key fail loudly in tests.
  • Make agent-facing paths free of locale lookups entirely.

Enforcement

  • CI lint: no CJK (or no non-ASCII) in agent-facing paths, with a path-and-reason allowlist.
  • A no-hard-coded-strings rule on the UI side, plus a pseudo-locale smoke render of the dashboard.
  • Parser tests run under LC_ALL=C and one non-English UTF-8 locale.
  • Snapshot test of the full injected session context, with a token budget assertion measured on the real model.
  • A review checklist line: "Which reader is this string for?" with code owners on the two string stores.

Reply language

  • Store reply_language per user; default to mirroring when unset.
  • Inject the preference into the agent's context as an explicit instruction, not an inference.
  • Instruct the agent to translate rather than quote internal text, and to keep code and paths verbatim.
  • Add a multilingual reply eval with English tool output and a non-English user.

Migration hygiene

When migrating existing text, move in the order that limits risk: first classify, then add the lint in report-only mode to count violations, then convert the agent-facing modules, then switch the lint to failing with the allowlist. Keep one release where both the old and new wording are accepted by any parser, since persisted notes and queued tasks written before the change will still contain the old language. Do not translate stored historical records in place; stored text that the agent reads later is part of its context and will simply be mixed until it ages out.

8. Limits of This Article

  • The model-comprehension studies cited are from 2022 to 2024 and involve older model families. They motivate a prior, not a measurement of today's models, and the vendor table measures multiple-choice knowledge, not agentic instruction following.
  • No tokenizer-specific Chinese-versus-English ratio is given, because none of the sources consulted gives one reliably for current models. Measure your own context.
  • The local locale demonstrations are from one host and show mechanism only.
  • The claim that English is the safest single agent-facing language is a risk-management judgment supported by the research and engineering considerations above, not a theorem. A team with a different working language and its own evals can reach a different conclusion, provided it picks one language per reader and enforces it.
  • This article proposes practices for tool and runtime authors. It does not claim that any specific product implements all of them.

References

Research

Vendor documentation and engineering guidance

Standards and i18n practice

Tooling