The performance gap is real and well-measured
Meta's FLORES-200 benchmark, released in 2022 as part of the No Language Left Behind project, evaluates machine translation across 200 languages. It made the gap visible: even after the largest investment any single research lab has put into low-resource translation, quality on languages like Khmer, Lao, and Burmese sat well below high-resource baselines, with chrF and BLEU deltas that translate directly into downstream model errors.
The pattern repeats in evaluation benchmarks built explicitly for APAC. AI4Bharat, the IIT Madras-led research initiative, has published successive Indic-language benchmarks showing that frontier models fluent in English degrade significantly on Tamil, Bengali, Telugu, and Malayalam – not as a deficit of intelligence, but as a deficit of in-language training and evaluation data. VinAI's PhoBERT and PhoGPT lines have made the same case for Vietnamese: a Vietnamese-trained model on Vietnamese tasks beats a much larger English-centric model trained on translated Vietnamese.
Why translation pipelines do not solve it
A common reflex is to source labels in English and translate them. It does not survive contact with reality. Three failure modes recur:
- Cultural specificity. Sentiment, formality, honorifics, and politeness markers in Thai, Korean, and Japanese carry meaning that has no direct English mapping. A toxicity classifier trained on translated English labels will systematically miss in-context insults and miscategorise polite refusals.
- Script and orthography. Khmer text routinely lacks word boundaries; Vietnamese diacritics carry phonemic and semantic weight; traditional and simplified Chinese share characters but encode different conventions. Pipelines that round-trip through English silently strip these distinctions.
- Domain language. Legal Vietnamese, medical Thai, regulatory Bahasa Indonesia – the technical vocabularies do not appear in mainstream training corpora at any meaningful frequency. Translation makes the surface plausible and the substance wrong.
What "good" annotation looks like for these languages
The pattern that consistently produces production-grade datasets for APAC languages has four properties:
- In-language, in-region annotators. Native speakers, recruited and trained inside the relevant market, with reviewers who can adjudicate culturally specific edge cases. Remote-only labelling pipelines staffed thousands of kilometres away routinely under-perform here.
- Localised guidelines. The labelling guideline is written in the target language first, with English as the secondary reference. Edge cases are encoded with in-language examples, not translated approximations.
- Explicit cultural taxonomy. Politeness levels, honorific usage, code-switching patterns, regional dialect markers – when these matter for the model, they appear as labels in the schema rather than living in annotators' heads.
- Native-language QA. The IAA panel is in-language. The disagreement reports are in-language. The auditing reviewer is fluent in the dialect, not just the language.
The economics shift
In-language APAC annotation is more expensive per unit than English. The downstream economics still favour it. The Stanford AI Index has documented year-over-year that the highest-quality production systems in non-English markets are the ones with the highest in-language data investment; the cost difference at the labelling stage compounds backwards into far smaller compute and re-labelling bills downstream.
For our clients across Vietnam, Thailand, Singapore, Malaysia, and the Indonesian archipelago, the realistic decision is not "in-language or translated". It is "in-language now, or in-language in 18 months after the translated dataset under-performs in market". The former is cheaper.
Where DataX Power fits
Our annotation practice runs in-language, in-region pods across APAC – Vietnamese natively from our Hanoi HQ, Thai through partners in Bangkok, Bahasa Indonesia from Jakarta, plus established pipelines for Mandarin, Cantonese, Korean, and Japanese. Where we operate, the QA panel and the labelling guideline are in the target language first. If your team is sizing an APAC-language dataset and trying to decide between translated and in-language, the cost gap is usually smaller than feared and the quality gap is larger than expected.
