AI marketing & MarTech
AI localisation: evaluate quality before publication
Evaluate AI localisation with an error rubric for meaning, terminology, voice and delivery. Includes a bilingual example and a release decision worksheet.
A translation can sound convincing while changing the condition that matters most. It may turn an optional feature into a guarantee, remove an eligibility rule or confuse the date of an event.
The useful question is whether the localised asset is ready for its intended audience and use. That requires more than a general fluency score. It requires a source comparison, an error rubric and an explicit release decision.
This guide helps marketing leads organise that review for AI-assisted English and Italian content. It complements the AI marketing guide: the focus here is quality acceptance, rather than choosing a tool or estimating a production benefit.
Define what the target version must preserve
Before reviewing sentences, agree which parts of the source are fixed and which may be adapted. Product facts, conditions, numbers and evidence boundaries usually need to remain faithful. Examples, sentence order and idioms may need a different expression for the target reader.
A translation brief should identify the market, audience, channel, source version and allowed adaptations. “Italian” alone does not establish whether the copy addresses a technical buyer, a consumer or an employee.
Record terminology that affects decisions: plan names, product functions, eligibility, costs and obligations. Include prohibited interpretations, not just preferred vocabulary. If a result is a reduction in cost, the target version must not present it as a reduction in hours.
The Enel localisation article preserves an owner-reported 75% cost reduction from a professional initiative. That statement does not establish language quality, hours saved or a transferable benchmark. A quality review needs its own evidence.
If the source is ambiguous, ask its owner to clarify it. Choosing a plausible interpretation in the target language can conceal a problem that belongs in the original.
Review against a shared error vocabulary
A useful rubric tells reviewers what went wrong and why it matters. Separate meaning, terminology, language, voice and delivery so the correction reaches the right person.
The MQM Council framework provides an error typology and severity-based approach to translation evaluation. Its dimensions include accuracy, fluency, terminology and style. The simplified worksheet below is a proposed marketing adaptation, not a certified MQM assessment or universal threshold.
Swipe or use the arrow keys to see every column.
| Dimension | Review question | Example of a defect | Likely owner |
|---|---|---|---|
| Meaning | Does the target preserve the source claim? | “May” becomes “will” | Bilingual reviewer |
| Terminology | Are decision-relevant terms consistent? | A plan name changes between assets | Terminology owner |
| Natural language | Can the intended reader understand it easily? | Literal idiom obscures the instruction | Language editor |
| Voice | Is the relationship with the reader appropriate? | Formal notice becomes a playful joke | Brand editor |
| Local conventions | Are dates, units and formats clear? | Ambiguous numeric date | Market editor |
| Delivery | Does the published asset work? | Caption clipped or link in the wrong language | Publisher |
Do not collapse these dimensions into “sounds good”. A reviewer may like the style and still miss an altered commercial condition.
Use the brand voice guide for expression choices. A voice preference should not override a factual correction.
Assign severity from impact, not irritation
Severity should describe the consequence for the intended use. An editor’s dislike of a phrase is not a sufficient reason to mark it critical.
For a pilot, agree three operational levels. A blocking issue changes a consequential claim, removes an important restriction or creates an unusable deliverable. A substantial issue requires correction because it interferes with understanding or consistency. A minor issue affects polish without changing the decision.
These are local working definitions. The same defect can have different impact across contexts: a missing word in a legal condition and a missing word in a decorative caption are not equivalent.
Set release rules before looking at the batch. For example, require all blocking and substantial issues to be resolved, then specify who may accept a remaining minor issue and why. Keep that exception visible.
An average score should not compensate for one serious mistake. Excellent fluency in nine paragraphs does not make an incorrect eligibility statement acceptable.
Avoid presenting a reviewer’s confidence as proof of accuracy. Confidence can help prioritise further investigation, but the claim still needs comparison with the authorised source.
Calibrate reviewers with difficult examples
Give reviewers a small shared set before assigning a full batch. Include a clear mistranslation, a legitimate adaptation, an ambiguous source and an awkward sentence that preserves meaning.
Ask each person to mark the issue, dimension, severity and proposed action independently. Then compare their decisions. Disagreement reveals missing rules or unclear context; it is not automatically a failure of the reviewer.
Keep an agreed example beside each rule. A rule saying “preserve modality” becomes easier to apply when it shows why “can support” cannot become “guarantees”.
Use the same source version during calibration. If reviewers compare different documents, their disagreement cannot diagnose the rubric.
Back-translation can flag a possible discrepancy, but it should not replace a competent bilingual comparison. A second AI pass can reproduce the original model’s assumptions or paraphrase away a problem.
For a general approach to evaluation sets and acceptance criteria, see evaluating AI output. Localisation adds language-specific judgement to that broader process.
A hypothetical English–Italian review
Imagine an invented software service preparing a campaign. The authorised English source says: “Customers on the Business plan can request a pilot. Availability depends on market approval.”
The AI-produced Italian draft reads: “Tutti i clienti Business hanno accesso immediato al servizio.” The sentence is fluent. It changes a request into access, adds immediacy and removes the market condition.
The reviewer marks an accuracy issue that blocks release. A faithful working alternative is: “I clienti con piano Business possono richiedere un progetto pilota. La disponibilità dipende dall'approvazione nel mercato di riferimento.”
The editor may improve that wording further, but cannot delete the condition simply to make the campaign shorter. If the business wants a shorter claim, the product owner must approve a precise alternative.
A second sentence uses “support” for both customer assistance and compatibility. Neither term is inherently wrong. The reviewer requests a glossary decision because the mixed usage could confuse readers across the landing page and help article.
Finally, the Italian call to action opens the English application form. The translation may be correct, yet the journey remains inconsistent. The publisher records a delivery defect.
All details in this example are hypothetical. They illustrate how different defects lead to different owners and actions, rather than documenting an actual campaign.
Review the asset in its final medium
A script and a finished video are different deliverables. Check the words, timing and presentation after production.
W3C’s media planning guidance covers accessibility needs such as captions, transcripts and description, depending on the content and medium. Plan the relevant outputs early and review their target-language versions alongside the main asset.
Captions can be accurate but unreadable at the chosen pace. A voice track can preserve words while stressing the wrong condition. A translated graphic can hide text or leave an important label in the source language.
Give the reviewer the final format, not only a spreadsheet of strings. Check mobile presentation, headings, image text, alternative text, links and metadata. A landing page title may introduce a promise absent from the approved body.
For voice production, confirm permission and intended use through the responsible project process. A quality score does not establish rights to use a person’s voice.
The content workflow guide helps place these checks before publication rather than after the campaign has started.
Use denominators that describe the review
“Five errors” is incomplete without a reviewed quantity and a sampling method. Five issues in a short page and five issues in a large library imply different investigation needs.
Record reviewed words or segments, languages, content types and selection method. When reporting issues per segment, explain whether one segment can contain several defects. Keep severity visible instead of reporting only a combined total.
Distinguish first-pass defects, corrections verified and issues discovered after release. A corrected batch demonstrates the released version’s acceptance, while the first-pass result helps diagnose the process.
Sampling is appropriate only for the question you intend to answer. A random sample can estimate recurring issues within its coverage; it does not prove that every unreviewed high-impact claim is correct.
Review consequential facts directly even when using a sample for routine text. Record exclusions so decision makers understand what the result does not cover.
Do not compare languages through raw counts if their reviewed volumes or content difficulty differ. An apparent quality gap may be a measurement gap.
A release decision worksheet
Use this record for each asset or coherent batch.
Swipe or use the arrow keys to see every column.
| Field | What to record |
|---|---|
| Source | Version, owner and authorised claim boundaries |
| Target | Market, language, reader and channel |
| Review coverage | Quantity, selection method and excluded material |
| Issues | Segment, source, target, dimension and severity |
| Correction | Approved wording and responsible person |
| Verification | Who checked the revised version and in which format |
| Decision | Accept, revise or block, with a reason |
| Exceptions | Accepted limitations and named release authority |
| Follow-up | Where later defects are reported and corrected |
Attach the actual source and target passages to an issue. “Terminology problem” without a location or proposed correction creates another round of clarification.
Keep a correction log that distinguishes one-off fixes from reusable rules. Update the glossary when a term recurs; update the brief when the same condition is repeatedly omitted.
If retrieval supplies the source material, the RAG knowledge base guide explains why source freshness and retrieval quality need separate checks.
Decide whether the workflow is ready to expand
Expand when you can show that representative assets meet the agreed criteria and that review capacity supports the expected workload. A favourable average on a convenient sample is a weaker basis.
Redesign when the same meaning error keeps returning, reviewers lack domain context or the source itself cannot be approved. More output increases the correction burden if the cause remains unresolved.
Quality and economics belong together, but they answer different questions. A cheaper process that repeatedly changes consequential facts has not met the same content requirement. Use the automation ROI guide only after defining acceptable output.
Start with a calibrated rubric and a representative batch. The immediate result should be a defensible release decision and a list of process improvements, not a universal quality claim.
If your team needs clearer bilingual acceptance criteria, tell me what you need to localise.
Apply this to your project
Tell me the decision you are facing, the constraints and what you have already tried.
Keep exploring
AI content workflows: from brief to editorial review
Build an AI content workflow with reliable sources, clear approval roles and useful review measures. Includes a reusable brief and publication checklist.
Brand voice with AI: make your writing recognisable
Build a brand voice rubric for AI writing using examples, exclusions and tone decisions. Includes a bilingual worked example and an editorial review sheet.
Marketing knowledge bases: when RAG is useful
Decide when a marketing knowledge base needs RAG. Check source ownership, retrieval, answer quality and permissions with a practical evaluation worksheet.