📄 Cross-Dialect Sentence Transformation: A Comparative Analysis of Language Models for Adapting Sentences to British English

Summary

The paper tests how well large language models can rewrite sentences from American, Indian and Irish English into Standard British English. The authors built a small parallel set that pairs each dialect sentence with a British reference. The introduction says 200 sentences per dialect, but the results section says 20. Three models were prompted in two steps, a procedure the authors call "promptuning": a seed prompt first describes the source dialect and the target, and then the sentences are given one by one. The models are Llama-2-70B, Falcon-180B and Mistral, which is named both Mistral-1 and Mistral-7B-Instruct-v0.1. Each output is compared with its British reference by cosine similarity. The authors also score the untouched dialect input against the same references. They report Llama-2-70B as the best model and find Indian and Irish input closer to British English than American input. However, the unchanged input already scores 0.81, 0.93 and 0.93, and most model outputs fall below these values. The study is therefore more useful as a task framing and as a lesson in evaluation design than as proof that the models adapt dialects well.

Key points

• Cosine scores (American / Indian / Irish): untouched input 0.81 / 0.93 / 0.93; Llama-2-70B 0.70 / 0.91 / 0.95; Falcon-180B 0.60 / 0.75 / 0.88; Mistral 0.63 / 0.93 / 0.91.
• Only Llama-2-70B on Irish (0.95) beats the do-nothing input, and Mistral only ties it on Indian; on American English every model scores clearly below the unchanged text.
• The data shows typical dialect contrasts:
  • lexical pairs such as queue/line, cinema/movies/pictures, peckish/hungry and tea/chai/tae;
  • Indian code-mixed items such as bhai, yaar and paisa vasool;
  • a sentence-final "na?" rendered as a British tag question.
• Several details are unclear or inconsistent: the dataset size (200 vs 20), the Mistral model name, which embedding the cosine similarity uses, and the Irish table, whose columns appear swapped.
• Planned future work includes more dialects, dialect-specific fine-tuning, human–AI collaboration and metrics designed for dialect translation.

Interesting ideas

• I1. Identity baseline: score the unchanged source against the reference next to every system, so you can see at once whether a transformation helps or harms; here it shows that most outputs are worse than doing nothing.
• I2. Parallel variant table: one meaning written in four varieties (queue/line, pictures/movies/cinema, peckish/hungry) gives a compact map of regional lexical variation.
• I3. Code-mixed loanwords: Indian English sentences mix in words like chai, yaar and bhai, and Irish spelling gives forms like tae; these are exactly the items that trip up standard pipelines.
• I4. Dialect discourse particles: the Indian sentence-final "na?" is turned into a British tag question, which shows that dialects mark confirmation requests in different ways.
• I5. Seed prompt before items: a fixed opening prompt states the source dialect and the target style, and the sentences follow, which keeps the instruction constant across the batch.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

• I1 (learn): Every repair rule or LLM-teacher rewrite could be checked against a "leave it as is" control, and kept only if it beats that control.
• I2 (world): The ontology could link regional variants such as queue and line to one concept, so the world of a text stays the same whatever dialect it was written in.
• I3 (tokens): The en crate could add lemma and POS entries for common dialect loanwords and spellings, mapping chai to tea and tae to tea, so tagging and lemmas stay stable.
• I4 (frames): The prag crate's speech-act detection could treat particles like "na?" as tag-question equivalents that ask the listener to confirm.
• I5 (learn): LLM teacher tiers could use a fixed seed instruction per dialect or genre, so silver annotations stay consistent and teachers can be compared fairly.