Korean Translationese Mitigation with Encoder–Decoder Models

15th YAICON · Team 그것팀되다

A compact research project on rewriting English-to-Korean translationese into more natural Korean while preserving the source meaning. We constructed a pseudo-parallel corpus and compared Korean-specific and multilingual encoder–decoder models.

Project snapshot

Item	Scope
Task	Translationese-to-natural-Korean style transfer
Data	Approximately 500K pseudo-parallel sentence pairs
Models	KE-T5-base, mT5-small/base, KoBART-base-v2
Evaluation	BERTScore, perplexity, MATTR, and qualitative comparison
Contribution	Data construction plus a fidelity–fluency comparison across model families

Key takeaway

The reported results show a trade-off rather than one universally best model: KE-T5 produced the lowest perplexity but sometimes changed meaning, while KoBART retained stronger semantic similarity and offered a more balanced qualitative profile. The metrics were produced within this student-project setup and should not be interpreted as a general benchmark ranking.

Repository map

data/Data_Processing.ipynb                 Pseudo-parallel data construction
data/Data_filtered.csv                     Small packaged data artifact
models/ke_t5_translationese_mitigation.ipynb
models/mt5_translationese_mitigation_base&small.ipynb
models/kobart-base-v2_translationese_mitigation.ipynb

Open the data notebook first to inspect the preprocessing assumptions, then run the model notebooks independently. The historical notebooks contain environment-specific paths that should be replaced with local relative paths before execution.

👥 Team Members

Name	Degree & Major	Role
Hyun Gu Kang (강현구)	B.A. German Language & Literature / B.Sc. Applied Statistics	Team Lead, KE-T5 Model, Preliminary Presentation
Min Gyu Kim (김민규)	M.Sc. Statistics & Data Science	Related Work, Final Presentation
Kyung Won Park (박경원)	M.Sc. Artificial Intelligence	Modeling (mT5-small / mT5-base / KoBART-base)
Hyun Bo Sim (심현보)	B.Sc. Electrical & Electronic Engineering	Data Preparation
Yumin Cheong (정유민)	M.D. Candidate	Data Preparation, Evaluation Metrics

📌 Background

Although recent advancements in large language models (LLMs) have significantly improved machine translation quality, translationese remains a persistent issue. Translationese refers to unnatural linguistic patterns that arise from the direct transfer of source-language structures or expressions into the target language. In English-to-Korean translation, this often results in awkward word order, overuse of pronouns (e.g., “그”, “그녀”), or overly formal constructions such as “~에 대하여”.

These artifacts not only reduce the readability and fluency of the translated text but also compromise the reliability of machine translation evaluation. For instance, Artetxe et al. (2020a) demonstrated that test sets containing translationese can lead to overly optimistic performance scores, thus misrepresenting the true effectiveness of cross-lingual models.

In this context, mitigating translationese in English–Korean translations presents both a linguistic and technical challenge. Hence, the goal of our project is to design and implement a method to detect and reduce translationese using fine-tuned encoder–decoder models.

🎯 Objectives

We aim to build a style transfer model that rewrites translationese Korean sentences into more natural Korean while preserving their original meaning.

📊 Dataset

Given the absence of parallel corpora for this task, we constructed a pseudo-parallel dataset based on the English–Korean parallel corpus from AI Hub, available at traintogpb/aihub-koen-translation-integrated-base-1m.

To create translationese-style sentences, we applied back-translation and filtered the results using automatic metrics such as BERTScore to ensure semantic consistency. The resulting pseudo-parallel dataset contains approximately 500K sentence pairs, built through the following steps:

1. Base Data: English–Korean parallel sentences from AI Hub
2. Back-Translation: Generation of translationese-style Korean using a fine-tuned NLLB model (NLLB-en2ko)
3. Filtering:
   • Retained sentence pairs with BERTScore (F1) ≥ 0.9 to ensure semantic preservation
   • Removed sentences containing non-Korean characters or noisy artifacts

The final dataset consists of pseudo-parallel sentence pairs that are suitable for supervised fine-tuning of style transfer models.

Each row in the CSV file includes:

• ko_translationese: Korean sentences generated via back-translation from English
• ko: Comparable native Korean sentences

🧠 Model and Training

We fine-tuned the following pretrained encoder–decoder models:

• KE-T5-base – Korean-specific model
• mT5-small and mT5-base – multilingual T5 variants
• koBART-base-v2 – Korean-specific BART-based Seq2Seq model Each model was trained using standard cross-entropy loss on the filtered pseudo-parallel dataset.

📈 Evaluation

To assess the effectiveness of translationese mitigation, we employed both quantitative and qualitative evaluation metrics.

🔢 Quantitative Metrics

• BERTScore (semantic preservation):
  All models showed a drop in BERTScore compared to the original (0.87), with final scores of:

  • mT5-small: 0.75
  • mT5-base: 0.83
  • KE-T5-base: 0.81
  • koBART-base-v2: 0.86

• Perplexity (fluency of generated text):
  Lower perplexity indicates more natural language.

  • KE-T5-base: 1.36
  • mT5-small: 12.84
  • mT5-base: 9.56
  • koBART-base-v2: 8.83

• MATTR (Moving Average Type-Token Ratio, measures lexical diversity):
  All models showed slightly improved lexical variety.

  • Original: 0.9941
  • mT5-small: 0.9966
  • mT5-base: 0.9960
  • KE-T5-base: 0.9977
  • koBART-base-v2: 0.9932

👀 Qualitative Analysis

• Human Evaluation:
  • KE-T5-base generated the most fluent and natural Korean expressions, although it occasionally distorted meaning (e.g., incorrect pronoun references or changes in subject).
  • mT5 models produced more literal translations with fewer semantic deviations, but retained some of the rigid and unnatural expressions typical of translationese.
  • KoBART-base-v2 struck a balance between fluency and fidelity: its outputs felt more natural than mT5’s, without the occasional over‐formal tone of KE-T5, and it preserved nuance better than the others, though minor context omissions sometimes occurred.

⚠️ Limitations

• Data quality: Even original AI-Hub translations often contained translationese, making clean ground truth difficult to obtain.
• Model accuracy:
  • KE-T5-base sometimes sacrificed semantic accuracy for fluency.
  • mT5-small/base tended to preserve semantic fidelity but produced more rigid, literal phrasing.
  • KoBART-base-v2 struck a balance by maintaining meaning fairly accurately without being as overly formal as KE-T5 and generating more natural expressions than mT5, though it occasionally omitted some contextual elements.
• CE Loss limitation: Cross-entropy loss alone may not sufficiently penalize stylistic artifacts.

🔮 Future Directions

• Explore data augmentation techniques to enrich stylistic diversity
• Introduce additional losses (e.g., classifier-guided loss, reconstruction loss, cycle loss) to improve style control

🙋 Acknowledgements

This project was presented at the 15th YAICON, a research contest held every semester by YAI, the AI academic society of Yonsei University.
We thank all members of YAI for their valuable feedback and support.

📄 License

This project is licensed under the MIT License.