COLING 2025

To Label or Not to Label: Hybrid Active Learning for Neural Machine Translation

Abdul Hameed Azeemi, Ihsan Ayyub Qazi, and Agha Ali Raza

Lahore University of Management Sciences

HUDS combines uncertainty and diversity sampling to select translation examples that are both informative and non-redundant, improving active learning for neural machine translation domain adaptation.

Paper ACL Anthology

31st International Conference on Computational Linguistics

Abstract illustration of uncertainty and diversity streams combining for hybrid selection.

Introduction

Active learning reduces annotation costs by selecting a small set of unlabeled examples for human labeling. Uncertainty sampling favors difficult examples but can repeatedly choose similar sentences, while diversity sampling covers the data distribution but may select examples that are varied yet trivial.

Hybrid Uncertainty and Diversity Sampling (HUDS) bridges these objectives. It prioritizes sentences that expose model uncertainty while maintaining broad coverage of the target domain, enabling more effective domain adaptation for neural machine translation.

HUDS active learning workflow. — HUDS stratifies examples by uncertainty, embeds and clusters each stratum, and jointly ranks uncertainty and diversity before annotation.

Hybrid uncertainty and diversity sampling

HUDS performs selection within uncertainty strata, so diversity is measured among examples with comparable uncertainty rather than across the entire pool.

1. Score and stratify uncertainty

The acquisition model computes length-normalized negative log-likelihood for unlabeled sentences. The uncertainty range is divided into ten strata.

2. Measure local diversity

Sentences are encoded with a pretrained BERT model. Within each uncertainty stratum, k-means forms a centroid and cosine distance measures how diverse each sentence is.

3. Rank with a hybrid score

A weighted score H(x) = λd(x,cᵢ) + (1−λ)uₓ combines normalized diversity and uncertainty. The experiments use λ = 0.5 and select the highest-scoring examples.

4. Label and retrain

Selected sentences are labeled, added to the training set, and used to update the translation model before the next active learning iteration.

Evaluation

We evaluate BART-base on multi-domain German–English translation using WMT14 as out-of-domain data and Medicine, Law, and IT corpora as target domains. Generalization is also tested on WMT14 and IWSLT 2014 English–French.

Each main iteration scores 20,000 unlabeled sentences and requests labels for 1,000 examples. HUDS is compared with Random, Normalized Sequence Probability, and In-Domain Diversity Sampling using SacreBLEU.

Target domain	HUDS	Strongest baseline
Medicine En–De	34.58	33.68
IT En–De	30.70	30.23
Law En–De	35.62	35.53
IWSLT En–Fr	35.5	35.1

Results

HUDS achieves the best final SacreBLEU in every evaluated domain. On Medicine it improves over the strongest baseline by 0.90 SacreBLEU, while gains on IT, Law, and IWSLT demonstrate consistent performance across domains and language pairs.

With a larger 5,000-example budget on IT, HUDS reaches 34.0 SacreBLEU after ten iterations, compared with 32.5 for Random, 32.3 for uncertainty sampling, and 29.1 for diversity sampling.

Examples selected by uncertainty, diversity, and HUDS. — Uncertainty sampling concentrates on difficult but similar examples, diversity sampling spreads across easy examples, and HUDS selects from the informative, diverse region.

What does HUDS select?

Selection trajectories show that HUDS prioritizes examples with both high uncertainty and high diversity in early rounds. As the model adapts, it progressively covers the remaining target-domain distribution instead of repeatedly querying one narrow region.

Uncertainty and diversity scores across active learning iterations. — Uncertainty versus diversity for Medicine, Law, and IT. Darker points denote earlier active learning iterations.

Conclusion

HUDS resolves a core tradeoff in active learning for translation: examples should be difficult enough to teach the model and different enough to avoid redundant annotation. Its stratified hybrid score delivers consistent gains across domains, budgets, and language pairs.

Citation

@inproceedings{azeemi-etal-2025-label,
  title     = {To Label or Not to Label: Hybrid Active
               Learning for Neural Machine Translation},
  author    = {Azeemi, Abdul Hameed and Qazi, Ihsan Ayyub
               and Raza, Agha Ali},
  booktitle = {Proceedings of COLING},
  pages     = {3071--3082},
  year      = {2025}
}