VLSP 2026 Challenge on Legal Machine Translation with Limited-Parameter Models and Terminology Consistency

Important dates

September 18: Training Data release

September 25: Public Test

October 9: Private Test

October 15: Result announcement

October 25: Paper submission

November 5: Acceptance notification

November 12: Camera-ready (= > Proceedings)

November 15: Workshop date

Task Description

Machine Translation (MT) in the legal domain is challenging because legal documents require high accuracy and consistent use of specialized terminology. Legal texts often contain complex sentences, domain-specific terms, and expressions whose meaning depends strongly on the legal context. Inconsistent translation of the same legal term can also lead to ambiguity or changes in meaning.

These challenges become more difficult when using relatively small models with limited computational resources.

Challenges to be addressed:

• Legal terminology: Legal documents contain many specialized terms and expressions that may have different meanings from their general-language usage. Systems need to translate these terms correctly.
• Terminology consistency: The same legal term or concept should be translated consistently throughout a document.
• Accuracy: Legal translation requires preserving the original meaning, including rights, obligations, conditions, and exceptions. Missing or incorrectly translated information can affect the meaning of a legal text.
• Complex legal language: Legal documents often contain long sentences, clauses, cross-references, and formal expressions that are difficult to translate correctly.
• Limited computational resources: Participants need to balance translation quality with inference speed and memory usage.

Teams will be provided with the same datasets, including development, and test datasets from the legal domain. Participants may use pre-trained models, public datasets, additional pre-training, fine-tuning, and data augmentation to improve their systems.

The shared task has the following technical constraints:

• The final model must contain fewer than 8B parameters.
• Participants may use pre-trained models and perform additional pre-training or fine-tuning.
• Participants may use data augmentation and publicly available datasets.
• Systems must not call external APIs during inference.
• The system must run locally using the submitted model.
• Domain: Legal
• Languages: English and Vietnamese

For Machine Translation, we focus on the following directions:

• English to Vietnamese (en→vi)
• Vietnamese to English (vi→en)

We set this shared task in a restricted context with limited computational resources. The final model is limited to fewer than 8B parameters, encouraging participants to explore effective training, data augmentation, domain adaptation, and terminology-aware methods rather than relying only on larger models.

Dataset

The final ranking will be based on a combination of translation quality, legal accuracy, terminology correctness and consistency, inference time, and memory footprint. The detailed evaluation protocol and weighting will be announced by the organizers.

Development and Test Data
● Parallel Corpora: English-Vietnamese
● Monolingual Corpora: English and Vietnamese
● Development set and (public) test set: English-Vietnamese 

The development set will be provided together with the training sets. Participants could facilitate those datasets while training to validate their models before applying them to the official (private) test set, which will be provided on the planned date. You can safely assume that the development set, the test set, and the official test set are in the same domain. Participants should use the public test set with an automatic metric to decide which systems to submit. We suggest that you could use SacreBLEU for evaluation of your machine system
 

Evaluation

System results will be evaluated using both automatic metrics and LLM-based evaluation. Only constrained systems satisfying the task requirements will be officially evaluated and ranked.

The evaluation will consider:

1. Translation Quality: Standard MT metrics such as SacreBLEU will be used.
2. Terminology Correctness: Evaluate whether key legal terms are translated correctly according to their legal meaning.
3. Terminology Consistency: Evaluate whether the same legal terms and concepts are translated consistently throughout the text.
4. Legal Translation Accuracy: Evaluate the accuracy and faithfulness of the translation, including legal meaning, rights, obligations, and conditions.
5. Inference Time: Measure the system's response/inference time under a standardized environment.
6. Memory Inference Footprint: Measure the peak memory usage during inference.

The final ranking will be based on a combination of translation quality, legal accuracy, terminology correctness and consistency, inference time, and memory footprint. The detailed evaluation protocol and weighting will be announced by the organizers.

Development and Test Data

• Parallel Corpora: English-Vietnamese
• Monolingual Corpora: English and Vietnamese
• Development set and (public) test set: English-Vietnamese 

The development set will be provided together with the training sets. Participants could facilitate those datasets while training to validate their models before applying them to the official (private) test set, which will be provided on the planned date. You can safely assume that the development set, the test set, and the official test set are in the same domain. Participants should use the public test set with an automatic metric to decide which systems to submit. We suggest that you could use SacreBLEU (Post, 2018) for evaluation of your machine system.

Official (private) test set will be posted here and informed via VLSP mailing list.

Data Format 

Input format:

• For the parallel data, training, development and public test sets will be provided as UTF-8 plaintexts, 1-to-1 sentence aligned, one “sentence” per line. Notice that “sentence” here is not necessarily a linguistic sentence but maybe phrases.
• For the monolingual corpora, we provide UTF-8 plaintexts, one “sentence” per line as you would see when you downloaded them

Output format:

• UTF-8, precomposed Unicode plaintexts, one sentence per line. Participants might choose appropriate casing methods in the preprocessing steps: word segmentation, true casing, lowercasing or leaving it all along. You might want to use those tools which are available in the Moses git repository.

Submission Format

Participants must submit a working Docker image that satisfies the following constraint:

• Self contained – the image contains all your model and its dependency, and must work offline. Do not use any online service/API in your code
• Accompanied by a Bash script that uploads an input text file to your Docker image and receives the corresponding translations in an output text file. This bash script will receive (1) the host: port web path of your Docker image, (2) the path to the input file, and (3) the path to the output file as arguments. Additionally, participants are encouraged to add output statistics to standard output, such as total run time, averaged sentences-per-second and words-per-second, etc.
• Compressed in a known format: .tar.gz | .tar.bz2 | .7z | .rar | .zip
• Provided a MD5 checksum for integrity verification.
• The compressed file is to be hosted in a known cloud storage service (e.g Google Drive, Microsoft OneDrive) and given appropriate download permission.
• Multiple submissions are allowed, but only the last submission will be evaluated.

Contact

Zalo Group: []

Registration

https://forms.gle/dP5Lc6pgAy2VPH9q6

Organizers

• Van-Vinh Nguyen (vinhnv@vnu.edu.vn) -VNU University of Engineering and Technology (VNU-UET)
• Hong-Viet Tran (thviet@vnu.edu.vn) -VNU University of Engineering and Technology (VNU-UET)
• Tien-Khoi Nguyen (nguyentienkhoi210@gmail.com)- VNU University of Engineering and Technology (VNU-UET)

References

1. Example