VLSP 2022 ASR Challenge: Automatic Speech Recognition

Shared Task Registration Form

Important dates

  • July 27, 2022: Registration opens

  • Aug 31, 2022: Registration closes

  • Sept 05, 2022: Dataset building starts

  • Sept 12, 2022: Dataset building ends

  • Sept 19, 2022: Training data and public test data for the ASR-T1 released

  • Oct 31, 2022: Private test set released for both ASR-T1 and ASR-T2 

  • Oct 31, 2022: Test result submission

  • Nov 15, 2022: Technical report submission

  • Nov 26, 2022: Result announcement (workshop days)

General Description

VLSP2022 ASR will feature two evaluation tasks. Teams can participate in one of the tasks or both.

Task-01 (ASR-T1): Focusing on a full pipeline development of the ASR model from scratch with limited training data condition. The organizer will provide two training datasets. The first dataset is around 5o hours of transcribed data. The second dataset is around 400 hours of untranscribed in-domain data. All participants are required to use only this provided data to develop models including acoustic and language models. Any use of another resource for model development is not acceptable. 

Task-02 (ASR-T2): Focusing on spontaneous speech in different real scenarios e.g., meeting conversation, lecture speech. For this task, the organization will not provide training data, participants can use all available data sources to develop their models without any limitation.

Evaluation metric

The quality of the models will be evaluated by the Syllable Error Rate (SyER) metric.

SyER = (S+D+I)/N

where

  • S is the number of substitutions,
  • D is the number of deletions,
  • I is the number of insertions,
  • C is the number of correct syllables,
  • N is the number of syllables in the reference (N=S+D+C)

Submission Guidelines

ASR Run Submission Format

Submissions have to be made in UTF-8, lower-case and one line for each utterance.

utterance_name<TAB>recognized_text_sequence

For example

0001.wav chào mừng các bạn đã tham dư cuộc thi

0002.wav tên tôi là nguyễn văn a

Output Conventions

Since there are cases that input speech can be interpreted in different ways, the below rules are applied to mitigate such an issue:

  1. Numbers, dates etc. need to be transcribed in words as they are spoken, not in digits

  2. Common acronyms such as nato, fifa, are written as one word, without any special markers between the letters. This applies no matter whether they are spoken as one word or spelled out as a letter sequence. All other letter spelling sequences are written as individual letters with space in between.

  3. For English words, names of people and places in other languages such as youtube, facebook, are written as it, not in Vietnamese pronunciation.

Organizers

  • Đỗ Văn Hải
  • Lê Quang Trung