Using the TED Talks to Evaluate Spoken Post-editing of Machine Translation

Jeevanthi  Liyanapathirana; Andrei Popescu-Belis

Using the TED Talks to Evaluate Spoken Post-editing of Machine Translation

Jeevanthi Liyanapathirana, Andrei Popescu-Belis

IDIAP Research Institute

Publikation: Bidrag til bog/antologi/rapport › Konferencebidrag i proceedings › Forskning › peer review

58 Downloads (Pure)

Abstract

This paper presents a solution to evaluate spoken post-editing of imperfect machine translation output by a human translator. We compare two approaches to the combination of machine translation (MT) and automatic speech recognition (ASR): a heuristic algorithm and a machine learning method. To obtain a data set with spoken post-editing information, we use the French version of TED talks as the source texts submitted to MT, and the spoken English counterparts as their corrections, which are submitted to an ASR system. We experiment with various levels of artificial ASR noise and also with a state-of-the-art ASR system. The results show that the combination of MT with ASR improves over both individual outputs of MT and ASR in terms of BLEU scores, especially when ASR performance is low.

Originalsprog	Engelsk
Titel	The LREC 2016 Proceedings : Tenth International Conference on Language Resources and Evaluation
Redaktører	Nicoletta Calzolari, Khalid Choukri, Thierry Declerck, Marko Grobelnik, Bente Maegaard, Joseph Mariani, Asuncion Moreno, Jan Odijk, Stelios Piperidis
Antal sider	8
Udgivelsessted	Paris
Forlag	European Language Resources Association
Publikationsdato	2016
Sider	2232-2239
ISBN (Elektronisk)	9782951740891
Status	Udgivet - 2016
Begivenhed	The 10th International Conference on Language Resources and Evaluation. LREC 2016 - Portorož, Slovenien Varighed: 23 maj 2016 → 28 maj 2016 Konferencens nummer: 10 http://lrec2016.lrec-conf.org/en/

Konference

Konference	The 10th International Conference on Language Resources and Evaluation. LREC 2016
Nummer	10
Land/Område	Slovenien
By	Portorož
Periode	23/05/2016 → 28/05/2016
Internetadresse	http://lrec2016.lrec-conf.org/en/

Emneord

Machine Translation
Spoken post-editing
Evaluation

Adgang til dokumentet

jeevanthi_liyanapathirana_using_the_ted_talks_publishersversionForlagets udgivne version, 188 KBLicens: Ikke-specificeret

http://www.lrec-conf.org/proceedings/lrec2016/pdf/1142_Paper.pdfLicens: CC BY-NC

Handle.net

Persistent link

Citationsformater

Liyanapathirana, J., & Popescu-Belis, A. (2016). Using the TED Talks to Evaluate Spoken Post-editing of Machine Translation. I N. Calzolari, K. Choukri, T. Declerck, M. Grobelnik, B. Maegaard, J. Mariani, A. Moreno, J. Odijk, & S. Piperidis (red.), The LREC 2016 Proceedings: Tenth International Conference on Language Resources and Evaluation (s. 2232-2239). European Language Resources Association. http://www.lrec-conf.org/proceedings/lrec2016/pdf/1142_Paper.pdf

Liyanapathirana, Jeevanthi ; Popescu-Belis, Andrei. / Using the TED Talks to Evaluate Spoken Post-editing of Machine Translation. The LREC 2016 Proceedings: Tenth International Conference on Language Resources and Evaluation. red. / Nicoletta Calzolari ; Khalid Choukri ; Thierry Declerck ; Marko Grobelnik ; Bente Maegaard ; Joseph Mariani ; Asuncion Moreno ; Jan Odijk ; Stelios Piperidis. Paris : European Language Resources Association, 2016. s. 2232-2239

@inproceedings{aabcd0baf2d34796880ed012a1dc96df,

title = "Using the TED Talks to Evaluate Spoken Post-editing of Machine Translation",

abstract = "This paper presents a solution to evaluate spoken post-editing of imperfect machine translation output by a human translator. We compare two approaches to the combination of machine translation (MT) and automatic speech recognition (ASR): a heuristic algorithm and a machine learning method. To obtain a data set with spoken post-editing information, we use the French version of TED talks as the source texts submitted to MT, and the spoken English counterparts as their corrections, which are submitted to an ASR system. We experiment with various levels of artificial ASR noise and also with a state-of-the-art ASR system. The results show that the combination of MT with ASR improves over both individual outputs of MT and ASR in terms of BLEU scores, especially when ASR performance is low.",

keywords = "Machine translation, Spoken post-editing, Evaluation, Machine Translation, Spoken post-editing, Evaluation",

author = "Jeevanthi Liyanapathirana and Andrei Popescu-Belis",

year = "2016",

language = "English",

pages = "2232--2239",

editor = "Nicoletta Calzolari and Choukri, { Khalid} and Thierry Declerck and Marko Grobelnik and Bente Maegaard and Joseph Mariani and Asuncion Moreno and Jan Odijk and Stelios Piperidis",

booktitle = "The LREC 2016 Proceedings",

publisher = "European Language Resources Association",

address = "France",

note = "The 10th International Conference on Language Resources and Evaluation. LREC 2016 ; Conference date: 23-05-2016 Through 28-05-2016",

url = "http://lrec2016.lrec-conf.org/en/",

}

Liyanapathirana, J & Popescu-Belis, A 2016, Using the TED Talks to Evaluate Spoken Post-editing of Machine Translation. i N Calzolari, K Choukri, T Declerck, M Grobelnik, B Maegaard, J Mariani, A Moreno, J Odijk & S Piperidis (red), The LREC 2016 Proceedings: Tenth International Conference on Language Resources and Evaluation. European Language Resources Association, Paris, s. 2232-2239, The 10th International Conference on Language Resources and Evaluation. LREC 2016, Portorož, Slovenien, 23/05/2016. <http://www.lrec-conf.org/proceedings/lrec2016/pdf/1142_Paper.pdf>

Using the TED Talks to Evaluate Spoken Post-editing of Machine Translation. / Liyanapathirana, Jeevanthi ; Popescu-Belis, Andrei.
The LREC 2016 Proceedings: Tenth International Conference on Language Resources and Evaluation. red. / Nicoletta Calzolari; Khalid Choukri; Thierry Declerck; Marko Grobelnik; Bente Maegaard; Joseph Mariani; Asuncion Moreno; Jan Odijk; Stelios Piperidis. Paris: European Language Resources Association, 2016. s. 2232-2239.

Publikation: Bidrag til bog/antologi/rapport › Konferencebidrag i proceedings › Forskning › peer review

TY - GEN

T1 - Using the TED Talks to Evaluate Spoken Post-editing of Machine Translation

AU - Liyanapathirana, Jeevanthi

AU - Popescu-Belis, Andrei

N1 - Conference code: 10

PY - 2016

Y1 - 2016

N2 - This paper presents a solution to evaluate spoken post-editing of imperfect machine translation output by a human translator. We compare two approaches to the combination of machine translation (MT) and automatic speech recognition (ASR): a heuristic algorithm and a machine learning method. To obtain a data set with spoken post-editing information, we use the French version of TED talks as the source texts submitted to MT, and the spoken English counterparts as their corrections, which are submitted to an ASR system. We experiment with various levels of artificial ASR noise and also with a state-of-the-art ASR system. The results show that the combination of MT with ASR improves over both individual outputs of MT and ASR in terms of BLEU scores, especially when ASR performance is low.

AB - This paper presents a solution to evaluate spoken post-editing of imperfect machine translation output by a human translator. We compare two approaches to the combination of machine translation (MT) and automatic speech recognition (ASR): a heuristic algorithm and a machine learning method. To obtain a data set with spoken post-editing information, we use the French version of TED talks as the source texts submitted to MT, and the spoken English counterparts as their corrections, which are submitted to an ASR system. We experiment with various levels of artificial ASR noise and also with a state-of-the-art ASR system. The results show that the combination of MT with ASR improves over both individual outputs of MT and ASR in terms of BLEU scores, especially when ASR performance is low.

KW - Machine translation

KW - Spoken post-editing

KW - Evaluation

KW - Machine Translation

KW - Spoken post-editing

KW - Evaluation

M3 - Article in proceedings

SP - 2232

EP - 2239

BT - The LREC 2016 Proceedings

A2 - Calzolari, Nicoletta

A2 - Choukri, Khalid

A2 - Declerck, Thierry

A2 - Grobelnik, Marko

A2 - Maegaard, Bente

A2 - Mariani, Joseph

A2 - Moreno, Asuncion

A2 - Odijk, Jan

A2 - Piperidis, Stelios

PB - European Language Resources Association

CY - Paris

T2 - The 10th International Conference on Language Resources and Evaluation. LREC 2016

Y2 - 23 May 2016 through 28 May 2016

ER -

Liyanapathirana J, Popescu-Belis A. Using the TED Talks to Evaluate Spoken Post-editing of Machine Translation. I Calzolari N, Choukri K, Declerck T, Grobelnik M, Maegaard B, Mariani J, Moreno A, Odijk J, Piperidis S, red., The LREC 2016 Proceedings: Tenth International Conference on Language Resources and Evaluation. Paris: European Language Resources Association. 2016. s. 2232-2239