Evaluating the Effects of Interactivity in a Post-Editing Workbench

Nancy  Underwood; Bartolomé  Mesa-Lao; Mercedes Garcia Martinez; Michael Carl; Vicent Alabau; Jesus Gonzalez-Rubio; Luis A. Leiva; Germán Sanchis-Trilles; Daniel  Ortíz-Martínez; Francisco Casacuberta

Evaluating the Effects of Interactivity in a Post-Editing Workbench

Nancy Underwood, Bartolomé Mesa-Lao, Mercedes Garcia Martinez, Michael Carl, Vicent Alabau, Jesus Gonzalez-Rubio, Luis A. Leiva, Germán Sanchis-Trilles, Daniel Ortíz-Martínez , Francisco Casacuberta

Universitat Politècnica de València

Publikation: Bidrag til bog/antologi/rapport › Konferencebidrag i proceedings › Forskning › peer review

Abstract

This paper describes the field trial and subsequent evaluation of a post-editing workbench which is currently under development in the EU-funded CasMaCat project. Based on user evaluations of the initial prototype of the workbench, this second prototype of the workbench includes a number of interactive features designed to improve productivity and user satisfaction. Using CasMaCat's own facilities for logging keystrokes and eye tracking, data were collected from nine post-editors in a professional setting. These data were then used to investigate the effects of the interactive features on productivity, quality, user satisfaction and cognitive load as reflected in the post-editors’ gaze activity. These quantitative results are combined with the qualitative results derived from user questionnaires and interviews conducted with all the participants.

Originalsprog	Engelsk
Titel	Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC'14)
Redaktører	Nicoletta Calzolari, Khalid Choukri , Thierry Declerck , Hrafn Loftsson , Bente Maegaard, Joseph Mariani , Asuncion Moreno , Jan Odijk , Stelios Piperidis
Antal sider	7
Udgivelsessted	Paris
Forlag	European Language Resources Association
Publikationsdato	2014
Sider	553-559
ISBN (Trykt)	9782951740884
Status	Udgivet - 2014
Begivenhed	The 9th International Conference on Language Resources and Evaluation. LREC 2014 - Reykjavik, Island Varighed: 26 maj 2014 → 31 maj 2014 Konferencens nummer: 9 http://lrec2014.lrec-conf.org/en/

Konference

Konference	The 9th International Conference on Language Resources and Evaluation. LREC 2014
Nummer	9
Land/Område	Island
By	Reykjavik
Periode	26/05/2014 → 31/05/2014
Internetadresse	http://lrec2014.lrec-conf.org/en/

Emneord

Post-editing
Machine translation
Evaluation
Interactive translation prediction

Adgang til dokumentet

http://www.lrec-conf.org/proceedings/lrec2014/summaries/289.html

Handle.net

Persistent link

Citationsformater

Underwood, N., Mesa-Lao, B., Martinez, M. G., Carl, M., Alabau, V., Gonzalez-Rubio, J., A. Leiva, L., Sanchis-Trilles, G., Ortíz-Martínez , D., & Casacuberta, F. (2014). Evaluating the Effects of Interactivity in a Post-Editing Workbench. I N. Calzolari, K. Choukri , T. Declerck , H. Loftsson , B. Maegaard, J. Mariani , A. Moreno , J. Odijk , & S. Piperidis (red.), Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC'14) (s. 553-559). European Language Resources Association. http://www.lrec-conf.org/proceedings/lrec2014/summaries/289.html

Underwood, Nancy ; Mesa-Lao, Bartolomé ; Martinez, Mercedes Garcia et al. / Evaluating the Effects of Interactivity in a Post-Editing Workbench. Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC'14). red. / Nicoletta Calzolari ; Khalid Choukri ; Thierry Declerck ; Hrafn Loftsson ; Bente Maegaard ; Joseph Mariani ; Asuncion Moreno ; Jan Odijk ; Stelios Piperidis. Paris : European Language Resources Association, 2014. s. 553-559

@inproceedings{5363a78f93cb4ec3966752d7f3009f7d,

title = "Evaluating the Effects of Interactivity in a Post-Editing Workbench",

abstract = "This paper describes the field trial and subsequent evaluation of a post-editing workbench which is currently under development in the EU-funded CasMaCat project. Based on user evaluations of the initial prototype of the workbench, this second prototype of the workbench includes a number of interactive features designed to improve productivity and user satisfaction. Using CasMaCat's own facilities for logging keystrokes and eye tracking, data were collected from nine post-editors in a professional setting. These data were then used to investigate the effects of the interactive features on productivity, quality, user satisfaction and cognitive load as reflected in the post-editors{\textquoteright} gaze activity. These quantitative results are combined with the qualitative results derived from user questionnaires and interviews conducted with all the participants.",

keywords = "Post-editing, Machine translation, Evaluation, Interactive translation prediction",

author = "Nancy Underwood and Bartolom{\'e} Mesa-Lao and Martinez, {Mercedes Garcia} and Michael Carl and Vicent Alabau and Jesus Gonzalez-Rubio and {A. Leiva}, Luis and Germ{\'a}n Sanchis-Trilles and Daniel Ort{\'i}z-Mart{\'i}nez and Francisco Casacuberta",

year = "2014",

language = "English",

isbn = "9782951740884",

pages = "553--559",

editor = "Calzolari, {Nicoletta } and {Choukri }, {Khalid } and {Declerck }, {Thierry } and {Loftsson }, Hrafn and Maegaard, {Bente } and {Mariani }, {Joseph } and {Moreno }, {Asuncion } and {Odijk }, {Jan } and Piperidis, {Stelios }",

booktitle = "Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC'14)",

publisher = "European Language Resources Association",

address = "France",

note = "The 9th International Conference on Language Resources and Evaluation. LREC 2014 ; Conference date: 26-05-2014 Through 31-05-2014",

url = "http://lrec2014.lrec-conf.org/en/",

}

Underwood, N, Mesa-Lao, B, Martinez, MG, Carl, M, Alabau, V, Gonzalez-Rubio, J, A. Leiva, L, Sanchis-Trilles, G, Ortíz-Martínez , D & Casacuberta, F 2014, Evaluating the Effects of Interactivity in a Post-Editing Workbench. i N Calzolari, K Choukri , T Declerck , H Loftsson , B Maegaard, J Mariani , A Moreno , J Odijk & S Piperidis (red), Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC'14). European Language Resources Association, Paris, s. 553-559, The 9th International Conference on Language Resources and Evaluation. LREC 2014, Reykjavik, Island, 26/05/2014. <http://www.lrec-conf.org/proceedings/lrec2014/summaries/289.html>

Evaluating the Effects of Interactivity in a Post-Editing Workbench. / Underwood, Nancy ; Mesa-Lao, Bartolomé ; Martinez, Mercedes Garcia et al.
Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC'14). red. / Nicoletta Calzolari; Khalid Choukri ; Thierry Declerck ; Hrafn Loftsson ; Bente Maegaard; Joseph Mariani ; Asuncion Moreno ; Jan Odijk ; Stelios Piperidis. Paris: European Language Resources Association, 2014. s. 553-559.

Publikation: Bidrag til bog/antologi/rapport › Konferencebidrag i proceedings › Forskning › peer review

TY - GEN

T1 - Evaluating the Effects of Interactivity in a Post-Editing Workbench

AU - Underwood, Nancy

AU - Mesa-Lao, Bartolomé

AU - Martinez, Mercedes Garcia

AU - Carl, Michael

AU - Alabau, Vicent

AU - Gonzalez-Rubio, Jesus

AU - A. Leiva, Luis

AU - Sanchis-Trilles, Germán

AU - Ortíz-Martínez , Daniel

AU - Casacuberta, Francisco

N1 - Conference code: 9

PY - 2014

Y1 - 2014

N2 - This paper describes the field trial and subsequent evaluation of a post-editing workbench which is currently under development in the EU-funded CasMaCat project. Based on user evaluations of the initial prototype of the workbench, this second prototype of the workbench includes a number of interactive features designed to improve productivity and user satisfaction. Using CasMaCat's own facilities for logging keystrokes and eye tracking, data were collected from nine post-editors in a professional setting. These data were then used to investigate the effects of the interactive features on productivity, quality, user satisfaction and cognitive load as reflected in the post-editors’ gaze activity. These quantitative results are combined with the qualitative results derived from user questionnaires and interviews conducted with all the participants.

AB - This paper describes the field trial and subsequent evaluation of a post-editing workbench which is currently under development in the EU-funded CasMaCat project. Based on user evaluations of the initial prototype of the workbench, this second prototype of the workbench includes a number of interactive features designed to improve productivity and user satisfaction. Using CasMaCat's own facilities for logging keystrokes and eye tracking, data were collected from nine post-editors in a professional setting. These data were then used to investigate the effects of the interactive features on productivity, quality, user satisfaction and cognitive load as reflected in the post-editors’ gaze activity. These quantitative results are combined with the qualitative results derived from user questionnaires and interviews conducted with all the participants.

KW - Post-editing

KW - Machine translation

KW - Evaluation

KW - Interactive translation prediction

M3 - Article in proceedings

SN - 9782951740884

SP - 553

EP - 559

BT - Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC'14)

A2 - Calzolari, Nicoletta

A2 - Choukri , Khalid

A2 - Declerck , Thierry

A2 - Loftsson , Hrafn

A2 - Maegaard, Bente

A2 - Mariani , Joseph

A2 - Moreno , Asuncion

A2 - Odijk , Jan

A2 - Piperidis, Stelios

PB - European Language Resources Association

CY - Paris

T2 - The 9th International Conference on Language Resources and Evaluation. LREC 2014

Y2 - 26 May 2014 through 31 May 2014

ER -

Underwood N, Mesa-Lao B, Martinez MG, Carl M, Alabau V, Gonzalez-Rubio J et al. Evaluating the Effects of Interactivity in a Post-Editing Workbench. I Calzolari N, Choukri K, Declerck T, Loftsson H, Maegaard B, Mariani J, Moreno A, Odijk J, Piperidis S, red., Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC'14). Paris: European Language Resources Association. 2014. s. 553-559