Evaluating the Effects of Interactivity in a Post-Editing Workbench

Nancy Underwood, Bartolomé Mesa-Lao, Mercedes Garcia Martinez, Michael Carl, Vicent Alabau, Jesus Gonzalez-Rubio, Luis A. Leiva, Germán Sanchis-Trilles, Daniel Ortíz-Martínez , Francisco Casacuberta

    Research output: Chapter in Book/Report/Conference proceedingArticle in proceedingsResearchpeer-review

    Abstract

    This paper describes the field trial and subsequent evaluation of a post-editing workbench which is currently under development in the EU-funded CasMaCat project. Based on user evaluations of the initial prototype of the workbench, this second prototype of the workbench includes a number of interactive features designed to improve productivity and user satisfaction. Using CasMaCat's own facilities for logging keystrokes and eye tracking, data were collected from nine post-editors in a professional setting. These data were then used to investigate the effects of the interactive features on productivity, quality, user satisfaction and cognitive load as reflected in the post-editors’ gaze activity. These quantitative results are combined with the qualitative results derived from user questionnaires and interviews conducted with all the participants.
    Original languageEnglish
    Title of host publicationProceedings of the Ninth International Conference on Language Resources and Evaluation (LREC'14)
    EditorsNicoletta Calzolari, Khalid Choukri , Thierry Declerck , Hrafn Loftsson , Bente Maegaard, Joseph Mariani , Asuncion Moreno , Jan Odijk , Stelios Piperidis
    Number of pages7
    Place of PublicationParis
    PublisherEuropean Language Resources Association
    Publication date2014
    Pages553-559
    ISBN (Print)9782951740884
    Publication statusPublished - 2014
    EventThe 9th International Conference on Language Resources and Evaluation. LREC 2014 - Reykjavik, Iceland
    Duration: 26 May 201431 May 2014
    Conference number: 9
    http://lrec2014.lrec-conf.org/en/

    Conference

    ConferenceThe 9th International Conference on Language Resources and Evaluation. LREC 2014
    Number9
    CountryIceland
    CityReykjavik
    Period26/05/201431/05/2014
    Internet address

    Cite this

    Underwood, N., Mesa-Lao, B., Martinez, M. G., Carl, M., Alabau, V., Gonzalez-Rubio, J., ... Casacuberta, F. (2014). Evaluating the Effects of Interactivity in a Post-Editing Workbench. In N. Calzolari, K. Choukri , T. Declerck , H. Loftsson , B. Maegaard, J. Mariani , A. Moreno , J. Odijk , ... S. Piperidis (Eds.), Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC'14) (pp. 553-559). Paris: European Language Resources Association.
    Underwood, Nancy ; Mesa-Lao, Bartolomé ; Martinez, Mercedes Garcia ; Carl, Michael ; Alabau, Vicent ; Gonzalez-Rubio, Jesus ; A. Leiva, Luis ; Sanchis-Trilles, Germán ; Ortíz-Martínez , Daniel ; Casacuberta, Francisco. / Evaluating the Effects of Interactivity in a Post-Editing Workbench. Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC'14). editor / Nicoletta Calzolari ; Khalid Choukri ; Thierry Declerck ; Hrafn Loftsson ; Bente Maegaard ; Joseph Mariani ; Asuncion Moreno ; Jan Odijk ; Stelios Piperidis. Paris : European Language Resources Association, 2014. pp. 553-559
    @inproceedings{5363a78f93cb4ec3966752d7f3009f7d,
    title = "Evaluating the Effects of Interactivity in a Post-Editing Workbench",
    abstract = "This paper describes the field trial and subsequent evaluation of a post-editing workbench which is currently under development in the EU-funded CasMaCat project. Based on user evaluations of the initial prototype of the workbench, this second prototype of the workbench includes a number of interactive features designed to improve productivity and user satisfaction. Using CasMaCat's own facilities for logging keystrokes and eye tracking, data were collected from nine post-editors in a professional setting. These data were then used to investigate the effects of the interactive features on productivity, quality, user satisfaction and cognitive load as reflected in the post-editors’ gaze activity. These quantitative results are combined with the qualitative results derived from user questionnaires and interviews conducted with all the participants.",
    keywords = "Post-editing, Machine translation, Evaluation, Interactive translation prediction",
    author = "Nancy Underwood and Bartolom{\'e} Mesa-Lao and Martinez, {Mercedes Garcia} and Michael Carl and Vicent Alabau and Jesus Gonzalez-Rubio and {A. Leiva}, Luis and Germ{\'a}n Sanchis-Trilles and Daniel Ort{\'i}z-Mart{\'i}nez and Francisco Casacuberta",
    year = "2014",
    language = "English",
    isbn = "9782951740884",
    pages = "553--559",
    editor = "Calzolari, {Nicoletta } and {Choukri }, {Khalid } and {Declerck }, {Thierry } and {Loftsson }, Hrafn and Maegaard, {Bente } and {Mariani }, {Joseph } and {Moreno }, {Asuncion } and {Odijk }, {Jan } and Piperidis, {Stelios }",
    booktitle = "Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC'14)",
    publisher = "European Language Resources Association",
    address = "France",

    }

    Underwood, N, Mesa-Lao, B, Martinez, MG, Carl, M, Alabau, V, Gonzalez-Rubio, J, A. Leiva, L, Sanchis-Trilles, G, Ortíz-Martínez , D & Casacuberta, F 2014, Evaluating the Effects of Interactivity in a Post-Editing Workbench. in N Calzolari, K Choukri , T Declerck , H Loftsson , B Maegaard, J Mariani , A Moreno , J Odijk & S Piperidis (eds), Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC'14). European Language Resources Association, Paris, pp. 553-559, Reykjavik, Iceland, 26/05/2014.

    Evaluating the Effects of Interactivity in a Post-Editing Workbench. / Underwood, Nancy ; Mesa-Lao, Bartolomé ; Martinez, Mercedes Garcia; Carl, Michael; Alabau, Vicent; Gonzalez-Rubio, Jesus; A. Leiva, Luis; Sanchis-Trilles, Germán; Ortíz-Martínez , Daniel ; Casacuberta, Francisco.

    Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC'14). ed. / Nicoletta Calzolari; Khalid Choukri ; Thierry Declerck ; Hrafn Loftsson ; Bente Maegaard; Joseph Mariani ; Asuncion Moreno ; Jan Odijk ; Stelios Piperidis. Paris : European Language Resources Association, 2014. p. 553-559.

    Research output: Chapter in Book/Report/Conference proceedingArticle in proceedingsResearchpeer-review

    TY - GEN

    T1 - Evaluating the Effects of Interactivity in a Post-Editing Workbench

    AU - Underwood, Nancy

    AU - Mesa-Lao, Bartolomé

    AU - Martinez, Mercedes Garcia

    AU - Carl, Michael

    AU - Alabau, Vicent

    AU - Gonzalez-Rubio, Jesus

    AU - A. Leiva, Luis

    AU - Sanchis-Trilles, Germán

    AU - Ortíz-Martínez , Daniel

    AU - Casacuberta, Francisco

    PY - 2014

    Y1 - 2014

    N2 - This paper describes the field trial and subsequent evaluation of a post-editing workbench which is currently under development in the EU-funded CasMaCat project. Based on user evaluations of the initial prototype of the workbench, this second prototype of the workbench includes a number of interactive features designed to improve productivity and user satisfaction. Using CasMaCat's own facilities for logging keystrokes and eye tracking, data were collected from nine post-editors in a professional setting. These data were then used to investigate the effects of the interactive features on productivity, quality, user satisfaction and cognitive load as reflected in the post-editors’ gaze activity. These quantitative results are combined with the qualitative results derived from user questionnaires and interviews conducted with all the participants.

    AB - This paper describes the field trial and subsequent evaluation of a post-editing workbench which is currently under development in the EU-funded CasMaCat project. Based on user evaluations of the initial prototype of the workbench, this second prototype of the workbench includes a number of interactive features designed to improve productivity and user satisfaction. Using CasMaCat's own facilities for logging keystrokes and eye tracking, data were collected from nine post-editors in a professional setting. These data were then used to investigate the effects of the interactive features on productivity, quality, user satisfaction and cognitive load as reflected in the post-editors’ gaze activity. These quantitative results are combined with the qualitative results derived from user questionnaires and interviews conducted with all the participants.

    KW - Post-editing

    KW - Machine translation

    KW - Evaluation

    KW - Interactive translation prediction

    M3 - Article in proceedings

    SN - 9782951740884

    SP - 553

    EP - 559

    BT - Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC'14)

    A2 - Calzolari, Nicoletta

    A2 - Choukri , Khalid

    A2 - Declerck , Thierry

    A2 - Loftsson , Hrafn

    A2 - Maegaard, Bente

    A2 - Mariani , Joseph

    A2 - Moreno , Asuncion

    A2 - Odijk , Jan

    A2 - Piperidis, Stelios

    PB - European Language Resources Association

    CY - Paris

    ER -

    Underwood N, Mesa-Lao B, Martinez MG, Carl M, Alabau V, Gonzalez-Rubio J et al. Evaluating the Effects of Interactivity in a Post-Editing Workbench. In Calzolari N, Choukri K, Declerck T, Loftsson H, Maegaard B, Mariani J, Moreno A, Odijk J, Piperidis S, editors, Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC'14). Paris: European Language Resources Association. 2014. p. 553-559