Post-editing Neural Machine Translation versus Phrase-based Machine Translation for English–Chinese

Yanfang Jia, Michael Carl, Xiangling Wang

Research output: Contribution to journalJournal articleResearchpeer-review

Abstract

This paper aims to shed light on the post-editing process of the recently-introduced neural machine translation (NMT) paradigm. Using simple and more complex texts, we first evaluate the output quality from English to Chinese phrase-based statistical (PBSMT) and NMT systems. Nine raters assess the MT quality in terms of fluency and accuracy and find that NMT produces higher-rated translations than PBSMT for both texts. Then we analyze the effort expended by 68 student translators during HT and when post-editing NMT and PBSMT output. Our measures of post-editing effort are all positively correlated for both NMT and PBSMT post-editing. Our findings suggest that although post-editing output from NMT is not always significantly faster than post-editing PBSMT, it significantly reduces the technical and cognitive effort. We also find that, in contrast to HT, post-editing effort is not necessarily correlated with source text complexity.
Original languageEnglish
JournalMachine Translation
Volume33
Issue number1-2
Pages (from-to)9-29
Number of pages21
ISSN0922-6567
DOIs
Publication statusPublished - 8 Mar 2019
Externally publishedYes

Bibliographical note

Epub ahead of print. Published online: 8. March 2019

Keywords

  • Neural machine translation
  • Phrase-based statistical machine translation
  • Temporal effort
  • Technical effort
  • Cognitive effort
  • Human assessment

Cite this

Jia, Yanfang ; Carl, Michael ; Wang, Xiangling. / Post-editing Neural Machine Translation versus Phrase-based Machine Translation for English–Chinese. In: Machine Translation. 2019 ; Vol. 33, No. 1-2. pp. 9-29.
@article{67d3806210374d1589580381ed749e2c,
title = "Post-editing Neural Machine Translation versus Phrase-based Machine Translation for English–Chinese",
abstract = "This paper aims to shed light on the post-editing process of the recently-introduced neural machine translation (NMT) paradigm. Using simple and more complex texts, we first evaluate the output quality from English to Chinese phrase-based statistical (PBSMT) and NMT systems. Nine raters assess the MT quality in terms of fluency and accuracy and find that NMT produces higher-rated translations than PBSMT for both texts. Then we analyze the effort expended by 68 student translators during HT and when post-editing NMT and PBSMT output. Our measures of post-editing effort are all positively correlated for both NMT and PBSMT post-editing. Our findings suggest that although post-editing output from NMT is not always significantly faster than post-editing PBSMT, it significantly reduces the technical and cognitive effort. We also find that, in contrast to HT, post-editing effort is not necessarily correlated with source text complexity.",
keywords = "Neural machine translation, Phrase-based statistical machine translation, Temporal effort, Technical effort, Cognitive effort, Human assessment, Neural machine translation, Phrase-based statistical machine translation, Temporal effort, Technical effort, Cognitive effort, Human assessment",
author = "Yanfang Jia and Michael Carl and Xiangling Wang",
note = "Epub ahead of print. Published online: 8. March 2019",
year = "2019",
month = "3",
day = "8",
doi = "10.1007/s10590-019-09229-6",
language = "English",
volume = "33",
pages = "9--29",
journal = "Machine Translation",
issn = "0922-6567",
publisher = "Springer",
number = "1-2",

}

Post-editing Neural Machine Translation versus Phrase-based Machine Translation for English–Chinese. / Jia, Yanfang; Carl, Michael; Wang, Xiangling.

In: Machine Translation, Vol. 33, No. 1-2, 08.03.2019, p. 9-29.

Research output: Contribution to journalJournal articleResearchpeer-review

TY - JOUR

T1 - Post-editing Neural Machine Translation versus Phrase-based Machine Translation for English–Chinese

AU - Jia, Yanfang

AU - Carl, Michael

AU - Wang, Xiangling

N1 - Epub ahead of print. Published online: 8. March 2019

PY - 2019/3/8

Y1 - 2019/3/8

N2 - This paper aims to shed light on the post-editing process of the recently-introduced neural machine translation (NMT) paradigm. Using simple and more complex texts, we first evaluate the output quality from English to Chinese phrase-based statistical (PBSMT) and NMT systems. Nine raters assess the MT quality in terms of fluency and accuracy and find that NMT produces higher-rated translations than PBSMT for both texts. Then we analyze the effort expended by 68 student translators during HT and when post-editing NMT and PBSMT output. Our measures of post-editing effort are all positively correlated for both NMT and PBSMT post-editing. Our findings suggest that although post-editing output from NMT is not always significantly faster than post-editing PBSMT, it significantly reduces the technical and cognitive effort. We also find that, in contrast to HT, post-editing effort is not necessarily correlated with source text complexity.

AB - This paper aims to shed light on the post-editing process of the recently-introduced neural machine translation (NMT) paradigm. Using simple and more complex texts, we first evaluate the output quality from English to Chinese phrase-based statistical (PBSMT) and NMT systems. Nine raters assess the MT quality in terms of fluency and accuracy and find that NMT produces higher-rated translations than PBSMT for both texts. Then we analyze the effort expended by 68 student translators during HT and when post-editing NMT and PBSMT output. Our measures of post-editing effort are all positively correlated for both NMT and PBSMT post-editing. Our findings suggest that although post-editing output from NMT is not always significantly faster than post-editing PBSMT, it significantly reduces the technical and cognitive effort. We also find that, in contrast to HT, post-editing effort is not necessarily correlated with source text complexity.

KW - Neural machine translation

KW - Phrase-based statistical machine translation

KW - Temporal effort

KW - Technical effort

KW - Cognitive effort

KW - Human assessment

KW - Neural machine translation

KW - Phrase-based statistical machine translation

KW - Temporal effort

KW - Technical effort

KW - Cognitive effort

KW - Human assessment

UR - https://sfx-45cbs.hosted.exlibrisgroup.com/45cbs?url_ver=Z39.88-2004&url_ctx_fmt=info:ofi/fmt:kev:mtx:ctx&ctx_enc=info:ofi/enc:UTF-8&ctx_ver=Z39.88-2004&rfr_id=info:sid/sfxit.com:azlist&sfx.ignore_date_threshold=1&rft.object_id=954927381279&rft.object_portfolio_id=&svc.holdings=yes&svc.fulltext=yes

U2 - 10.1007/s10590-019-09229-6

DO - 10.1007/s10590-019-09229-6

M3 - Journal article

VL - 33

SP - 9

EP - 29

JO - Machine Translation

JF - Machine Translation

SN - 0922-6567

IS - 1-2

ER -