Machine Translation Errors and the Translation Process: A Study across Different Languages

Michael Carl, María Cristina Toledo Báez

Publikation: Bidrag til tidsskriftTidsskriftartikelForskningpeer review

Resumé

The paper describes an experiment in which two groups of translators annotate Spanish and simplified Chinese MT output of the same English source texts (ST) using an MQMderived annotation schema. Annotators first fragmented the ST and MT output (i.e. the target text TT) into alignment groups (AGs) and then labelled the AGs with an error code. We investigate the inter-annotator agreement of the AGs and their error annotations. Then, we correlate the average error agreement (i.e. the MT error evidence) with translation process data that we collected during the translation production of the same English texts in previous studies. We find that MT accuracy errors with higher errorevidence scores have an effect on the production and reading durations during postediting. We also find that that from-scratch translation is more difficult for ST words which have more evident MT accuracy errors. Surprisingly, Spanish MT accuracy errors also correlate with total ST reading time for translations (post-editing and from-scratch translation) into very different languages. We conclude that expressions with MT accuracy issues into one language (English-to-Spanish) are likely to be difficult to translate also into other languages for humans and for computers – while this does not hold for MT fluency errors.
OriginalsprogEngelsk
TidsskriftJournal of Specialised Translation
Udgave nummer31
Sider (fra-til)107-132
Antal sider26
ISSN1740-357X
StatusUdgivet - jan. 2019
Udgivet eksterntJa

Emneord

  • Translation quality assessment
  • Machine error annotation
  • Inter-rater agreement
  • Post-editing
  • From-scratch translation
  • Translation accuracy
  • Translation effort
  • Translation modes

Citer dette

@article{3cd23dca61ba4660a02dda1426165187,
title = "Machine Translation Errors and the Translation Process: A Study across Different Languages",
abstract = "The paper describes an experiment in which two groups of translators annotate Spanish and simplified Chinese MT output of the same English source texts (ST) using an MQMderived annotation schema. Annotators first fragmented the ST and MT output (i.e. the target text TT) into alignment groups (AGs) and then labelled the AGs with an error code. We investigate the inter-annotator agreement of the AGs and their error annotations. Then, we correlate the average error agreement (i.e. the MT error evidence) with translation process data that we collected during the translation production of the same English texts in previous studies. We find that MT accuracy errors with higher errorevidence scores have an effect on the production and reading durations during postediting. We also find that that from-scratch translation is more difficult for ST words which have more evident MT accuracy errors. Surprisingly, Spanish MT accuracy errors also correlate with total ST reading time for translations (post-editing and from-scratch translation) into very different languages. We conclude that expressions with MT accuracy issues into one language (English-to-Spanish) are likely to be difficult to translate also into other languages for humans and for computers – while this does not hold for MT fluency errors.",
keywords = "Translation quality assessment, Machine error annotation, Inter-rater agreement, Post-editing, From-scratch translation, Translation accuracy, Translation effort, Translation modes, Translation quality assessment, Machine error annotation, Inter-rater agreement, Post-editing, From-scratch translation, Translation accuracy, Translation effort, Translation modes",
author = "Michael Carl and B{\'a}ez, {Mar{\'i}a Cristina Toledo}",
year = "2019",
month = "1",
language = "English",
pages = "107--132",
journal = "Journal of Specialised Translation",
issn = "1740-357X",
publisher = "Journal of Specialised Translation",
number = "31",

}

Machine Translation Errors and the Translation Process : A Study across Different Languages. / Carl, Michael; Báez, María Cristina Toledo.

I: Journal of Specialised Translation, Nr. 31, 01.2019, s. 107-132.

Publikation: Bidrag til tidsskriftTidsskriftartikelForskningpeer review

TY - JOUR

T1 - Machine Translation Errors and the Translation Process

T2 - A Study across Different Languages

AU - Carl, Michael

AU - Báez, María Cristina Toledo

PY - 2019/1

Y1 - 2019/1

N2 - The paper describes an experiment in which two groups of translators annotate Spanish and simplified Chinese MT output of the same English source texts (ST) using an MQMderived annotation schema. Annotators first fragmented the ST and MT output (i.e. the target text TT) into alignment groups (AGs) and then labelled the AGs with an error code. We investigate the inter-annotator agreement of the AGs and their error annotations. Then, we correlate the average error agreement (i.e. the MT error evidence) with translation process data that we collected during the translation production of the same English texts in previous studies. We find that MT accuracy errors with higher errorevidence scores have an effect on the production and reading durations during postediting. We also find that that from-scratch translation is more difficult for ST words which have more evident MT accuracy errors. Surprisingly, Spanish MT accuracy errors also correlate with total ST reading time for translations (post-editing and from-scratch translation) into very different languages. We conclude that expressions with MT accuracy issues into one language (English-to-Spanish) are likely to be difficult to translate also into other languages for humans and for computers – while this does not hold for MT fluency errors.

AB - The paper describes an experiment in which two groups of translators annotate Spanish and simplified Chinese MT output of the same English source texts (ST) using an MQMderived annotation schema. Annotators first fragmented the ST and MT output (i.e. the target text TT) into alignment groups (AGs) and then labelled the AGs with an error code. We investigate the inter-annotator agreement of the AGs and their error annotations. Then, we correlate the average error agreement (i.e. the MT error evidence) with translation process data that we collected during the translation production of the same English texts in previous studies. We find that MT accuracy errors with higher errorevidence scores have an effect on the production and reading durations during postediting. We also find that that from-scratch translation is more difficult for ST words which have more evident MT accuracy errors. Surprisingly, Spanish MT accuracy errors also correlate with total ST reading time for translations (post-editing and from-scratch translation) into very different languages. We conclude that expressions with MT accuracy issues into one language (English-to-Spanish) are likely to be difficult to translate also into other languages for humans and for computers – while this does not hold for MT fluency errors.

KW - Translation quality assessment

KW - Machine error annotation

KW - Inter-rater agreement

KW - Post-editing

KW - From-scratch translation

KW - Translation accuracy

KW - Translation effort

KW - Translation modes

KW - Translation quality assessment

KW - Machine error annotation

KW - Inter-rater agreement

KW - Post-editing

KW - From-scratch translation

KW - Translation accuracy

KW - Translation effort

KW - Translation modes

UR - https://sfx-45cbs.hosted.exlibrisgroup.com/45cbs?url_ver=Z39.88-2004&url_ctx_fmt=info:ofi/fmt:kev:mtx:ctx&ctx_enc=info:ofi/enc:UTF-8&ctx_ver=Z39.88-2004&rfr_id=info:sid/sfxit.com:azlist&sfx.ignore_date_threshold=1&rft.object_id=1000000000281223&rft.object_portfolio_id=&svc.holdings=yes&svc.fulltext=yes

M3 - Journal article

SP - 107

EP - 132

JO - Journal of Specialised Translation

JF - Journal of Specialised Translation

SN - 1740-357X

IS - 31

ER -