Word Transition Entropy as an Indicator for Expected Machine Translation Quality

Michael Carl, Moritz Schaeffer

    Research output: Chapter in Book/Report/Conference proceedingArticle in proceedingsResearchpeer-review

    Abstract

    While most machine translation evaluation techniques (BLEU, NIST, TER, METEOR) assess translation quality based on a set of reference translations, we suggest to evaluate the literality of a set of (human or machine generated) translations to infer their potential quality. We provide evidence which suggests that more literal translations are produced more easily, by humans and machine, and are also less error prone. Literal translations may not be appropriate or even possible for all languages, types of texts, and translation purposes. However, in this paper we show that an assessment of the literality of translations allows us to (1) evaluate human and machine translations in a similar fashion and (2) may be instrumental to predict machine translation quality scores,
    Original languageEnglish
    Title of host publicationProceedings of the Workshop on Automatic and Manual Metrics for Operational Translation Evaluation. MTE 2014
    EditorsKeith J. Miller, Lucia Specia, Kim Harris, Stacey Bailey
    Place of PublicationParis
    PublisherEuropean Language Resources Association
    Publication date2014
    Pages45-50
    Publication statusPublished - 2014
    EventThe Workshop on Automatic and Manual Metrics for Operational Translation Evaluation. MTE 2014 - Harpa Conference Centre , Reykjavik, Iceland
    Duration: 26 May 201426 May 2014
    http://mte2014.github.io/

    Workshop

    WorkshopThe Workshop on Automatic and Manual Metrics for Operational Translation Evaluation. MTE 2014
    LocationHarpa Conference Centre
    CountryIceland
    CityReykjavik
    Period26/05/201426/05/2014
    OtherHeld in connection to the LREC 2014: The 9th edition of the Language Resources and Evaluation Conference
    Internet address

    Bibliographical note

    Workshop on The Ninth International Conference on Language Resources and Evaluation. LREC 2014

    Cite this

    Carl, M., & Schaeffer, M. (2014). Word Transition Entropy as an Indicator for Expected Machine Translation Quality. In K. J. Miller, L. Specia, K. Harris, & S. Bailey (Eds.), Proceedings of the Workshop on Automatic and Manual Metrics for Operational Translation Evaluation. MTE 2014 (pp. 45-50). European Language Resources Association. http://mte2014.github.io/MTE2014-Workshop-Proceedings.pdf