An Economic Model of Machine Translation

With the advent of free, online translation services such as Google Translate, many people are now able to obtain information relatively effortlessly from a wide variety of foreign language sources. The translations from these services are often worse than those provided by professional, human translators, however, and the tradeoff between these two alternatives is not always clear. When should a professional be used, and when is machine translation sufficient? In this study, we discuss factors involved in the decision and illustrate their use with a predictive model.


Introduction
Spending on translation services was approximately US$45 billion in 2020 [11], and machine translation is playing a greater role as humans cannot keep up the pace with demand [10]. For example, less than 0.1% of digital content is currently being translated [9]. However, the decision when to use a human or a machine for translation is not always evident, and there has been little attempt to develop an economic model of the variables involved in this choice ( [6], [12], [24], [26]).
In this paper, we show that the costs of language translation are influenced by the urgency and required accuracy of the information, which in turn is determined by the importance of the material. A group of 47 students used Google Translate to obtain the English equivalent of two text samples written in Korean. Results show that the multilinear model of these variables had a statistically significant fit. The paper concludes with limitations and directions for future research.

Background
Some researchers believe that machines could ultimately eliminate the need for humans in the language translation process [21]. However, others, especially professional translators, believe artificial intelligence should have little or no role, principally because it takes away from their business ( [7], [14]). For example, an analysis of 13,000 tweets on Twitter showed that negative perceptions of machine translation outweighed positive ones by a ratio of 5 to 1 [17], even though professional translators can benefit from automated assistance.
Researchers have investigated the costs of language translation for at least 60 years [18], but there has been relatively little effort into developing an economic model of the factors involved. Economic theory suggests that translation costs, a consumer's willingness to pay, required accuracy, the importance and urgency of the information involved, translation frequency, and many other variables are important ( [3], [19]). Some of these variables are discussed in more detail below:

1.
Cost Several online translators such as Google Translate and Microsoft Bing Translator are available for free [23], up to certain limits of words and usage frequency. For example, at one time, Microsoft charged US$10 per million characters translated per month, with discounts at higher volumes, e.g., US$9 per million characters for 64 million characters per month and less than US$8 per million characters for one billion characters per month [20]. In comparison, Google Translate charged US$20 per million characters, but the fee is only for characters actually translated. Professional human translators charge various rates. In one study [8], the cost to translate a 30-page survey in English to Spanish was $10,426. According to one site (https://www.costowl.com/b2b/translation-servicedocument-cost.html), translations costs can range from US$30 to $125 per page or $0.07 to $0.40 per word depending upon the text difficulty and the languages involved. For example, 5,000 words translated from Chinese to English costs $1,054.27 while 5,000 words translated from Arabic to English costs $1,331.71.

2.
Speed Information has a time value. That is, at a certain point, information could be worthless. If a translation is needed for a meeting in one hour, receiving the text a day later is of no value [25]. It is obvious that machines are much faster than humans in text manipulation, and computers can generate words much faster than a human can type. In one study [1], for example, two humans fluent in English and Spanish translated 10 samples of English text in an average of 61.5 minutes while machine translation took 0.315 minutes, approximately 195 times faster than the human translators. Several online sites such as https://translated.com/ allow users to request translations from professionals in 145 language choices (many of these are dialects, e.g. Chinese -Traditional Hong Kong and Chinese -Traditional Macao). But, few if any translations can be obtained within at least several hours. Rush charges can cause rates to increase 25% to 50%.

3.
Importance and Accuracy Required accuracy depends upon the importance of the subject material. That is, the consequences of mistranslation must be considered. Mistranslations in some cases can cause harm to reputations, legal problems, and even death. Professional human translators can make mistakes, but they typically provide equivalent or better accuracy than machine translation [2]. By guaranteeing accuracy, professional translators can charge seven times more than the standard rate [15].
On the other hand, translation often does not need to be perfect [16], and machine translation might be a viable alternative. One study [4] showed that Google Translate was accurate enough with several languages to meet the TOEFL (Test of English as a Foreign Language) standards for several universities in the United States, and a translation with low accuracy is often preferable to no translation at all.
In one study [22], researchers found that human translation of 25 documents took from 17 hours to 6 days or 1.58 to 5.88 words per minute at a cost of US$130 to $1220 while machine translation with human postediting generated 10 to 30 words per minute, with equal accuracies, presumably at little or no cost. The study also showed that humans translated 11 documents containing an average of 721 words each to Amharic, Arabic, Burmese, Chinese, Farsi, Khmer, Lao, Karen, Nepali, Russian, Somali, Spanish, Swahili, Tigrinya, and Vietnamese in an average of 238.43 hours (0.06 words per minute) at an average cost of $0.67 per word. Editing after machine translation can increase speed and accuracy while lowering costs [13].

Decision Model
The purpose of this study is to develop a model showing the influence of accuracy, speed, and costs in the decision whether to use human or machine translation.
According to one estimate [5], humans can translate at about 0.18 words per second, but the speed of Google Translate is at least 155 words per second. However, a human fluent in the source and target language might not be available immediately. A professional translator might give a perfect translation, but the fee might be too much considering the estimated importance of the source text. Based upon these assumptions, consider the following scenarios: 1. An English speaker who knows very little Spanish would like to get the gist of a magazine article written in Spanish. The reader estimates the importance is 50% but is not willing to spend more than $US2. Machine translation might be 95% accurate, and it is free to use immediately. Human translation will take a day, is 100% accurate, and the cost is $100. Machine translation is probably the best choice. 2. A legal document written in French must be translated to German for a €10million business deal within a week. In this case, professional translation is better. 3. A medical diagnosis must be translated from English into Farsi while a patient is in the doctor's office. A Farsi interpreter will charge US$100 but will not be available until tomorrow. Google Translate is free, available immediately, but might be only 90% accurate. In this case, it is not clear which choice is better.
The choice of human or machine translation might be described by the following equation: (speed * weight) * (cost * weight) * (accuracy * weight) * limit This model gives an objective result, but it is probably not optimal, as shown with scenario 2. Further research including the possible use of neural networks is required to explore these variables more thoroughly.

Translation Study
In an attempt to validate part of this model, we used two different source texts written in Korean. The children's story shown in Appendix 1 was deemed to be relatively inconsequential and easy to read ( Two groups of undergraduate students (sizes 23 and 24) at a university in the northeastern region of the United States also edited the documents after translating to English with Google Translate. Afterwards, they answered the following questions: 1. This text was important (1 = disagree, 4 = neutral, 7 = agree) 2. How much would you pay a professional translator to translate this to English? (US dollars) 3. How long would you be willing to wait for this text to be translated? (minutes) 4. 4. How good do you think your final translation to English is? (0=cannot understand any of it, 5=many errors but somewhat understandable, 8=some errors but understandable, 10=perfect English) 5. How good do you think the translation should be? (0=cannot understand any of it, 5=many errors but somewhat understandable, 8=some errors but understandable, 10=perfect English) Results of the survey for the children's story are shown in Table 1. The estimated importance was not significantly different from the neutral value of 4 (p = 0.21), but the values for estimated accuracy and required accuracy were significantly higher than the neutral value of 5 (both with p < 0.001). The estimated translation accuracy was significantly less than the required accuracy (p = 0.031), but the accuracy was not significantly different from 8 -some errors but understandable (p = 0.066). That is, the students did not think the story was very important, but they thought the machine translations were relatively understandable. Surprisingly, they stated that they were willing to pay about $61 dollars for the translation.  Table 2 shows the results of the survey for the official document translation. The estimated importance was significantly higher than the neutral value (p < 0.001), indicating the students recognized the text as official information. The values for accuracy and required accuracy also were significantly higher than neutral (both with p < 0.001). The estimated translation accuracy was not significantly less than the required accuracy (p = 0.224). Even though the information was more important and had more words, this group of students was willing to pay only $42 for a translation, considerably less that the other group was willing to pay for a translation of the children's story. In both cases, the students took about one minute to translate and edit the Korean texts, well below the 105 and 87 minutes they said they would be willing to wait. One minute was the same amount of time that the Korean speaker took to use Google Translate and edit the results. However, the Korean human translator took 17 minutes to manually translate the children's story and 14 minutes for the official document (8.3 and 12.4 words per minute, respectively). Thus, the machine translation with post-editing took only about 5.9% and 7.1% of the time, respectively.
The accuracies of the translations also matter, and the machine-translated texts edited by the students were rated as 7.25 on average with a standard deviation of 1.91. The official document renditions by computer were rated as 7.42 on average with a standard deviation of 1.08. These evaluations of accuracy were not significantly different from what the students estimated (p = 0.49 and p = 0.46, respectively), and the easy text accuracy was not significantly different from what the students required for accuracy (p = 0.600). However, the translation accuracy for the difficult text was significantly less that what the students required (p = 0.005). Assuming the human translation accuracy has a rating of 9 out of 10, the accuracy of the students' edited machine translations for both the simple story and the official document were significantly lower (p = 0.036 and p < 0.001, respectively). For the story, the reduced accuracy probably does not matter, but it is more of a concern for the official document.
The times needed by the professional translator (17 and 14 minutes) were well below the times the students said they were willing to wait (105 and 89 minutes), and the accuracies of the human were significantly higher than the students' edited machine translations. The machine translation of the story met the requirements, but the official document machine translation accuracy did not. Finally, the cost is not justified. Assuming the same rate of US$0.67 for human translation stated in an earlier study [22], a professional might charge US$94.47 for the story and US$116.58 for the document, both well above the amount the students said they were willing to pay (US$61 and US$42).
Machine translation of the children's story is the best choice because the accuracy was sufficient, it was faster, and the cost was under budget, while the professional charge was not. For the official document, while the professional fee was over the budget, the machine translation accuracy was not sufficient. Using the model stated above and assumptions about the relative importance of speed, cost, and accuracy, we have: Thus, machine translation for the official document might also be the best choice. Table 3 shows a regression analysis using a dependent value of the translation cost and independent values of perceived text importance, time willing to wait, perceived accuracy of the result, and required accuracy. The model was significant (p = 0.034), and importance and accuracy seem to be the most influential factors, although the coefficients were not significant.

Conclusion
The decision whether or not to use machine translation involves several factors including costs, importance, speed, and accuracy. Obviously, if a translation is needed quickly or without cost, professional human translation is not an option. On the other hand, if near perfect accuracy is needed, there is ample time, and funds are available, this latter choice is probably best. In this research, we tested an economic model of when machine translation is viable. Results show that a multilinear regression model provides a significant fit using these variables.
There are several limitations to this research. First, the translation in the validation study was limited to Korean to English. Other language combinations might result in more or less accuracy. Second, only two randomly chosen text samples were taken to illustrate the decision process. Third, students only estimated how important information was or how much they would be willing to pay for a translation. The actual values of these variables might be very different. Finally, only a multilinear model was used. A neural network model would probably provide a better fit. These and other limitations must be investigated in future research.
• The meaning of residents is, people who have an address in the Metropolis of Seoul and who actually live there. • The meaning of visitors is, people who have addresses in a city other than in the Metropolis of Seoul stay in the Metropolis of Seoul at the time regardless of the purpose of the visit, such as a commute, business, or travel.