The MSc in Multilingual Computing and Localisation at the University of Limerick is the longest running post-graduate localization course in the world. Functioning in distancelearning mode, it attracts a wide range of practitioners from Ireland and abroad.

As part of the course, students undertake a six month research project and, because they are usually industry-based, these dissertations tend to address real-world issues. This year was no exception, with dissertation topics including an assessment of the potential of crowdsourcing quality control in localization; the impact of localization efforts on the popularity of mobile apps in the Arab market; and the evaluation of statistical machine translation as a component in a cross-language (Greek-English) information retrieval system. Several of the dissertations from this course focused specifically on Western European localization issues, three of which are outlined here.

Crowdsourced localization testing in games

Saúl Ruiz Calleja’s work studies community forum feedback as a means of identifying English-to-Spanish localization issues in World of Warcraft (WoW), a massively multiplayer online role-playing game (MMORPG). While addressing a predominantly Western European issue (Spanish localization), its player base of over 100 million people illustrates that the games industry is truly worldwide in scope. Likewise, crowdsourcing initiatives are rapidly entering corporate discourse worldwide, as a new and innovative way of tackling effort-intensive problems in all sectors of business.

User generated content and feedback is continuing to become an ever-greater component of the web. Forums, for example, allow users to connect instantly with people who share the same interests in order to upload, amend and review content and systems. What is true of the web in general is also true of computer games: there are many online community wikis or forums specific to online game communities and, in these, players write guides, present instructions and discuss many aspects of game play and problems that they encounter when playing. The discussions are often so extensive that developers read them to see how players are currently experiencing the game’s functionality and playability.

As with many other software products, video games have become multinational and require localization for many different locales. The effort required to assure localization quality is proportional to the markets reached and, for bigger games, can be extensive. In addition, larger games are often released simultaneously around the world (simship), a situation that requires localization to be present even in the initial steps of the development process. This can create some difficulty for localization efforts, as text may be localized before the game has been completed, meaning that the localizer has less complete knowledge of, or access to, the game context, resulting in lower-quality localization.

A partial solution is to fine-tune the localization by employing user feedback after product delivery, through community forums. This is a form of crowdsourcing: a new and innovative way of tackling problems by (implicitly) soliciting help from the wider internet community. Typically the “crowd” in this instance are gamers with a particular interest in the subject area that drives them to perform the task voluntarily, without formal remuneration. For instance, the motivation for such gamer communities may be to obtain a high-quality localized version of their favorite games. This can be considered a specific kind of crowdsourcing known as fan work.

Crowdsourcing in this manner has the advantage of testing the game when end-product context has been established. It also obtains feedback from users who have a deep knowledge of the game and its associated characters; not always the case when professional translators are employed in an outsourced localization process.

Ruiz Calleja’s thesis investigated the potential of this practice. It assessed if users would engage in this process and, if so, if they would identify significant, relevant errors that prompted remedial action by the development team.

For his study Ruiz Calleja targeted one localization forum. This forum had to have a relatively healthy activity rate, so a successful game was a prerequisite, as was a forum that localized the game for a large audience. WoW is an online game where players from around the world assume the roles of heroic fantasy characters and explore a virtual world full of mystery, magic and endless adventure. MMORPGs are games that allow thousands of gamers to play in the game’s evolving virtual world at the same time via the internet. In the case of WoW, there are 100 million lifetime players in over 240 countries. It has tens of thousands of pages and the in-game text is about six million words.

In 2011 the WoW Wikia estimated that there were 80 thousand players based in Spain, but, given the population of Spanish speakers in South and North America, the number of Spanish speaking players is probably much in excess of this number. Hence the WoW Spanish forum was considered an appropriate localization forum choice. The forum started in November 2010 and, up until June 17, 2015, contained 743 posts. It should be noted that individual posts could contain a number of error reports: for example, one post included 65 error reports. After manual inspection, a total of 1,067 individual error reports were counted: an average of 228 error reports per year.

All 743 posts were analyzed and a content analysis approach was adopted. In this case content analysis was directed at the errors identified. Specifically, the process required that each individual error report relating to localization or translation was transferred, numbered and recorded in a spreadsheet. The associated date and author were then recorded. Additional information included the severity of the problem, its likely cause (linked to a new release for example) and details were also stored on whether it had been discussed by players. The categories and types of errors recognized were identified and counted. Likewise, the quality of the feedback they provided was determined by its correctness (through manual inspection of the game) and by evaluating the developers’ responses to the issues raised.

The errors were reported by a small cohort of gamers (284) and most of these (184) only reported one error. However, 11 of the gamers reported 15 or more errors and three reported 50 or more errors, suggesting that some had become serial testers for the game. All the reported errors were non-redundant, a quality attribute that allows developers to inspect them without the anticipation of wasting their time. While most of the errors were minor, there was a small proportion that could be considered serious (approximately 3%). Regardless of their severity, a manual inspection of the game by a native Spanish speaker suggested that the vast majority of them (86%) were valid. This impression of high validity was reinforced by the developers’ rate of response: 83.8% of the errors raised on the forum were responded to. In addition, the vast majority of these errors were subsequently fixed. So overall, the impression is of non-redundant, high-validity localization-error feedback from the crowd, prompting change from the development team. The minor nature of the majority of these errors does suggest though that while they obviously hinder gamers’ enjoyment, they do not stop game-play.

Five error types dominated the data set, contributing 67% of all the error instances. These were, in decreasing order, mistranslation (17%), mistakes reflecting mistyped keyboard characters (16%), untranslated material (14%), linguistic agreement errors that typically referred to males as females and vice-versa (10%) and mismatches in naming, caused by incorrect usage of the glossary (10%).

Overall, the study suggests that, for this forum at least, users do have a relevant role in the localization of games, providing high quantity and high quality feedback. In addition, although this feedback has limitations, the findings suggest that companies can make use of it as an important quality review component in their evolution process.

Rapid eLearning and localization readiness

Maria Pilar Cabezón’s work assesses the degree to which tools that generate eLearning software are built with localization of the end-product in mind. The market for self-paced eLearning in Western Europe is growing at a rate of 5.8% per year and revenues are expected to reach nearly €7.5 billion in 2016. However, in order to achieve market saturation in Western Europe, a diverse set of locales must be accommodated. This dissertation concludes that, while there is work remaining in this area, localization is being considered and incorporated by developers of eLearning-generation systems.

eLearning can be defined as the process of learning through a collection of learning methods using digital technologies, which enable, distribute and enhance learning. It started to emerge in the 1990s as a more flexible and affordable approach to developing and delivering learning content that, with the help of modern technologies, could overcome some of the barriers and constraints that characterized traditional methods. For example, it allows learning at one’s own pace, at one’s own time and in one’s own location.

The development of eLearning materials has become rapid in order to satisfy the needs of a fast-changing, productivity-based world, and consequently, this dissertation investigated the extent to which rapid eLearning development tools support localization.

Due to its ubiquitous nature, eLearning has the potential to reach students all over the world and that, in itself, presents a wide range of challenges from a cultural perspective: when the goal is to reach a global audience, we need to ensure that the eLearning materials we create are localized in terms of linguistic adaptation and are culturally appropriate for the learners. In order to keep the number of steps in such a localization process low, and thereby minimize the costs incurred in a successful adaptation of eLearning products, it is increasingly important that tools used to generate eLearning materials provide functionality that supports localization efforts.

The objective of Pilar Cabezón’s dissertation was to investigate if these new, rapid eLearning development tools support the adaptation of the product with respect to cultural and linguistic variation, and why it is important that they do so. While there are many studies that investigate eLearning and localization in isolation, there aren’t many that focus on the support these emerging, eLearning development tools offer for localization. This is because the two industries tend to work in isolation from each other, where localization begins after development ends. This dissertation is an attempt to bring these two fields closer together.

Five rapid eLearning development tools widely used today were selected for analysis: Adobe Captivate (AC), Articulate Storyline (AS), Composica (C), iSpring Suite 7.1 (iS) and Lectora Inspire (LI). They were chosen from a wide range of possible tools based on several criteria:

•They needed to be “rapid” eLearning development tools

•They were commonly used by the industry

•They were comprehensive tools that could be used from beginning to end to compose interactive eLearning materials, as opposed to popular tools whose primary aim was to create just animations or videos

•They rank as the top tools in eLearning industry reports, user blogs and so on

•Each of them had to be supplied by a different company, giving a better sampling of the marketplace

They were tested against a specific set of localization features through a localization feature specification template designed by the author for the purpose (see the full list in Table 1). Full but trial versions of the individual tools were inspected against this template by the author. Likewise the documentation associated with the tools was also analyzed against this template, in order to double check the conclusions. After this analysis, the tool publishers were given an opportunity to comment on the findings, again in order to verify their accuracy.

The results of the investigation were positive, showing that localization is largely recognized by developers of eLearning development tools. On average, each of the development tools supported 8.2 of the 13 localization features specified (see Table 1). However, this average also shows that there is room for improvement, in particular with respect to allowing graphic localization, pseudo-code translation and simple word counts. When compared, the features available exceed those nonavailable for all of the tools. Composica and Storyline are the highest achievers, followed by Captivate and Lectora.

The features most widely supported were for right-to-left languages, multilingual spell-checking, voice recording, support for the Unicode standard and the ability to publish in different languages. Extraction and reimporting of translated text as well as the expansion of table cell sizes were supported in most tools, but slightly less commonly. In contrast, XLIFF support and the ability to build glossaries within the eLearning tool were supported only in the minority of tools analyzed.

In conclusion, the dissertation found that support for localization within rapid eLearning development tools is good, but there is room for improvement. Some of the features not present are commonly found in other types of tools, such as computer-aided translation tools. That is the case for translation glossaries or for performing pseudo-translations. However, as the world becomes more global, such tools will be used more widely and more users will be advocating for these additions.

Mirrored user interfaces in apps localized for the Arabic language

Dorota Pawlak’s dissertation focused on the effect a specific localization effort has on the popularity of mobile applications in the Arabic-speaking market — namely that of mirroring the user interface to reflect their right-to-left reading style. Arabic speakers make up an increasing proportion of the western European population, currently estimated at about five million, and thus represent a growing market sector for such applications. Determining how specific localization efforts impact the popularity of these applications is of prime concern for companies when assessing where to place their scarce localization resources.

There is little doubt that Arabic is one of the most widely spoken languages in the offline and online world. Yet, in this era of intensive localization, Arab culture tends to be pushed to the background due to insufficient knowledge, technical skills or limitations imposed by the Arabic script. As a result, localization of websites, software products and mobile applications into Arabic is often limited to translation of text and replacement of culturally inappropriate graphics. This approach is not always welcomed by Arabic users who, as relevant studies suggest, often defer to usage of the English version of digital products in order to have a better experience and understanding of all functions and features of a system.

During localization into Arabic, the user interface has to be mirrored, which involves changing the appearance of an application from being left-to-right to right-to-left. In this way, the text in the interface should be adjusted to the right-to-left reading order of Arabic script, but also, the interface elements should have a  right-to-left orientation. For example, in Windows, the exit function represented by the “x” should now be at the top left of the screen, rather than the top right. This makes the application more intuitive and usable to Arabic speakers. Errors in  right-to-left localization are easily noticeable and could be even more critical than other error types, as the incorrect position of menus or buttons influences navigation and strongly suggests that the application was not developed by a local publisher.

Pawlak’s dissertation examined the relationship between the popularity of mobile applications localized into Arabic and the quality of mirrored user interface. The reason for this focus, and particularly this  right-to-left orientation issue, is the observation that a rising demand for localization into Arabic has not corresponded to a growing number of high-quality products: mobile applications or websites are often translated and not enough focus is placed on the correct  right-to-left localization. Such products are very likely to confuse Arabic speaking users who might be forced to change their navigation patterns. Thus, this analysis will help to find out if mobile applications localized for the Arab market indeed contain a large number of errors in their  right-to-left localization, and to determine if the quality of their mirrored user interface influences the popularity of the localized application in Arab countries. Consequently, it assesses the more generic assumption: that high-quality localization helps to achieve higher sales figures and increases return on investment, in the specific context of Arabic-localized apps.

Thirty mobile applications localized into Arabic were examined in terms of quality of the mirrored user interface. These products were selected from the “ranking app” list delivered by one of the platforms for mobile application statistics, which shows data categorized by country, operating system and application category (www.appannie.com).

As the largest mobile penetration in the Arab countries exists in Saudi Arabia, the list of the most popular free applications was retrieved for this country. From this list, 15 highly popular and 15 lower ranked applications (as of June 2015) were analyzed as localized from English into Arabic.

Each application was manually tested for the  right-to-left localization issues documented in Table 2. In large applications 15 randomly chosen screens were tested and, in applications that had 15 or less screens, all screens were tested. Finally, the number and type of the recorded errors was compared to the application ranking to verify if applications that are popular in Saudi Arabia offer a high-quality user interface.

The results show an average of just over ten mirror errors per app. In total there were 366 errors, 211 of which were associated with the higher ranking apps and 155 associated with the lower ranking apps.

While the results did not show any relationship between popularity and the quality of the mirrored interface, the results did suggest a nonintuitive relationship between publisher size and the quality with which user interfaces have been localized for a bidirectional language. Popular mobile app publishers, which release a significant number of products on the market, tend to pay less attention to the correct right-to-left localization, releasing their applications with more such errors.

On the other hand, less popular publishers, or publishers that have developed a small number of applications (less than ten), offer products that are more adjusted to the expectations of Arabic speakers in terms of reading order and, in general, they contain fewer errors in the user interface. Moreover, publishers that localize a major part of their mobile applications into Arabic also offer a better quality of right-to-left user interface.

Based on the results and established relationships, it could be inferred that smaller publishers pay more attention to correct localization, possibly to compensate for the lack of popularity and brand exposure that the bigger publishers usually have. In contrast, larger publishers seem to thrive based on their popularity alone, rather than the localization efforts of their developers. 

Acknowledgements:

This work was partially supported with the financial support of the Science Foundation Ireland grant 13/RC/2094.