Last week, the European Parliament voted in favor of the E-Contentplus program, set to support the development of multilingual content for innovative, on-line services across the EU. The budget will be about €149m for the 2005 to 2008 period. Program focus will be on improving the accessibility and usability of geographical information, cultural content and educational material in a large multilingual/multicultural marketplace. The underlying agenda is to boost broadband take up in Europe by making content better networked, more easily localizable and hence more attractive. Anyone interested in joining in had better bookmark this site and track developments.

The current eContent program (with a budget of only € 100 million) has funded market-seeding projects for access to public sector information, innovating production in multilingual/cultural environments, and facilitating the digital content marketplace (rights management). You can find out about them here. In the field of multilingual production, they cover such topics as subtitling systems (eTitle ) and ontology building for sharing legal information (LOIS).

These projects act as test beds for technology solutions, with a group of organizations working together on a real-world response that will eventually convert into a commercial service. In other words, the European Commission provides you with development money to grab a potential business opportunity.

Since this is public money, there should be more pooling of knowledge derived from these projects. For example, most of the translation and localization problems are solved in sui generis ways. There might be more interest in such programs if terminology, translation memory and other resources developed in these projects had to meet reasonable exchange standards, for example, so that some of the assets lived beyond the project in a collective way.

It is also nearly impossible to get any serious (i.e. drilling down below the usually thin website content) information about

(a) how the project went (intangibles such as what was learnt and shareable, what proved particularly problematic, what interesting ideas the whole effort spun off) and

(b) what happens to the tangible results.

When projects (such as eTitle) are packing in translation automation technology, speech recognition technology and the like, it would be good to seriously raise the learning level for the whole community by auditing these projects and providing feedback to the sector stakeholders rather than just to the Commission.

One solution would be to demand that a project Wikipedia be created, summarizing technical knowledge about the project in a semi-canonical form. Another is to develop a (possibly collective) project blog to make the work more responsive to their respective latent communities. There is naturally no need to reveal everything in the kitchen cupboard. But avoiding duplication, pushing the technology envelope and leveraging the existing infrastructure strike me as being basic desiderata for such projects.

How much “official” translation was there and into how many languages in government departments and other power centers in the world before the League of Nations, the UN, SW and LW radio (the one-way global speech communication platform for the pre-digital generation) and finally the various modern institutions of global governance, made multilinguality a household word, as it were? Obviously it would have depended on your colonial spread.

During the 1800 to 1950 period, presumably France in its role as provider of Gallic clarté did little official translation into African languages (or Arabic), or any relevant Indian languages in its Chandernagore and Pondicherry outposts. One doubts that Germany did much in its short-lived Namibia, Cameroon and Togo colonies, nor the Brits in their African land grab. However, the administrative complexity of British Empire in a Brahmin-educated India raises more interesting questions about multilingual communication, and it would be good to have a thorough history of the linguistic relations between the colonizer (many of whose faithful Indian Civil Service henchman learnt local languages and legal systems to a high degree of proficiency, just as they had learnt the classics at school) and colonized (who have ended up learning various forms of local English in addition to their multiple languages).

Going further back to the Renaissance, diplomacy made use of Latin, French or other lingua franca, and nations usually tended as in the Spanish case in Latin America to extend the sway of their own language to the colonized (read Ivan Illich’s provocative Vernacular Values on all this). One imagines that the Vatican (or perhaps just the proselytizing Jesuits and other religious orders) did quite a bit of translating of religious texts and some official documents, even though Latin must have been a handy administrative esperanto for centuries Whether the Romans ever bothered to hire a 3rd century Visigoth, 2nd century Carthaginian or 0 CE Palestinian (presumably after Pontius Pilate had washed his hands, he wrote up his report in Latin) to localize Rome’s decrees would require research beyond the call of this posting.

What sparked this reflection was an item in the January 15th Times Literary Supplement (need to subscribe) by A.D. Harvey on the role of English writers such as John Buchan and Rudyard Kipling in the British ad hoc propaganda effort in the First World War once hostilities had broken out. I was struck by the amount of rapid translating going on to beam anti-Germany messages out to fence-sitters and neutrals. By June 1915, Wellington House (the propaganda admin center) had distributed “2.5 million books and pamphlets in 17 languages, as well as editions of the Bryce Report on German atrocities in 30 languages.” I calculate this report (written after the German invasion of Belgium in the autumn of 1914) to be about 30,000 words long, so it was only a three week job for a single translator, plus another three weeks, say, for typesetting and printing etc. But the chaps in charge – presumably unused to this sort of a multilingual workflow – were obviously capable of marshalling and managing translation resources (in Britain?) in war time.

Compared with the supposed language spread of old empire (and possibly the Bible translation agenda), 30 looks like a lot of languages for your average European administration to handle. Which were they? They would have at least included standard Western European, Scandinavian and Slavic languages (as well as Asian?) to reach that number. According to the same article, John (39 Steps) Buchan went to France and wrote a patriotic The Battle of the Somme: First Phase which was published in November 1916 and “quickly translated” into Danish, Dutch, Spanish and Swedish.

Any leads on “official” translation practice of this sort before say 1939 would be most welcome.

People delivering products and services in the area of translation, multilingual content management and the like need to be able to appreciate the real importance of the debate raging about the semantic web, which increasingly looks like a another nerds’ battleground. One problem is: how do we build the metadata and/or taxonomies – the bracketed tags – needed to help documents and services bind automatically and seamlessly into useful resources. Top down, through small technical committees who set standards or bottom-up through the vast distributed contributions of millions of users aided by simple aggregation techniques?

The solution chosen – perhaps the fudge of solutions – matters because language issues emerge at every step of the way. Will multilingual taxonomies need to be built, or will they emerge mushroom like from mega-scale group practices, prompting in turn the offer of further taxonomy aggregation services as a niche market? Should public money be thrown at large scale semantic web R&D projects that slide into obsolescence before they are completed, or should it be channeled into supporting focused language technologies for minority communities that tend to be crushed by the invisible hand of the market?

This comment is from Peter Norvig, Director of Search Quality at Google speaking at SDForum’s Semantic Technologies Seminar. He highlights the granularity of the language problems confronting web searching – it’s the data and whtehr it’s been spell-checked, not the metadata, stupid. Interestingly, he is mainly a bottom-up guy when it comes to solving the who does what issue of the semantic web:

Semantic technologies are good for essentially breaking up information into chunks. But essentially you get down to the part that’s in between the angle brackets. And one of our founders, Sergey Brin, was quoted as saying, “Putting angle brackets around things is not a technology by itself.” The problem is what goes into the angle brackets. You can say, “Well, my database has a person name field, and your database has a first name field and a last name field, and we’ll have a concatenation between them to match them up.” But it doesn’t always work that smoothly.

Here’s an example of a couple days’ worth of queries at Google for which we’ve spelling-corrected all to one canonical form. It’s one of our more popular queries, and there were something like 4,000 different spelling variations over the course of a week. Somebody’s got to do that kind of canonicalization. So the problem of understanding content hasn’t gone away; it’s just been forced down to smaller pieces between angle brackets. So there’s a problem of spelling correction; there’s a problem of transliteration from another alphabet such as Arabic into a Roman alphabet; there’s a problem of abbreviations, HP versus Hewlett Packard versus Hewlett-Packard, and so on. And there’s a problem with identical names: Michael Jordan the basketball player, the CEO, and the Berkeley professor.

Peter van Dijck has an interview with the editor in chief of the Dewey Decimal System about translating the DD taxonomy. It’s worth noting that of the 35 translated versions of the DD translations, many strategic languages (Arabic, Chinese, Spanish, French, German etc) are either in progress or are very recent. This experience would provide an interesting case study in translating and managing multilingual taxonomies for some future semantic web, even though DD is a method for classifying books, not concepts.