In its latest development of language-generating AI, OpenAI has created GPT-3, its most massive language model to date. The AI represents some of the best and worst parts of language.
Speech-related AI is breaking new ground. OpenAI has developed a new AI software called Generative Pre-Trained Transformer 3, or more commonly GPT-3. The new language-generating AI represents the San Francisco-based AI laboratory latest development in its mission to steer the creation of intelligent machines.
Relying on a statistical model, the algorithm has better capacity to perform human-like speech patterns if exposed to more text. In order to train a large enough statistical language model, OpenAI sourced from the biggest set of text ever amassed, including a mixture of books, Wikipedia articles, and billions of pages of text from the internet.
GPT-3’s size is one of the factors that differentiate it from its predecessors. With that in mind, OpenAI provided the GPT-3 with 175 billion parameters — the weights of the connections between the network’s nodes, and a proxy for the model’s complexity —in relation to the GPT-2’s 1.5 billion parameters and GPT-1’s 117 million parameters.
Based on a graph published in The Economist, the GPT-3’s massive number of parameters makes distinguishing its AI-generated news articles from human-generated ones nearly equivalent to a guess at random. Furthermore, GPT-3 can even write poetry, such as this verse about Elon Musk:
“The sec said, ‘Musk,/your tweets are a blight./They really could cost you your job,/if you don’t stop/all this tweeting at night.’/…Then Musk cried, ‘Why?/The tweets I wrote are not mean,/I don’t use all-caps/and I’m sure that my tweets are clean.’/’But your tweets can move markets/and that’s why we’re sore./You may be a genius/and a billionaire,/but that doesn’t give you the right to be a bore!’”
This new development fulfills several tech predictions in recent years and signals promising advances in the field of AI language modeling, but not without flaws that have perpetually marred both AI imaging and AI text generation. Despite GPT-3’s grammatically fluent text, the statistical word-matching does not reflect understanding of the world.
Melanie Mitchell, a computer scientist at the Santa Fe Institute, said that the text generated by GPT-3 “doesn’t have any internal model of the world — or any world — and so it can’t do reasoning that requires such a model.”
The result has led it into similar pitfalls discovered in the GPT-1 and 2, namely the AI’s inability to distinguish language promoting racism, anti-Semitism, misogyny, homophobia, or any other oppressive language that it finds in its sources. OpenAI even added a filter to the GPT-2 to disguise the problem of mimicking bigotry by limiting the model’s ability to talk about sensitive subjects. However, the issue still poses a risk to GPT-3, which has already reproduced prejudiced text.
To work around the problem, OpenAI has added a filter to a newer model of the GPT-3, but the fix may just be a band-aid at this point. Still, with such rapid development of new language models, the GPT-3 will likely soon be replaced by a version with an even larger scale and maybe some power of discernment.
Jonathan Pyner is a poet, freelance writer, and translator. He has worked as an educator for nearly a decade in the US and Taiwan, and he recently completed a master’s of fine arts in creative writing.
Debates will be transcribed and translated by a new state-of-the-art MT system that keeps humans in the loop
Translated has been selected by the European Parliament to automatically transcribe and translate parliamentary multilingual debates in real-time, covering the 24 official languages used by the institution. The service will be provided by new software available both through fully-localized web and mobile applications, and live streaming APIs for third-party developers. It it purported to be the first human-in-the-loop speech machine translation (MT) system, and should leverage context and user feedback to adapt the output in less than one second.
The product will be developed in collaboration with two companies that have already worked with Translated in building products for professional translators: Fondazione Bruno Kessler (FBK), a world-leading research center in MT and automatic speech recognition (ASR); and PerVoice, an ASR world-leading provider. Within the next 12 months, the consortium will release a prototype to be tested by the European Parliament. This solution will be considered alongside solutions provided by two other groups, following rules put forth in “Live Speech to Text and Machine Translation Tool for 24 Languages.” The best-performing tool will be confirmed as the official one for the following two years.
The new product is not a simple concatenation of ASR and MT, but a new, fully-integrated system in which the MT algorithms are tolerant of ASR errors. This approach will not only help deliver more contextualized translations, but it will also open up the opportunity to improve the quality of the output while the plenary session is happening. This is possible thanks to the human correction feedback that the tool allows by both the end-users and a team of professional translators.
“For this project, we are bringing together ten years of research in machine translation and speech recognition,” says Simone Perone, Translated’s vice president of product management. Some of the new AI models that will be used have already been put to work successfully in products such as ModernMT (an MT that improves from corrections and adapts to the context), Matecat (a computer-assisted translation tool that makes post-editing easy), and Matesub (the first subtitling tool offering suggestions during the transcription, now in beta and due to be released in September 2020).
Investors received multiple alerts about a security breach at Appen.
Appen Limited (ASX:APX), a multilingual AI enablement and machine learning company headquartered in Sydney, issued a report on July 30th (Australian time) that malicious actors had hacked a third-party provider and stolen access to its systems. Appen is listed on the Australian stock market, ASX.
The third-party system was being used on a trial basis and Appen ceased using it immediately. The company believes that the attack was random and not targeted specifically to its repositories, since other companies were also victims of similar incidents. Even though it was determined that the hackers gained access to Appen’s Annotation Platform, which contained customer and crowd names, company names, email addresses, encrypted passwords, IP addresses, and historical login and log off times, and some phone numbers, the IT security team asserts that the incident is limited in nature and not material.
According to the release issued to the Australian Stock Exchange, Appen — which analyzes data for eight of the largest ten technology companies in the world — has not suffered any interruption to its operations and has reported the incident to legal authorities.
The statement affirms that customer AI training data, the core business of the company, is stored separately and there is no evidence that it was stolen. The unauthorized access was detected as soon as it occurred, and Appen took the necessary steps to secure its systems. Relevant clients were contacted and had their passwords and security tokens reset. A cyber forensics firm was hired to assist in the investigation.
According to Dow Jones, Appen shares were up 1.2% at AUD 36.11. After a previous filing announcement before the breach, shares had been up as much as 3%.
This week, Appen LTD, a global leader in machine learning and artificial intelligence (AI) data, reported a 6.9% increase in share value — the most in 11 weeks. Appen is of note to the localization industry primarily as a seller of training data that can be used in applications such as machine translation.
The Motley Fool reported last month that Appen has seen a steady increase in share prices since March. The news comes as a boon to the company’s shareholders, who saw a significant drop in prices in mid-February as countries began responding to COVID-19. In spite of this year’s challenges, the company is in the middle of a strong upswing.
Indeed, during one of the price peaks, the founder and the CEO of the company cashed in their shares for a combined $61 million. Since March, the price has increased 99% from its $17.14 low to a high of $34.17. Following a year of immense success in 2019, with a 47% increase in revenue to $536 million, Appen has proven its resiliency during the pandemic.
With over 1 million contractors in over 130 countries and 180 languages, the company studies human speech and interactions with each other and with AI to collect training data that teaches AI models and machine learning algorithms to (theoretically) make good decisions. Its worldwide crowd of contributors paired with its innovative data collection platform ensures premium localization of text, images, audio, video, and sensory content, which has built it a strong reputation in a variety of industries.
Appen claims to be the data industry’s “most advanced AI-assisted data annotation platform.” Working with tech companies such as Apple, Google, Microsoft, and Adobe, Appen sells data sets to assist with machine translation, proofing tools, automatic speech recognition, computer vision, semantic search, text-to-speech, virtual assistants and chatbots.
Marketing games for international venues, celebrating the Lunar New Year with Chinese characters on jerseys, or recruiting a global class of players, the National Basketball Association is no stranger to localizing its content. But deepening AI and machine learning technologies promise an approach to its global fan base like never before.
The NBA announced in April that it has made plans to enter a multi-year partnership with Microsoft to create a more personalized, localized experience for its international fan base. But while fans around the world already enjoy watching NBA broadcasts in 47 languages broadcast in 215 countries, the partnership promises fundamental innovations to modernize fan interaction using artificial intelligence and machine learning technology.
One of the ways the alliance will change NBA content is by creating a direct-to-consumer platform on Microsoft Azure, Microsoft’s cloud computing service, which will provide data analytics, computing, storage, and networking that they anticipate will allow them to personalize fan experience “through state-of-the-art machine learning, cognitive search and advanced data analytics solutions,” according to the report.
Analyzing metrics around fan behavior is nothing new for the NBA, but with such data at its fingertips, the NBA-fan relationship may flourish like never before. NBA senior vice-president of direct to consumer Chris Benyarko said about the benefits of machine learning AI curating fan experience, “Instead of the fan having to pick and choose and turn them on or off one by one, the platform is now starting to behave like a game producer, automatically selecting and presenting the game in a different way.”
Deb Cupp, Corporate Vice President of Enterprise and Commercial Industries at Microsoft, said, “The AI eventually learns that I like to learn about stats, so it’s going to start presenting me more information about stats as I go into the game… It’s this experience where instead of just watching a game, it actually has the opportunity to interact in a way that matters to me as that fan.”
With fans in all different time zones who experience the game and consume content in a variety of ways, the NBA will be able to gather a global array of data to bring in new fans and retain old ones. What that will mean for sports remains to be seen, but the machine learning will likely provide invaluable information on fan behavior worldwide and grant the NBA a chance to solidify its place as a premier global sport.
Jonathan Pyner is a poet, freelance writer, and translator. He has worked as an educator for nearly a decade in the US and Taiwan, and he recently completed a master’s of fine arts in creative writing.
Let’s look over the past year to get a sense of how much progress we have made, and what progress may lie before us. AI has become a norm in our lives and is now part of the vernacular more than ever. People have accepted that their lives interact multiple times a day with AI, and that such technology is becoming ubiquitous.
What does 2019 hold for us? Well, if we pay attention to the predictions of 16th-century seer Nostradamus, we should brace ourselves for flooding, wars and a strike by a meteor. Nothing all that cheerful but, assuming we survive all of that, what does 2019 hold for technology trends?
AI
We’ve seen an explosion in the use of AI in the delivery of neural machine translation during 2018; expect this to continue in 2019 and beyond. AI is the catch-all term to cover machine learning (ML) and deep learning. Machine learning is an over-arching term for the training of computers, using algorithms, to parse data, learn from it and make informed decisions based on accrued learning. Examples of ML in action is Netflix showing you what you might want to watch next. Or Amazon suggesting books you might want to buy.
Within the localization industry, the use of AI in the form of machine translation (MT) in several forms has significantly improved translation quality outputs, sped up translation of huge quantities of data and reduced the price of translation to make it economically viable.
AI refers to computer systems built to mimic human neural abilities and to perform tasks such as image recognition, parsing speech forms, discerning patterns from complex data sets and informing accurate decision-making. What’s more, AI can do these tasks faster, cheaper and more accurately than humans. Although AI has been around since the 1950s, it can be truly said that it has now come of age. This maturity has been propelled by the ever-increasing computational power now available in the cloud.
According to Forbes, five out of six people use AI technology each day. These services include such things as navigation apps, streaming services (such as Netflix), smartphone personal assistants, dating apps and even smart home devices (such as remote-activated home security systems). Additionally, AI is used in recommendation engines used by eCommerce sites to schedule trains, to predict maintenance cycles and for other mission-critical business tasks.
For the localization industry, AI will become a highly-integrated component of MT systems. The role of the human translator will continue evolving to that of an editor of MT texts, rather than translator of raw texts. In addition, pricing models will continue to move from the traditional price per word based on word volumes to pricing on a time-measured rate. MT will become an integral part of the standard workflow. The reality of real-time translation — driven by such technology as the internet of things (IOT) — will see project managers and editors managing workflows of projects required by customers who need a constant flow of updated information. MT will become part of the translation process just as much as other computer-aided translation tools did in the past. And, as ever, evolving technology will bring with it a desire for speedier and cost-effective solutions.
Machine learning
ML will continue to grow as a tool used by most localization departments as the requirement for the speedy translations of large datasets continues to be a driver in the industry.
ML is a subset of AI: with ML, computers are automated to learn to do something that they are not initially programmed to do. So, ML is an over-arching term for the training of computers to use smart algorithms to automate actions, to parse complex data and to learn patterns from this learning thus enabling the computer to make informed decisions based on this accrued knowledge. ML can be broadly broken down into two types of learning: supervised and non-supervised learning.
For supervised machine learning, the training data is pre-labelled and consists of an aligned input data set and desired output data set. For example, an input data set could be a translation memory. An ML algorithm analyses the training data and maps how to convert future inputs to match the learned, desired output data sets.
Unsupervised ML is like supervised ML; however, the input data sets are not pre-classified or labelled. The goal of unsupervised machine learning is to find hidden structures in the unlabelled data.
So how does this impact the localization industry? Well, suppose you want to build a translation system to translate from Zulu to French, without any Zulu-French training data. The solution is, you can combine both supervised and unsupervised approaches to achieve this. You can use an English-Zulu data set in combination with an English-French data set, and using unsupervised machine learning, the system can learn how to translate from Zulu into French.
This approach is commonly referred to as “zero-shot” machine learning — expect to hear more about this in 2019 for MT systems for long-tail languages.
Blockchain
While blockchain is most widely known as the technology behind cryptocurrencies, it offers security that is useful in many other ways.
In simple terms, blockchain can be described as data you can add to, but not take away from or change. These blocks of data can be “chained” together to create incredible secure data repositories. Not being able to change any previous blocks is what makes it so secure.
This enhanced security is why blockchain is used for cryptocurrencies. It is also why it will play a significant role in localization where it will used to protect information such as a client’s financial details, and to protect and preserve translation memories; especially in translation memories used in distributed translation workflow scenarios.
Edge computing
Cloud computing has now become mainstream: most of all global companies now rely on this centralized hosting structure for machine learning and powerful computational power. This cloud market is dominated by just a few gigantic companies such as Amazon, Microsoft, Google and IBM. However, now that we’ve been using cloud computing for some time, companies have realized that accessing all data from a central repository introduces a time-delay latency, which in turn slows down the delivery of services which can, in turn, increase costs. The “round trip” made by cloud-based data is seen by many of today’s companies as a hindrance to their business growth.
Technology stands still for no man, and so, for many, the cloud has reached its peak as a service for some technologies. The cloud will continue to be used to analyze and process huge swathes of data, but the advent of the IoT (connected security systems, electronic appliances, vending machines, automated lighting), where data processing needs to be high speed, if not real time, demands a different model. So the logical and necessary next move is to move this data processing to the edge. The edge simply means that data processing is moving from a far-away storage location to a geographical site closer to the data source. The advent of powerful computer chips that allows such processing to be done locally has expedited this move to the edge. Indeed, many of today’s cloud setups automatically look to place the processing of data at the optimum edge site for that data’s requirements.
So, edge computing solves the latency problem by simply moving the data processing closer to home. Closer to home means less time spent uploading and downloading data. Instead of the centralized storage model, which has hitherto driven AI, companies are moving their data into the “local community” to be processed. This move will undoubtedly make data access much faster and facilitate the growing demand for real-time computing.
How will this impact localization? Well, in 2019 we can expect to see the edge model used in domain-adapted MT systems, and distributed translation workflows that are designed to meet the increasing demand for data distribution in real-time.
Conclusion
We are on the verge of an explosion in the use of AI. The inevitable growth of AI will fundamentally re-shape how companies manage translation workflows; the very engine of their work process. Real-time translations will often become the norm.
I also predict that changes will happen at a human level; for example, the role of the translator will change from that of translator of raw text to that of editor of huge volumes of high-quality MT-produced text. This will be a beneficial change, allowing translators to increase their capacity and so increase their income. In 2019, we predict that the overall transformation effected by the advent of AI at all levels of the industry will bring with it an increased velocity of production, an improved efficiency in the delivery of translations, and a reduction in the cost of translating huge volumes of data.
Translated has been selected by the European Parliament to automatically transcribe and translate parliamentary multilingual debates in real-time, covering the 24 official languages used by the institution. The service will be provided by new software available both through fully-localized web and mobile applications, and live streaming APIs for third-party developers. It it purported to be the first human-in-the-loop speech machine translation (MT) system, and should leverage context and user feedback to adapt the output in less than one second.
The NBA 
