An analysis conducted by The Spanish Group — a human-powered translation company — compared the official translations of some of this year’s Eurovision song lyrics to translations done by ChatGPT and Google Translate. The findings revealed that AI misinterpreted the original lyrics and created translations that changed the tone and overall meaning of the songs.
The Eurovision Song Contest took place in Malmö, Sweden, on May 7-11, 2024. Millions of viewers watched artists from 37 countries perform original songs in a variety of languages. Behind every melody and verse lie linguistic intricacies, where a wrong translation can give a whole different meaning to a song.
The Spanish Group’s analysis directly compared the native lyrics as supplied on the official Eurovision website, Eurovision’s official English translation of the native lyrics, and the native lyrics translated via ChatGPT and Google Translate. Songs with English lyrics that did not have an official translation were not included in the analysis.
Analysis Results
In Croatia’s entry Rim Tim Tagi Dim by Baby Lasagna, the officially translated lyric “I won’t go far and I’ve sold my pride” turns into “I don’t go far and sold my cow” when translated by ChatGPT and Google Translate. Similarly, Google Translate somehow came up with “the jail is very nice and very sophisticated” instead of the official translation “everything is very beautiful and very perfected.”
Spain’s controversial Eurovision song Zorra includes the steamy lyric “I’m just a vixen to you.” ChatGPT translated it as “I’m just a fox,” while Google Translate produced “I’m just a bitch.”
Estonia’s entry received an eye-catching translation error. Like the official translation, ChatGPT used the word “uniformed” to describe police officers carrying out a raid in a village, generating the lyrics “uniformed men visiting” (compared to the official “a visit from men in uniform”). Google Translate, on the other hand, produced “I’m fine with fit men in the village.”
Translations of Georgia’s entry Firefighter showcased multiple nuanced differences, including “I’m running like tigers,” which ChatGPT and Google Translate believe to be “I’m flying like a hawk.” This is alongside more obvious mistakes from the two AI tools, such as thinking that “dove” means the same thing as “firefighter.”
The table at the end of this article shows more discrepancies between the official and AI translations. The full dataset, including translations where there were no significant differences, is available upon request (please email contact@essentialcontent.co.uk).
AI Translation Quality Concerns
As CEO of The Spanish Group, I believe our analysis shows the pitfalls of relying on free translation tools. While AI-powered translation is really useful for informal, low-stakes tasks, it isn’t suitable for official or complex tasks. Free translation software currently lacks the accuracy and sense of nuance required to do important work.
The discrepancies found in our analysis of Eurovision song lyric translations underscore the limitations of AI tools. While they’ve made strides, they often miss the cultural context that human translators grasp effortlessly. Our analysis highlights the unique value humans bring to the translation process, preserving the intended meaning. In a globalized world, human translators remain essential for effective cross-cultural communication.
So, if you and your friends are having a Carpool Karaoke moment, you probably want to make sure that you’re singing from the same translation sheet.
Table of Differences
The following table shows differences between the official and AI translations.
| Country | Official translation vs. ChatGPT translation vs. Google Translate translation |
| Albania | “set a fire” vs. “light a fire” vs. “start a fire”
“I’ll be standing tall” vs. “I’ll stand tall” vs. “It will stand tall” “gon’ get up” vs. “won’t” vs. “won’t” “titan” vs. “masked titan” vs. “titan” “I’ll be brave” vs. “I’ll be awake” vs. “I will be smart” “Tonight I’ll rise” vs. “Tonight I will rise” vs. “Tonight I will get up” |
| Austria | “running through” vs. “flows in” vs. “runs in”
“You” vs. “Just” vs. “He” “I won’t let it show” vs. “Won’t let it show” vs. “I won’t let it be seen” “haunted” vs. “heart and” vs. “heart and” |
| Belgium | “All we chase” vs. “Everything we pursue” vs. “Everything we follow”
“kids” vs. “children” vs. “kids” “it’s gonna” vs. “it will” vs. “it will” “the living” vs. “the” vs. “no descriptor” |
| Croatia | “cow” vs. “pride” vs. “cow”
“dance” vs. “drawing” vs. “dancing” |
| Cyprus | “don’t align” vs. “are at stake” vs. “is on the line”
“guys like you” vs. “boys like you” vs. “guys like you” |
| Czech Republic | “Pathetic and I’m over it” vs. “I have wanted to tell you something” vs. “I’m tired of explaining anything to you”
“I’ll truly” vs. “And whether I’ll” vs. “And I will really” “I get violent” vs. “You are as cruel as it gets” vs. “You’re so fake, so fake” “Whether” vs. “And whether” vs. “And I will really” |
| Denmark | “Eating up” vs. “devours” vs. “eating away at”
“Fighting” vs. “struggling” vs. “fighting” |
| Estonia | “Here” vs. “visible” vs. “to be seen”
Variation in the types of mushrooms mentioned. |
| Finland | “Silence, I say” vs. “Quiet, I say” vs. “Shut up, I say”
“And the rule is no rules” vs. “And the rule is: There are no rules” |
| France | “sorry if I disturb you” vs. “sorry if I’m bothering you”
“it was crazy” vs. “it was insane” |
| Georgia | “Running” vs. “Flying” vs. “Rising”
“Firefighter” vs. “Dove” vs. “Phoenix” “Put out the fire” vs. “The fire ignites” vs. “We’re rising from these ashes” “Did we build empires, just to watch them burn?” vs. “We didn’t build this civilisation to destroy it” vs. “You know I’ll be there” “My heart’s beating louder” vs. “My heart is bursting with excitement” vs. “I’ll be there” “Running like tigers” vs. “Flying like a hawk” vs. “I see very clearly” “No fear in my eyes though” vs. “Without fear” vs. “(You know that) I’ll be there” “The ceiling is falling” vs. “The sky is breaking” “The windows are burning” vs. “The walls are burning” “It’s getting harder to breathe” vs. “Breathing is getting harder” “Can you hear me calling?” vs. “Do you hear how I ring?” “I see ashes falling” vs. “I see that the clouds are dispersing, forming” “I’m down on my knees” vs. “We’ve never been” “But I will save this love” vs. “We’re called to rise higher, higher, higher” “Come dance with the thunder” vs. “Fly away with the whirlwind” “Love makes us stronger” vs. “Because love makes us stronger” “From lovers to fighters, oh why do we do this?” vs. “(The sky is breaking)” “We’re meant to rise higher, and higher, and higher” vs. “(Breathing is getting harder)” “(I see it in the air)” vs. “(We’re running through these ashes)” vs. “(We’re rising from these ashes)” |
| Germany | “Running” vs. “Escaping”
“Average” vs. “Special” vs. “Blessed” vs. “Privilege” “Under” vs. “Going under” “No one gives a” vs. “Nobody cares about” “What’s soon to come” vs. “What will come soon” “Blessing” vs. “Blessed” vs. “Blessing” “Stronger” vs. “Getting stronger” “Game” vs. “Simply a game” “Can’t be won” vs. “Cannot be won” “Run” vs. “Running” vs. “On the run” vs. “Always on” “Close” vs. “Close but never done” “Done” vs. “Done, done, done, done” “Break out” vs. “Get out” vs. “Get out of my head” “Free” vs. “When I’m free” “Lost” vs. “Trapped” “Identity” vs. “My identity” “Head up” vs. “Hold my head up” “Keep pace” vs. “Keep up the pace” “Haunted” vs. “How I’m haunted” “Voices” vs. “Voices deep within” “Fade” vs. “Never fade away” vs. “Will never fade away” “Silence” vs. “Escape from the silence” “Screaming” vs. “Calling out” “Guidance” vs. “Guidance” “Fighting for” vs. “What am I fighting for?” “Sick and tired” vs. “I’ve had enough and I’m tired” vs. “Can’t do this anymore” vs. “I can not anymore” vs. “I can’t take it anymore” |
| Greece | “May pain not come to us” vs. “Pain hasn’t come to us, thankfully”
“I fall and move like dice” vs. “I fall and I lie down in the dawn” “I pretend to forget your name” vs. “I act like I forget your name” “And everything changes around me abruptly” vs. “And everything changes around me suddenly” “Where will the wind take us?” vs. “Where the wind will take us” “I fall and roll like dice” vs. “I fall and I lie down in the dawn” “I pretend to forget your scent” vs. “I act like I forget your scent” “And let whatever happens happen” vs. “Let it bring us whatever it wants afterwards” “And let whatever happens happen” vs. “Let it bring us whatever it wants” “And let whatever happens happen” vs. “And even if it brings us whatever it wants afterwards” “Even if I stay alone” vs. “Alone and if I stay” “I will always be waiting for you” vs. “Always waiting for you” “I’m trembling like a flame” vs. “I tremble like a leaf” “Like a lighted up match” vs. “Like a doomed spirit” “When the dawn comes” vs. “When it rejoices” “I pine away” vs. “It poisons me” “I die alone if you are not here” vs. “I only die if you’re someone else” |
| Italy | “sittin’” vs. “stay” vs. “stay”
“None” vs. “None” vs. “Nobody” “tall” vs. “high” vs. “head high” “Cause” vs. “Because” vs. “That” “It comes to me” vs. “It gets to me” vs. “It comes to me” |
| Lithuania | “going” vs. “go” vs. “going”
“dark, it” vs. “dark, it” vs. “dark and” “night” vs. “morning” vs. “hello” “Wait up, wait up” vs. “Wait, wait, wait” vs. “Shake, shake, shake” “standing in silence while the radio’s playing” vs. “stand in silence, with the radio playing” vs. “stand in the silence, the radio playing” “anymore” vs. “anymore” vs. “anymore, I don’t want to dance anymore” “have to dance” vs. “need to dance” vs. “but I need to dance” |
| Luxembourg | “act like a child” vs. “pretend that the party” vs. “have a child”
“Where are you going?” vs. “No, but where are you going?” vs. “No, but where are you going?” “straight towards defeat” vs. “Straight towards defeat” vs. “You don’t have to brake” “Lead battles” vs. “Fight battles” vs. “Lead battles” “Set out to conquer it” vs. “Go conquer it” vs. “Set out to conquer it” “And that’s it” vs. “And that’s it” vs. “And There you go” |
| Norway | “I was a very fine and beautiful maiden” vs. “I was once so fair and lovely a maiden” vs. “I was such a friend and subject for a while”
“With an evil stepmother. My mother had died.” vs. “With a stepmother cruel, my mother she died” vs. “With stepmother hurt, my mother died” “She transformed me into a sword and a needle” vs. “She transformed me into a swan and a needle” vs. “She turned me into a sword and a needle” “And sent me off to the King’s estate.” vs. “And sent me away from the king’s court” vs. “And send me off ten the king’s court” “The wrath of my stepmother grew” vs. “And my stepmother’s anger was most fierce” vs. “And my stepmother’s anger felt the most” “When all the fine people loved me most.” vs. “When all the fairest ones liked me best” vs. “When everyone they liked liked me the best” “She gave me the hide of a grey wolf” vs. “She gave me a home as a gray wolf” vs. “She gave me a wolf gray hair” “And forced me to go into the woods alone.” vs. “She swore me to wander alone in the forest” vs. “She swore me to walk alone in the forest” “And I would never be whole and good” vs. “And never will I be whole and good” vs. “And I will never be whole and well” “Before I had drunk of my brother’s blood.” vs. “Until I drink my brother’s blood” vs. “Before I drink my brother’s blood” |
| Portugal | “We keep going” vs. “We continue like this”
“Long strides, stuck in the starting line” vs. “Big steps, caught in the start” vs. “Long strides, caught in the match” I ask the morning star to make the day rise again” vs. “I ask the mother star to make the day Be born anew” vs. “I ask the mother star to make the day Born again” “I am whole” vs. “I am all the size” vs. “I’m every size” |
| San Marino | “But it’s kind of ironic” vs. “But it’s somewhat ironic” vs. “But it’s kind of ironic”
“Looking for my veins?” vs. “Looking for my veins?” vs. “Searching my veins?” “If you don’t love me” vs. “If you don’t want me” vs. “What if you don’t love me” |
| Serbia | “Neither peace, nor sleep” vs. “I don’t have peace, I don’t have sleep” vs. “I have no peace, I have no sleep”
“The night frightens the day” vs. “Night doesn’t let the day come” vs. “not let night come day” “The lonely ones suffer” vs. “Hard for those who are alone” vs. “woe to him who is alone” “Everything is quiet just like under the water” vs. “Everything is quiet under water” vs. “Everything is quiet under water” “I scream, but the sound doesn’t come out” vs. “I scream but it’s not heard” vs. “I scream but it is not heard” “There is a white flare behind the mountain” vs. “A white shine behind the mountains” vs. “behind the mountains a white glow” “This is the road for wounded” vs. “This road is for the wounded” vs. “this time is for the wounded” “Because all the stars have fallen asleep” vs. “because the stars are all asleep” vs. “because the stars are all asleep” “Prayers also don’t help” vs. “Prayers don’t help either” vs. “not even prayers help” “Where have lilac ramondas disappeared to?” vs. “Where have the lila ramondas disappeared” vs. “where the lilac ramondas disappeared” |
| Slovenia | “And went into the silent night” vs. “and walked into the quiet night” vs. “and left in a quiet night”
“Voiceless cries for help” vs. “silent cries for help” vs. “silent cries for help” “I seek peace” vs. “in the waves I seek peace” vs. “I seek peace in the waves” “I surrender to the light” vs. “surrender to the light” vs. “I surrender to the light” “Find me, wound me, protect me” vs. “Find me, wound me, defend me” vs. “Find me, wound me, defend me” “(Who do you chase when you leave me?)” vs. “(Who do you chase when you leave me?)” vs. “(Who are you chasing, who are you abandoning me?)” “Lift me up, let me go, love me” vs. “Lift me, leave me, love me” vs. “Pick me up, let me go, kiss me” “(Who hunts you when you fear me?)” vs. “(Who chases you when you fear me?)” vs. “(Who is hunting you, who are you afraid of me?)” “Whom do you fear” vs. “Who do you fear” vs. “who are you afraid of” “I am a frameless mirror” vs. “I am a mirror without edges” vs. “I saw a mirror without slaves” “Just a reflection of your fears” vs. “Only the reflection of your fears” vs. “Here are all your fears” “Veronika” vs. “Veronica” vs. “Veronica” “Only she knows your truth” vs. “Only she knows your truth” vs. “She knows your truth” |
| Spain | “I know I’m just a vixen to you” vs. “I know I’m just a fox” vs. “I know I’m just a bitch”
“Can’t be bothered changing for you” vs. “Changing for you is too much effort” vs. “Changing for you makes me lazy” “If I head out solo, I’m a bit of a vixen” vs. “If I go out alone, I’m the fox” vs. “If I go out alone, I’m the bitch” “If I’m having fun, I’m the biggest vixen” vs. “If I have fun, the biggest fox” vs. “If I have fun, the sluttiest” “If I stretch the night till it’s daylight” vs. “If I extend and it’s daylight” vs. “If I extend and it’s daylight” “I’m an even bigger vixen” vs. “I’m even more of a fox” vs. “I’m even more of a slut” “From vixen, I might just turn into a jackal” vs. “From a fox, I’ll turn into a jackal” vs. “From a fox I will become a jackal” “I’m a picture-perfect vixen” vs. “And now she’s a picture-perfect fox” vs. “And now she’s a postcard bitch” |
| Ukraine | “Pa-pa-path is winding and rocky” vs. “Pebbles, pebbles, pebbles crunching, rocky”
“But you should know: your own happiness is in your hands” vs. “but know this: in your hands lies your own happiness” “Come on, little girl, light up” vs. “Come on, little one, burn” “I’m not holy, I’m alive” vs. “I’m not holy, I’m alive” “Even when we were little, we searched for a path” vs. “Even the smallest of us sought a path” “On her shoulders, still a young girl” vs. “On her shoulders, still a young girl” “She carries a burden of pain, seeing it as a flaw” vs. “carrying a jug of pain, and she sees that it’s a flaw” “Again not as she should be, or too tender” vs. “Again, not so delicate, not too soft” “But the years are flying! You should be aware: the end is near” vs. “And the years fly by! And yet: the end is near” “May there be energy in you not the elbow-deep, but the finger-deep” vs. “let there be no spark in you, but a fire in your fingers” “But there are saints in the sky, their feet have seen this earth, you know” vs. “But in heaven there are saints, their feet touched this earth, you know” “Mother Teresa and Diva Maria are with us” vs. “With us Mama Teresa and Diva Maria” |
