Application Translation with Mercurial

Draft
This page is not complete.

Check what is available for translation

1. Find out on which branch localization is done for your locale: Read your localization team's page by clicking on the team with your language code (e.g. "de" for German). The localization lead for your language knows the branch. The default localization branch is aurora.
2. Next check what is still untranslated by going to Localization status page and click on the page for your language.
3. In the section "Applications & Sign-offs", you will find different products and branches which are currently in translation.
   • gaia-* denotes branches of Firefox OS, the operating system for mobile devices.
   • Firefox and fx-* denote branches of Firefox for Desktop, the browser for desktops and notebooks running, Windows, Mac OS or Linux.
   • Fennec and fennec-* denote branches of Firefox for Mobile, the Firefox browser for the Android operating system.
   • Thunderbird and tb-* denote branches of Thunderbird, the mail client, newsgroup and feed reader and chat client. The localization of the Lightning calendar add-on also takes place on these branches.
   • SeaMonkey and sea-* denote branches of SeaMonkey, the suite consisting of browser, mail and chat client and more.
   • Lightning and cal-* denote branches of Lightning, the calendar add-on for Thunderbird. The localization of Lightning now happens on Thunderbird's tb-* branches, the cal-* branches have been retired. 
4. In this example, we assume Firefox for Desktop shall be localized and that the localization branch is aurora. Scroll to the fx_aurora text.
5. Read the text next to it.
   • If it is 'Translated', the branch has already been translated. Either translate a different product now or contact the person responsible for localization from the team's page (see above) so he can notify you by mail when the next strings for translation become available.
   • If it is a red text consisting of a number and the word 'missing', then there are untranslated texts and you can move on with this guide.
      

Required Tools

Build pre-requisites

First, get the required programs to compile Mozilla applications like Firefox and Thunderbird from Build Instructions. This is only necessary once.

File comparison program

For the translation, it is recommend to use file comparison program to compare the English file (containing the new texts) with the file of your locale (which is still missing the new strings). This example uses WinMerge: Download and install it. A comparison of alternatives (e.g. for Linux and Mac OS) is available on Wikipedia.

File and text editor

For editing individual files a text editor can be a better choice than a file comparion program, e.g. because the file is new and everything has to be translated, the file is too customized to compare it to the English one, or special functionality is required (e.g. bookmarking untranslated lines). Use a text editor you are comfortable with. This example uses Notepad++.

Progress tracking tool

To track which strings and files already have been translated and later also to see which new strings have already been tested in the product as being a good translation for the functionality behind the translated strings and that the translated texts fit into the available, the translation progress should be tracked. This can be done in a text editor or word processor or any other tool in which you can attach different kind of states to the individual texts to translate. The Firefox extension Scrapbook is used in this guide. Install it into your default Firefox profile (you will have more than one profile when following this guide).

Getting the current Texts in English and your Locale

Obtaining the English texts with the source code

Get the source code by downloading the following file:

• Firefox Desktop or Firefox for Android: Download the mozilla-<branch>.hg file (e.g. mozilla-aurora.hg) from https://ftp.mozilla.org/pub/mozilla.org/firefox/bundles/ The file is not small, but after initially downloading it, you only need to download the latest changes in the future.
• Thunderbird, Lightning or SeaMonkey: Download the comm-<branch>.hg file (e.g. comm-aurora.hg) from https://ftp.mozilla.org/pub/mozilla.org/thunderbird/bundles/

In the next step, we will unpack the source code:

1. Open an input shell, e.g. by pressing the Windows key and R on the keyboard or by calling the "Run..." command from the Windows "Start" button, and then typing cmd and pressing the Enter key.
2. Launch the installed MozillaBuild toolset by launching it with its path, e.g. C:\Mozilla\Coding\Buildtools\MozillaBuild\start-shell-msvc2010.bat You can autocomplete the file path after typing a few characters and then pressing the Tabulator key.
   If you launch the start-shell-l10n.bat, compiling features will be disabled. You can create translation patches, but creating language packs for the application, binary repacks of the application (changing the localization of the program), and compiling the application likely won't work.
3. You are now in the Mozilla development environment which emulates a Linux environment. Now go to the directory where you want to put the source code. It's highly recommended to put it onto the fastest hard drive in the device. File paths here work starting with a slash (/) followed by the drive letter, and then the directory structure like on Windows, but using slashes (/) instead of backslashes (\). So let's go to C:\Mozilla by typing
   cd /c/Mozilla/
4. Create a new directory for the source code:
   mkdir mozilla-aurora
5. Initialize the new directory as new repository:
   hg init mozilla-aurora
6. Go into the new directory:
   cd mozilla-aurora
7. Unpack the source code:
   hg unbundle /path/to/the/downloaded/mozilla-aurora.hg
   This can take some time, mostly depending on the performance of the hard drive.
8. Extract the files:
   hg update

Obtaining the texts of your localization

1. Go back to the parent folder of mozilla-aurora:
   cd ..
2. Find the localization repository for your branch and language on http://hg.mozilla.org/ E.g. for aurora and German (language code 'de'), it is http://hg.mozilla.org/releases/l10n/mozilla-aurora/de/ Copy that url to the clipboard.
3. Copy the repository to your computer by running the following command:
   hg clone http://hg.mozilla.org/releases/l10n/mozilla-aurora/de/ de-mozilla-aurora
   This should proceed pretty fast.
   On Windows, you can paste the url by calling the Paste command from the window menu in the upper left corner of the window.

Configuring your Mercurial user settings

Still in the command prompt, go to your home directory in the file system, e.g. for a Windows user, it looks like C:\Users\MyName , depending on your Windows version:

cd /c/Users/MyName

Now list all the files here:

ls -a

If there is no file called .hgrc , create it with

> .hgrc

This file stores your general Mercurial settings (Mercurial is the tool which manages the source code and its history of changes).

Now go in the file system to the file and open it in your favorite text editor. Windows users can use e.g. Notepad++.

Paste the following content into the file:

[ui]
username = Firstname Lastname <mynick@example.com>
merge = internal:merge
[alias]
qexport = export -g -r qtip
qexportall = diff -r qparent
[defaults]
commit = -v
diff = -p -U 8
qdiff = -p -U 8
qnew = -U
[diff]
git = 1
showfunc = 1
unified = 8
[extensions]
hgext.mq =
progress =
[hooks]

Put in your name and email address which later will be public on the internet after your translation patch got accepted and uploaded. Save the file and close it.

Enabling Mercurial queues (revertable patches)

Some time will pass between the moment in which you start the localization, finish it, test it, improve it, upload it for review and get review (maybe not approved on first try). Other localizers will likely have done changes to the translation, either adding new texts, removing obsolete ones or improving the current texts. This can cause conflicts between the changes you created and theirs. For this reason, updating patches kann be necessary so Mercurial is still able to find the lines in the files which it has to replace or where it has to add them. Mercurial queues allow to revert changes and also pull changes and reapply your local changes.

Go back to the localization directory:

cd /c/Mozilla/de-mozilla-aurora

Initiate Mercurial queues:

hg qinit

Get to know the translation style

For having good chances that the translation suggestions get accepted, they should match the translation style of the texts in the application already translated. In general, the labels in the applications should have the same style like the operating system, e.g. use the same order of subject, verb and objects; decline verbs or not like the operating system does.

The following resources can help figuring out the translation style and translate according to them:

• Check your localization team's page by clicking on the team with your language code (e.g. "de" for German) for a style guide (e.g. style guide for German).
• Transvision offers searching for texts and text ids ("entities" or "keys"), allowing to quickly find translations for already translated texts.
• Microsoft's terminology search allows fast lookups of localized technical terms.

Choosing what to translate

Go to the Localization status page and click on the '<number> missing' label of the Firefox branch on which you want to translate, e.g. fx_aurora. The page which opens has a bar holding information about how many texts (strings) have already been translated, how many there are in English and your locale and have the same text, and how many are missing in your local. The interesting part follows below as list of missing texts.

The first line shows the locale for which the texts are being translated (in this example: de - German). With the second line start the file paths of the files which require changes. The file paths follow the directory structure of the locale repository downloaded for translation. In this example, the file aboutDialog can be found in the folder browser/chrome/browser/. The text striked through are texts which have been removed in the English version and can also be removed from the locale being worked on. In this example we will translate the one new text with the id privatebrowsingpage.howToStart4 in the file aboutPrivateBrowsing.dtd.

Tips for finding files with usually easily translateable texts:

• Look for files in browser/chrome/browser/
• Look for a file with only a few changes
• Suggestions
  • Files starting with about...
  • Files in the preferences directory (browser/chrome/browser/preferences/)
  • The browser.dtd and browser.properties files if there aren't too many new strings.

Tracking the translation progress

If you translate more than a few strings, you usually want to keep track of what has already been translated (and later if the text has been tested in the product as fitting into the available area and suiting the functionality it describes). This guide achieves this with the Firefox extension Scrapbook.

1. You installed Scrapbook earlier, now open it in Firefox either by calling it from the Firefox menu bar (press the Alt key to show it if necessary) using "View" > "Sidebar" > "Scrapbook" or by clicking its toolbar button which you likely have to add by toolbar customization if you didn't do this before.
2. Drag the tab with the localization error showing missing and obsolete strings to the sidebar.
3. After Scrapbook has saved the web page to your local disk, click with the right mouse button on the page in the sidebar and choose "Properties" from the context menu which gets opened. Rename it something meaningful, e.g. "Firefox <version number> translation".
4. After you saved the dialog, click with the left mouse button on the page in the sidebar to open the locally saved copy.

Opening English files and the target locale side-by-side

The localization report shows the IDs of added or removed texts, but the English text itself is still unknown. We get this by comparing the English version of the file with the one of the locale we are translating in. To do so,

1. Launch WinMerge.
2. From its "File" menu, choose "Save project".
3. For the file path of the left directory, go to the downloaded mozilla-aurora source code and then into its browser/locales/en-US/ folder. In this example, the absolute file path is C:\Mozilla\mozilla-aurora\browser\locales\en-US.
4. Enable read-only mode for the left directory.
5. For the file path of the right directory, go to the downloaded translation directory de-mozilla-aurora and then into its browser folder. In this example, the absolute file path is C:\Mozilla\de-mozilla-aurora\browser.
6. Save this comparison as project so you can easily relaunch this comparision without selecting the directories again. A good name is e.g. translation-mozilla-aurora-browser.winmerge
7. Open the comparison you just saved.

The translation

1. Bring the console with Mercurial to the front.

2. Start tracking changes you make against the localization repository by telling it that with the qnew subcommand to save these changes as patch in a mercurial queue. This patch needs a name, choose something meaning full like "Firefox-<version number>-more-details". In this example, Firefox-29-aboutPrivateBrowsing will be used.
   hg qnew Firefox-29-aboutPrivateBrowsing

3. Switch back to WinMerge and navigate to browser/chrome/browser/ and open the aboutPrivateBrowsing.dtd file. You will see something like this:
   
   On the left, you see an overview of the comparison of the English and the non-English file. Gray parts indicate blocks of texts which can't be found in that file, dark yellow represents changed lines and light yellow the parts of text in it which is different.
4. Below the license header which should never changed, the texts are organized in the following format:
   <!ENTITY stringid "Text which will be shown in Firefox">
5. As you can see there is a bigger block of text not recognized. This is due to the texts which have in the English versionmoved from below the string with the id privatebrowsingpage.howToStart3 to a position above it. Cut and paste these strings in the localized file so their position corresponds to the on in the English file.
6. Press the F5 key or click the reload button in WinMerge's toolbar to re-compare the files.
7. Now copy the empty lines below privatebrowsingpage.learnMore and the line <!-- TO BE REMOVED POST-AUSTRALIS --> to the localized file. This text is a comment (which in .dtd files always can be found between <!-- and -->). There is no need to translate this text.
8. Now we copy the two missing lines from the English file and insert them in the localized file:
   <!-- LOCALIZATION NOTE (privatebrowsingpage.howToStart4): please leave &newPrivateWindow.label; intact in the translation -->
<!ENTITY privatebrowsingpage.howToStart4               "To start Private Browsing, you can also select &newPrivateWindow.label; from the menu.">
   The first line is a comment providing information on the localization of the text in the following line. &newPrivateWindow.label; has to be left unchanged so it can be replaced with the text of the string id &newPrivateWindow.label; whis is stored elsewhere.
9. Translation time: Translate the text. Afterwars, the line should look somehow like this:
   <!ENTITY privatebrowsingpage.howToStart4               "Der private Modus kann auch über den Menü-Eintrag &newPrivateWindow.label; gestartet werden.">
10. Save the file.

Exporting the changes as patch

A patch is a file containing recorded changes to a repository. It can be used to apply these changes to a copy of the repository yet without these changes.

1. Tell Mercurial to store the changes in the Mercurial queue with the qref command. We also create a commit message which will describe the changes in the patch and later get used when the patched gets checked in ('added') to the official repository with the localized files. For the commit message, we have to use the -m command line flag,
   hg qref -m "Firefox 29: Localize missing string in aboutPrivateBrowsing.dtd"
   You can change the commit message in the Mercurial queue every time by calling hg qref again.
2. Now export the patch to the file system using the qexport command which is defined in the .hgrc file (see above):
   hg qexport > ../firefox-29-aboutPrivateBrowsing-v1.patch
   This creates a patch file called firefox-29-aboutPrivateBrowsing-v1.patch containing the changes in the parent directory of de-mozilla-aurora (in this example: C:\Mozilla\).

Proof-reading and altering the patch

Now proof-read the patch. This detects often more than 90 percent of the possible reasons for the reviewer to not accept the patch. If you find that changes have to be done, edit the real aboutPrivateBrowsing.dtd file from the localization repository, save it and follow the steps from Exporting the changes as patch (see above).

Validating the translation in the product

This guide hasn't yet a section about the validation of the translation. Please follow the instructions from the QA phase section of the Localization quick start guide.

Submitting the patch for review

Now the patch has to be shared so the people currently trusted to change the official translation can review the suggested changes.

1. File a bug ("ticket") in the Mozilla Localization product and choose as component the localization for which you have translated.
2. Choose a meaningful bug summary which will be shown for searches etc. like in this example "Firefox 29: aboutPrivateBrowsing.dtd"
3. Fill any important and additional information into the description field.
4. Click the 'Add attachment' button.
5. Select the exported patch file with the file picker. The checkbox for 'patch' should get automatically checked.
6. Add a meaningful name for the attachment, e.g. aboutPrivateBrowsing.dtd, patch, v1
7. Go to the review 'Flags' category and set the one for review to ?, meaning that you request review for the patch.
8. In the text field next to it, add the mail address of the person leading the translation of the locale. This information can be found on the page of the localization team.
9. Submit the bug report.
10. Now you have to wait for the review of that person. This can happen on the same day, on the next weekend or later, but should happen in the timeframe given for the localization of aurora. If the person requests changes, follow the instructions under Proof-reading and altering the patch. If the reviewer accepts the changes, the patch will get uploaded and your translation will officially be added to Firefox.