Addon tests

In addition to using Mozmill to test the application itself, it is also possible to test any kind of add-on. That said we would really appreciate the efforts from add-on authors to hop on to create tests for their add-ons. Once we have tests available we will run those on a daily basis with all versions of Firefox and on any major platform. For detailed information please check our Add-on Test-run project page.

Creating a test

Creating a test for an add-on is not much different from creating Mozmill tests for Firefox itself. Check our Mozmill Tests tutorial to get in touch with Mozmill.

Using the repository

You can store all the tests locally on your hard drive or you can also make them publicly available for everyone. If you don't have any problems with our MPL license, your tests can be also included in our mozmill-test repository.

In order to get tests checked in, make sure that the following conditions are fulfilled:

• Create a patch for all the tests which should be checked-in. See the coding guidelines and how the review process works.
• Your tests should be located under a sub-folder that uniquely identifies your extension. It is recommended that you use the extension id inside the addons folder unless this is the GUID as these make it difficult to identify addons. Further use sub folders like proposed below to store shared modules, and separate between normal tests and restart tests. See the tests for the Selenium IDE extension as an example.

mozmill-tests
|-tests
  |-addons
    |-myextension@example.org
      |-restartTests
      |-lib
      |-tests 

• All tests have to pass when they get run with Mozmill in the appropriate Firefox release. Please also check our documentation how to work with branches.
• Place an addon.ini file into your extensions root folder which specifies the download location of the extension for the specific branch. This information is necessary to get the automation running in our add-ons test-run script.

[download]
linux=URL
mac=URL
win=URL

If questions remain don't hesitate to contact Henrik Skupin.

Running a test

Once the test file has been created and a couple of tests have been added, this test can be run with Mozmill inside the desired application. We offer different ways to start a test-run but always use the add-on test-run script to make sure it works with our automated test-run.

Using Mozmill itself to run a test

First, of course, the extension has to be installed. When using the Mozmill add-on,it would have to be done manually. But with the command line client everything can be automated:

• Download the version of the extension you want to test to the local drive, e.g. to c:\mozmill\foobar.xpi.
• Run the test with Mozmill and supply the extension with the -a (--addons) option:

$ mozmill --binary="c:\firefox 3.5\firefox.exe" --test=tests/addons/testExtension@example.org/tests/example.js --addons="c:\mozmill\foobar.xpi"

Using the automated add-on test-run script

To be able to run the add-ons test-run script some conditions have to be fulfilled. Make sure you have Mozmill installed as command line client and also install the Mercurial Python module via pip or easy_install. Afterward clone our mozmill-automation repository to your local disk. Then you will be able to start the test-run with the following command:

$ ./testrun_addons.py --with-untrusted /c/Program\ Files/Mozilla\ Firefox/firefox.exe

If you only want to run tests against a specific set of add-ons, simply specify those addons as separate --addons options:

$ mozmill --addons=testExtension@example.org /c/Program\ Files/Mozilla\ Firefox/firefox.exe