Thunderbird MozMill Testing

MozMill is the framework that we're using to run interactive tests.  Mozmill tests effectively run the entire application through unit tests.

Running Mozmill Tests

Running a test suite:

Run this from objdir/ to run all the tests in the test suite.

make mozmill

For builds on Windows (created by mozilla/mach and implicitly build/pymake/make.py):

mozmake mozmill

Obsolete: In the past, the following command had been used for builds on Windows (created by mozilla/mach and implicitly build/pymake/make.py):

../build/pymake/make.py mozmill

Running a specific test:

For example to run all the folder-display tests, go to the objdir and run:

make SOLO_TEST=folder-display mozmill-one

For builds on Windows (created by mozilla/mach and implicitly build/pymake/make.py):

../build/pymake/make.py SOLO_TEST=folder-display mozmill-one

SOLO_TEST may be a directory containing tests within mail/test/mozmill, or a specific test file.

To run just a single test ("all the tests in a single file"), like test-message-window.js, again go to the objdir and run:

make SOLO_TEST=folder-display/test-message-window.js mozmill-one 

For builds on Windows (created by mozilla/mach and implicitly build/pymake/make.py):

../build/pymake/make.py SOLO_TEST=folder-display/test-message-window.js mozmill-one

Running MozMill Tests with pre-compiled Thunderbird ('packaged tests')

If you don't have a self-compiled build of Thunderbird, then you can run tests against pre-compiled Thunderbird builds. This is known as running in the 'packaged tests' style.

See this page for how to run in package test style.

Linux & VNC Server

Linux runs have a lot of issues with focus related problems in more recents of gecko.  Also, it can be annoying to lose your session to the tests.  With the landing of bug 548172 if you have Xvnc installed and exposed via /usr/bin/vncserver and a password defined in ~/.vnc/passwd, we will use it to spin up a VNC session and run the mozmill tests in that session.  On Fedora Core 12, this is provided by the tigervnc-server package.

There are two things you need to do / be aware of:

1. You must create a password at ~/.vnc/passwd for this to work.  This is accomplished by invoking the "vncpasswd" script.
2. The default xstartup script (~/.vnc/xstartup), at least on my system, uses the default X startup junk from /etc.  That may start up more than you really want spun up in the testing session.  For example, when I connected with vncviewer I noticed that my gnome-panel was in there running a system monitor.  I personally have whacked my xstartup file to lose the /etc stuff and just invoke xfwm4 (the xfce window manager which I specially installed):

#!/bin/sh
vncconfig -iconic &
unset SESSION_MANAGER
unset DBUS_SESSION_BUS_ADDRESS
OS=`uname -s`
xsetroot -solid grey
xterm -geometry 80x24+10+10 -ls -title "$VNCDESKTOP Desktop" &
xfwm4 &

You can connect to the vnc server to see what it is doing by invoking "vncviewer :99".  (We always run mozmill tests on display 99, currently.)  You will need to enter the password.

Developing Tests

The Thunderbird mozmill tests and infrastructure are stored in http://mxr.mozilla.org/comm-central/source/mail/test/mozmill.

Each set of tests is stored in a sub-directory, the tests that are run on a full mozmill run are listed in http://mxr.mozilla.org/comm-central/source/mail/test/mozmill/mozmilltests.list.

MozMill has its own documentation that is useful to examine to get used to working with MozMill. However, Thunderbird has its own set of APIs/styles that it uses to interact with mozmill, and these are contained in the http://mxr.mozilla.org/comm-central/source/mail/test/mozmill/shared-modules/ directory. This code is shared across the different test sub-directories being included in the tests where required, documentation for these functions is contained within the code in doxygen format.

These files give examples of MozMill tests in Thunderbird and are considered relatively simple examples to get started with:

• http://mxr.mozilla.org/comm-central/source/mail/test/mozmill/junk-commands/test-junk-commands.js
• http://mxr.mozilla.org/comm-central/source/mail/test/mozmill/content-policy/test-js-content-policy.js
• http://mxr.mozilla.org/comm-central/source/mail/test/mozmill/search-window/test-right-click-to-open-search-window.js

Need info on:

• Support modules
• Profiles
• Running MozMill in a normal profile
• ...

Installing MozMill

Starting Thunderbird 9, MozMill no longer needs to be installed separately. It is included in the sources and will be installed into a virtual environment during the build process. So you can proceed with Running Mozmill Tests.

Thunderbird's MozMill test harness currently requires using MozMill 1.4.2b1; more recent versions will not work at this time.  If you need to use a more recent version of mozmill for other software development, you will want to look into virtualenv.

Windows

You'll need MozillaBuild 1.4 or higher for this. Start a MozillaBuild prompt and use it to install MozMill 1.4.2b1:

 easy_install http://pypi.rhaptos.org/mozrunner/mozrunner-2.5.2.tar.gz
 easy_install http://pypi.rhaptos.org/jsbridge/jsbridge-2.3.7b1.tar.gz
 easy_install http://pypi.rhaptos.org/mozmill/mozmill-1.4.2b1.tar.gz

Mac/Linux

You'll need root privileges for this.

 sudo easy_install http://pypi.rhaptos.org/mozrunner/mozrunner-2.5.2.tar.gz
 sudo easy_install http://pypi.rhaptos.org/jsbridge/jsbridge-2.3.7b1.tar.gz
 sudo easy_install http://pypi.rhaptos.org/mozmill/mozmill-1.4.2b1.tar.gz

If you, for some reason, need a different version, see MozMill Development Install.