Automated testing is an increasingly significant and important part of MailNews development. It allows reviewers to quickly pick up regressions in patches and helps developers to think about different cases.
This page and its sub-pages describe (and link to) the available test mechanisms within MailNews, and provide supporting information for developers and testers.
Functional testing
xpcshell-tests and make check
These are core test harnesses to which MailNews adds its own tests. These tests are run against almost every changeset that gets committed to the Thunderbird and SeaMonkey code bases.
xpcshell-testsare run in JavaScript without any chrome present. This makes them especially useful for testing at the component / module level.xpcshell-testsare the quickest tests to run. MailNews provides additional facilities as described on the MailNews automated testing page (see below).- "
make check" is used to run compiled code tests. These tests can go beyond the limitations ofxpcshell-tests(for example, access c++ code) and other testing options that don't test at the right level.
Testing frameworks
The AsyncTestUtils Extended Framework supports:
- Asynchronous test operation: A reasonably convenient means of writing tests that involve asynchronous operations by using generators. In the long term we would like to lose this extra complexity in favor of the event-loop-spinning style of operation used by mozmill.
- Message synthesis: Create messages by specifying what properties the messages should have.
- Message injection: Inject synthesized messages via local or IMAP (using a fake server).
- Message manipulation: Mark messages as read, tag them, untag them, move them to other folders, trash them, etc.
- Enhanced logging: Supports generating rich JSON streams to disk or over the network for consumption by LogSploder or other tools.
Performance testing
Mail leak and bloat tests
These tests start up Thunderbird or SeaMonkey and record any leaks found, as well as the total memory requirement. Although currently based on simple tests, these are useful for determining if new leaks have been introduced.
For more information, see Leak and Bloat Tests.
MailNews facilities
MailNews provides some additional facilities to aid the testing of MailNews protocols.
- Fake servers: these are JavaScript-written servers that can be used to simulate IMAP, POP, SMTP and NNTP servers.