Creating custom Firefox extensions with the Mozilla build system

There is a wealth of material on creating extensions for Firefox. All of these documents currently assume, however, that you are developing your extension using XUL and JavaScript only. For complex extensions, it may be necessary to create components in C++ that provide additional functionality. Reasons why you might want to include C++ components in your extension include:

• Need for high-performance beyond what can be delivered by JavaScript code.
• Use of third-party libraries written in C or C++.
• Use of Mozilla interfaces that are not exposed via XPCOM (e.g. NSPR).

This article describes how to set up the development environment for a large, complex Firefox extension with any or all of the above-mentioned requirements. I should also stress that you do not have to build Mozilla or use the Mozilla build system if you want to create C++ components for Mozilla. If you are just looking to create an XPCOM component or two, this is probably overkill, and you might want to take a look at this guide instead. On the other hand, if you are an experienced developer or team, and you know that you are going to build a large, complex extension, you would do well to consider the approach described in this article.

One final note: I’ve only tried these techniques inside Firefox, but they’ll probably work more or less unchanged on other Gecko-based platforms like Thunderbird or Seamonkey. If someone can confirm this and/or provide guidelines for what’s different, I’ll update the article to incorporate this information.

Bambi Meets Mozilla

None of this is for the faint of heart. In particular, the initial step involves building Mozilla, which is a huge - nay, gargantuan! - project. Many intelligent developers have been driven to the brink of insanity trying to build it for the first time. If you're not an experienced C++ developer, I wouldn’t even bother. Stick to JavaScript.

On Windows Platforms

The first time I built Mozilla I used this guide. I can’t even remember why anymore, but I got stuck in a number of places, and the whole affair ended up taking far longer than I originally expected. Much furniture was smashed, much hair torn out by the roots. Here’s a comprehensive looking guide that’s gotten good reviews. Follow every step methodically and you’ll probably be alright. Focus on the fact that once you get the build working, it’ll probably work effortlessly from then on. Maybe.

On Other Platforms

On other platforms, namely Linux and MacOS, the process is much easier. All the tools for building are available built-in, and therefore all you have to do is run some commands in the terminal. You can find full instructions for almost any OS here.

Structuring Your Project

Mozilla includes a number of complex extensions that are integrated into its build process. It has thus been necessary to solve all of the issues involved in creating and registering XPCOM components, building JAR files and manifests, installing the lot into the Firefox extensions/ directory and so forth. So it behooves us to piggyback on this infrastructure to build our extension.

First of all, think of a catchy name for your extension and create a directory with that name under the /mozilla/extensions/ directory. Use only lowercase letters. You should see a bunch of other directories (inspector/, reporter/ and so forth) at the same level in the build tree.

Note that before actually building anything, the Mozilla build system invokes a configuration process that generates the actual makefiles used for the build from makefile templates called Makefile.in. The actual makefiles tend to be very similar or identical to the templates, but the extra flexibility gained from having the makefiles generated dynamically is one of the things that makes the build system so powerful.

Anatomy of a Simple C++ Extension

We assume that you are using C++ to write XPCOM components that can be used either from other C++ components or from JavaScript. The process of creating a component is actually relatively straightforward when the Mozilla build system is used.

In the simplest case, a component will consist of a single main directory with two subdirectories, public/ and src/. The main directory and each subdirectory must contain a Makefile.in (from now on I’ll just refer to this file as a makefile although we know that it is actually used to generate the real makefile). This makefile says two things. First of all, it lists the subdirectories that make up the extension, so the build system knows where to look for additional makefiles. Secondly, it instructs the build system to create a new extension, rather than copying the components directly into Firefox’s binary directory. The main advantage of using an extension is that it is easy to package everything up and install it on another machine.

So here’s your basic, plain-vanilla top-level makefile (Makefile.in in the main extension directory):

DEPTH		= ../..
topsrcdir	= @top_srcdir@
srcdir		= @srcdir@
VPATH		= @srcdir@
include $(DEPTH)/config/autoconf.mk
MODULE = myextension
DIRS		= public src
XPI_NAME		= myextension
INSTALL_EXTENSION_ID	= myextension@mycompany.com
XPI_PKGNAME		= myextension
DIST_FILES = install.rdf
include $(topsrcdir)/config/rules.mk

A detailed description of the make process, describing the key features of this makefile, can be found here. MODULE and XPI_NAME are both set to the name of your extension; they should be repeated in all project makefiles so that all of the files land in the same place in the XPI staging area (see below). INSTALL_EXTENSION_ID is the unique ID of your extension. This can be a GUID, but the format shown above is prettier and, let’s face it, a lot easier to remember. You don’t have to provide an XPI_PKGNAME, but if you do an XPI file, suitable for distribution, is automatically created in the root of the XPI staging area (/mozilla/$(MOZ_OBJDIR)/dist/xpi-stage/).

Every extension must include an install.rdf file that tells Firefox how to install it. This file should be located in the main extension directory and look something like this:

<?xml version="1.0"?>
<RDF xmlns="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
     xmlns:em="http://www.mozilla.org/2004/em-rdf#">
  <Description about="urn:mozilla:install-manifest">
    <em:id>myextension@mycompany.com</em:id>
    <em:version>0.1</em:version>
    <em:targetApplication>
      <!-- Firefox -->
      <Description>
        <em:id>{ec8030f7-c20a-464f-9b0e-13a3a9e97384}</em:id>
        <em:minVersion>1.0+</em:minVersion>
        <em:maxVersion>1.0+</em:maxVersion>
      </Description>
    </em:targetApplication>
    <!-- front-end metadata -->
    <em:name>My First Extension</em:name>
    <em:description>Just an example.</em:description>
    <em:creator>allpeers.com</em:creator>
    <em:homepageURL>http://www.allpeers.com/blog/</em:homepageURL>
  </Description>
</RDF>

There's a detailed description of the format of the install.rdf file. Use the DIST_FILES variable in the makefile to tell make to copy the file into the extension directory and (optional) XPI file.

Public Interfaces

The public/ directory contains any interfaces that need to be accessed by other modules. These can be IDL files describing XPCOM interfaces, which are used to generate normal C++ header files for inclusion in your source files. They can also be normal C++ header files that are to be used directly by other modules. The easiest way to accomplish the latter is to use inline implementations for all methods so you don’t have any additional linking dependencies. Otherwise you will have to link statically to your module if you use these public headers in other modules. Personally I would discourage this practice (among other things, static linking means the same code gets loaded more than once into memory, and the code won’t be available from JavaScript or other non-C++ languages) and encourage the use of XPCOM wherever possible.

The makefile in the public/ directory should follow this model:

DEPTH		= ../../..
topsrcdir	= @top_srcdir@
srcdir		= @srcdir@
VPATH		= @srcdir@
include $(DEPTH)/config/autoconf.mk
MODULE		= myextension
XPIDL_MODULE	= myextension
XPI_NAME = myextension
EXPORTS = \
  myHeader.h \
  $(NULL)
XPIDLSRCS = \
  myIFirstComponent.idl \
  myISecondComponent.idl \
  $(NULL)
include $(topsrcdir)/config/rules.mk

XPIDL_MODULE is the name of the generated XPT file that contains type information about your IDL interfaces. If you have multiple modules, make absolutely sure that you use a different value for XPIDL_MODULE for each one. Otherwise the first module’s XPT file will be overwritten by the second and you’ll get NS_ERROR_XPC_BAD_IID errors when you try to access its IDL interfaces from your code. The files under EXPORTS are copied directly to the /mozilla/$(MOZ_OBJDIR)/dist/include/ directory and are thus accessible from other modules (the value of MOZ_OBJDIR is defined in /mozilla/.mozconfig). XPIDLSRCS are run through the IDL processor, and the generated C++ headers are copied into the same include directory. In addition, an XPT (type library) file is generated and placed in the components/ subdirectory of your extension.

Source Files

Now it’s time to create the makefile and source files in the src/ subdirectory. If you're implementing interfaces that you've described using IDL, the easiest way to do this is to leave the src/ directory empty and run make on the public/ directory only; this will be explained shortly.

Then open the generated header file for your interface from /mozilla/$(MOZ_OBJDIR)/dist/include/. It contains stubs for the component .H and .CPP files that you can copy and paste into your implementation files. All you have to do is fill in the implementation stubs in the C++ file and you’re good to go.

Here’s an example of the makefile you need to place into your src directory:

DEPTH		= ../../..
topsrcdir	= @top_srcdir@
srcdir		= @srcdir@
VPATH		= @srcdir@
include $(DEPTH)/config/autoconf.mk
IS_COMPONENT = 1
MODULE = myextension
LIBRARY_NAME =  myExtension
USE_STATIC_LIBS = 1
XPI_NAME = myextension
CPPSRCS	= \
  myFirstComponent.cpp \
  mySecondComponent.cpp \
  myExtension.cpp \
  $(NULL)
include $(topsrcdir)/config/rules.mk
EXTRA_DSO_LDOPTS += \
  $(MOZ_COMPONENT_LIBS) \
  $(NSPR_LIBS) \
  $(NULL)

In this example, the first two files contain the implementation of the extension’s two components. The final file, myExtension.cpp, contains the code necessary to register these components, as described in the next section.

Registering Your Components

This article explains how to register XPCOM components in Gecko 2.0 and later.

Building It

As mentioned above, you’ll probably want to build your extension immediately after creating your IDL files in order to generate the C++ stubs for your component implementations. I’m assuming that you’ve already built Firefox successfully. If not, return immediately to the beginning of this article and don’t come back til you have a functioning firefox.exe. Do not pass go. Do not collect $200.

Still here? Okay, now we have to modify your .mozconfig (in the /mozilla/ root directory) so that your extension is built along with Mozilla. Add the following line at the end of the file:

ac_add_options --enable-extensions=default,myextension

Now launch make from the Mozilla root:

make -f client.mk build

Even if you have an up-to-date Firefox build, you’ll have to wait a while for make to recurse over the entire Mozilla source tree looking for new stuff (on my machine, which is pretty fast, this takes a good 10-15 minutes). Eventually it will reach your extension and generate a bunch of stuff under /mozilla/$(MOZ_OBJDIR)/:

• Exported header files and generated header files (from IDL) in dist/include/
• Static libraries for your modules in dist/lib/ (in case other modules want to link statically to your stuff instead of using XPCOM).
• XPI file in dist/xpi-stage/myextension.xpi.
• Generated makefiles for your projects in extensions/myextension/ (remember, we’re under /mozilla/$(MOZ_OBJDIR)/.
• Everything else in dist/bin/extensions/myextension@mycompany.com/.

A lot of this stuff won’t get created on this first pass since make will gag when it doesn’t find the source files for your components. Don’t worry about this; all you need are the generated header files that contain the C++ implementation stubs. Go back and flesh out the C++ implementation of your components so that the build can complete next time. Remember that you should never, ever modify any of these generated files. Always modify the files used to generate them and rerun make. If you’re changing the generated files directly, you’re probably doing something wrong.

The process of walking the entire Mozilla tree takes a long time. If you already have a Mozilla build, you can avoid this by creating a makefile for your extension directly. Go to the root of your $(MOZ_OBJDIR) and (from a bash-compatible shell) enter:

../build/autoconf/make-makefile extensions/myextension

If your $(MOZ_OBJDIR) is located outside your $(TOPSRCDIR), you'll need to do:

$(TOPSRCDIR)/build/autoconf/make-makefile -t $(TOPSRCDIR) extensions/myextension

in order for the script to know where your source is (it'll use the extension path you gave it relative to the current dir to figure out where you want your makefiles to go).

This will generate the proper makefile for your extension. Whether you build the whole Mozilla tree or take this shortcut, you can build from now on by going to /mozilla/$(MOZ_OBJDIR)/extensions/myextension/ and typing "make" on the command line. It should build your component without bothering with the rest of Mozilla. If everything works out, you’ll see your XPI file in the XPI staging area. You’ll also see the "exploded" version of the XPI (i.e. the unzipped directory structure) underneath /mozilla/$(MOZ_OBJDIR)/dist/bin/extensions. (If something goes wrong, figure out what, fix it and then come back here and add it to this article.)

To make sure that the build really finished, launch Firefox and check that your extension is listed when you select Tools/Extensions. If you are using Firefox as your regular browser (and if you’re not, why not!?), you might be annoyed by the fact that you have to close regular Firefox before running your custom-built version. If so, try setting the MOZ_NO_REMOTE environment variable to "1" before running the development version of Firefox. You’ll also need to use a different profile for your development version:

firefox -P development

Where development is replaced with the name of the extra profile you’ve created. This will let you run both versions of Firefox simultaneously, saving you oodles of time over the course of the build/test cycle.

No Place Like Chrome

Yippee-ki-yay! Now you have an extension that does, well, absolutely nothing. It’s time to do something with those groovy components that you’ve implemented and registered. The simplest way to do this is to write some JavaScript and XUL code. At this point, it would be very helpful to have a bit of experience writing "regular" extensions (i.e. without using custom C++ components). If you’ve never done this, I strongly recommend that you think of a cool idea for something simple that you’ve always wanted to tweak in Firefox and write it. Just displaying a new menu item that opens a "Hello, World!" dialog box would be already be a great exercise to get warmed up with.

Assuming you know how to write XUL/JavaScript extensions, you’re aware that the most important stuff goes in the chrome/ directory of your extension. Well, the fact that you’re also using C++ components doesn’t change that one whit. So now you need to create the normal content/, locale/ and skin/ directories in which to place your chrome files. Personally I like placing these directly under the root directory of my module, but I don’t suppose it makes any difference if you prefer putting them under a chrome/ subdirectory or whatever. Let freedom reign!

Once you’ve written the necessary chrome files (for instance, an overlay that adds a menu item to instantiate and use one of your components), you need to package them up as part of your extension. This is accomplished through the use of a JAR Manifest. For our simple extension example, this file might look something like this:

myextension.jar:
%  content myextension %content/
%  locale myextension en-US %locale/en-US/
%  skin myextension classic/1.0 %skin/classic/
%  overlay chrome://browser/content/browser.xul chrome://myextension/content/MyExtensionOverlay.xul
	content/MyExtensionOverlay.js		(content/MyExtensionOverlay.js)
	content/MyExtensionOverlay.xul		(content/MyExtensionOverlay.xul)
	locale/en-US/MyExtension.dtd		(locale/en-US/MyExtension.dtd)
	locale/en-US/MyExtension.properties	(locale/en-US/MyExtension.properties)
	skin/classic/MyExtension.css		(skin/classic/MyExtension.css)

Place this code in a file called jar.mn in the root directory of your extension, making sure that the paths in parentheses point to actual files (when interpreted relative to the root directory). You also have to make one small change to the makefile in the same directory, adding the following line:

USE_EXTENSION_MANIFEST = 1

This tells make to create a single manifest file called chrome.manifest instead of creating separate manifests with goofy names for each package.

Now launch make again, and you should see a chrome subdirectory appear in your extension (/mozilla/$(MOZ_OBJDIR)/dist/bin/extensions/myextension@mycompany.com/). Note that the chrome directory contains a JAR (i.e. ZIP) file with all the chrome files listed in jar.mn as well as a complete directory structure mirroring that of the JAR file. The directory structure, however, is empty. Why? I don’t know. Don’t worry about this, the files in the JAR are the ones that are actually used.

Keeping it Complex

If you’re developing a really complex extension with lots of XPCOM components, you’ll probably want to divide your code up into smaller modules.

Kinda, Sorta Complex Extensions

For a moderately complex extension, it’s probably enough just to subdivide the code into a single level of modules. Let’s assume that you have a base/ module that defines a bunch of basic XPCOM components and an advanced/ module that defines some chrome as well as other components that use the basic components. Your complete directory structure will look something like this:

• myextension
  • base
    • public
    • src
  • advanced
    • content
    • locale
      • en-US
      • ...other locales...
    • public
    • skin
      • classic
      • ...other skins...
    • src

Other than that, nothing really changes. The makefiles in the base/ and advanced/ directories should look more or less like your original root makefile, remembering to change the DEPTH variable to account for the fact that they’ve moved a level further away from the Mozilla root. You also need to remove the DIST_FILES variable since that’s going to be in the top-level makefile. Every makefile that generates anything should define the XPI_NAME variable to make sure generated files go into your extension and not into the global components/ directory. In fact, just define this in every makefile to be safe. You can use the same MODULE in both base/ and advanced/ so that all the generated include files go into the same directory, but make sure that you don’t use the same XPIDL_MODULE in the two public/ directories or one of the component type libraries (i.e. XPT files) will overwrite the other one and all hell will break loose.

Each module must also have a different value for the LIBRARY_NAME variable. This is the name of the generated dynamic library, so if we call the libraries "myBase" and "myAdvanced", we’ll end up with myBase.dll and myAdvanced.dll (on Windows, at least). And each of these modules is going to have a separate C++ file for registering components. So there will be two files that look like myExtension.cpp in the original example, say Base.cpp and Advanced.cpp. Finally, each module will obviously have its own jar.mn, though they can reference the same JAR filename and package name if you want all the chrome files to be organized in a single JAR file and package. The only file that really stays put is install.rdf, which still exists once and only once in the extension root directory.

As for the top-level makefile, it will now look like this:

DEPTH		= ../..
topsrcdir	= @top_srcdir@
srcdir		= @srcdir@
VPATH		= @srcdir@
include $(DEPTH)/config/autoconf.mk
MODULE = myextension
DIRS		= base advanced
XPI_NAME               = myextension
INSTALL_EXTENSION_ID   = myextension@mycompany.com
XPI_PKGNAME		= myextension
DIST_FILES = install.rdf
include $(topsrcdir)/config/rules.mk

Seriously Complex Extensions

At some point, even a single module may grow to the point where you want to divide it further into submodules. The difference between having separate modules and having a single module with separate submodules is that the submodules all share the same file for registering components (the famous myExtension.cpp file), and when compiled they create a single dynamic library. The decision to split a module into submodules is all about code organization; it doesn’t really affect the final product at all.

To split a module into submodules, first create a subdirectory for each submodule. Then create an additional directory called build/. Each submodule will be configured to create a static library, and the build/ directory will pull these libraries together to create a single dynamic component library. Confused? Here’s an example, showing just the advanced/ subbranch of the myextension/ directory:

• advanced
  • build
  • intricate
    • public
    • src
  • multifarious
    • public
    • src
  • content
  • locale
    • en-US
    • ...other locales...
  • skin
    • classic
    • ...other skins...

As you can see, we’ve split advanced/ into two submodules: intricate/ and multifarious/, and we’ve added an additional build/ subdirectory. We’ve left the chrome directories directly under advanced/, since they aren’t tied to any specific submodule. This means that jar.mn will stay in the same place.

The intricate/ and multifarious/ makefiles will look a lot like the original advanced/ makefile, but we’ll need to tweak them a bit. As always, we have to adjust the DEPTH variable since the makefiles are deeper in the directory structure. And we should change the LIBRARY_NAME to indicate that we’re generating a static library for each submodule. By convention the "_s" suffix is used for this purpose. So let’s call them "myIntricate_s" and "myMultifarious_s". Finally, we define the variable FORCE_STATIC_LIB, resulting in a makefile that starts something like this:

DEPTH		= ../../../../..
topsrcdir	= @top_srcdir@
srcdir		= @srcdir@
VPATH		= @srcdir@
include $(DEPTH)/config/autoconf.mk
MODULE = myextension
LIBRARY_NAME = myIntricate_s
FORCE_STATIC_LIB = 1
USE_STATIC_LIBS = 1
XPI_NAME = myextension
...more stuff here...

The build makefile pulls together the static libraries generated by the submodules and creates a single (dynamic) component library:

DEPTH		= ../../../..
topsrcdir	= @top_srcdir@
srcdir		= @srcdir@
VPATH		= @srcdir@
include $(DEPTH)/config/autoconf.mk
IS_COMPONENT = 1
MODULE = myextension
LIBRARY_NAME = myAdvanced
USE_STATIC_LIBS = 1
XPI_NAME = myextension
DEFINES += XPCOM_GLUE
SHARED_LIBRARY_LIBS = \
		$(DIST)/lib/$(LIB_PREFIX)myIntricate_s.$(LIB_SUFFIX) \
		$(DIST)/lib/$(LIB_PREFIX)myMultifarious_s.$(LIB_SUFFIX) \
                $(DIST)/lib/$(LIB_PREFIX)xpcomglue_s.$(LIB_SUFFIX) \
                $(DIST)/lib/$(LIB_PREFIX)xul.$(LIB_SUFFIX) \
                $(DIST)/lib/$(LIB_PREFIX)nss3.$(LIB_SUFFIX) \
		$(NULL)
CPPSRCS		= \
		Advanced.cpp \
		$(NULL)
include $(topsrcdir)/config/rules.mk
LOCAL_INCLUDES += \
        -I$(srcdir)/../intricate/src \
        -I$(srcdir)/../multifarious/src \
        $(NULL)

The makefile in the advanced/ directory should list the intricate/, multifarious/ and build/ directories in its DIRS variable. Make sure that build/ comes last since it can’t create the component library until the other makefiles have completed.

Other Topics

Adding Data Files to Your Extensions

In some cases, you may wish to include additional files in your extension that don’t belong in the chrome/ subdirectory. Examples might be database files or XML schemas. This can be achieved by adding a custom step to your makefile that copies the files from the source tree into the extension’s target directory.

Copying Data Files Into Target Directory

Let’s say that you have some data files containing statistical information that you want to include in your extension and make available to your components. You’ve placed these files, which have the extension .TXT, into a stats/ subdirectory under your extension directory in the source tree. The following makefile rule can be used to copy these files into the final target directory of the extension:

export::
	if test ! -d $(FINAL_TARGET)/stats; then \
		$(NSINSTALL) -D $(FINAL_TARGET)/stats; \
	fi
	$(INSTALL) $(srcdir)/*.txt $(FINAL_TARGET)/stats

Accessing Data Files From Components

The trick to accessing your data files is to figure out where the home directory of your extension is. Rumor has it that at some future date, this will possible through the nsIExtensionManager interface or something similar. In the meantime, there is a simple and reliable hack that can be used to achieve this. In the implementation of any JavaScript XPCOM component, there is a special __LOCATION__ (two leading and two trailing underscores) symbol that points to the component’s implementation file. So you can write a simple component which deduces the root directory of your extensions by extrapolating from its location.

This article explains how to create an XPCOM component in JavaScript. You’ll need an IDL file for an interface that looks something like this:

interface myILocation : nsISupports
{
    readonly attribute nsIFile locationFile;
};

Place the IDL file in the public/ directory of your project or subproject. In the src/ directory, place the JavaScript file that implements the component. The component implementation will include the methods for retrieving the path or file for the extension’s home directory:

myLocation.prototype =
{
  QueryInterface: function(iid)
  {
    if (iid.equals(nsISupports))
      return this;
    if (iid.equals(myILocation))
      return this;
    Components.returnCode = Components.results.NS_ERROR_NO_INTERFACE;
    return null;
  },
  get locationFile()
  {
     return __LOCATION__.parent.parent;
  }
}

This assumes that the component resides in a subdirectory of the extension directory (by convention, this directory is called components/). The parent property of __LOCATION__ returns the components/, and the parent of this is the extension directory.

The last step is to modify the makefile of the source directory where you placed your JavaScript file so that it is copied into the appropriate location in the extension:

export::
	$(INSTALL) $(srcdir)/*.js $(FINAL_TARGET)/components

Now you can instantiate an instance of this component and use the locationFile property to get an nsIFile interface that points to your extension’s home directory.

Using Third-Party Libraries

For more sophisticated extensions, you may want to integrate third-party libraries that provide specialized functionality for database connectivity, image processing, networking and the like. If you want your extension to run on all Firefox platforms, you will need to have the source code for the library in question, so I assume that this is available.

The most convenient approach from the perspective of the development cycle is to create a Mozilla-style makefile for the library. This works well for libraries that have a straightforward make process without extensive configuration. A good example of this is the SQLite library included in the Mozilla build tree at db/sqlite. By adapting the makefile in this way, the library is created as part of the standard Mozilla build process, which eliminates additional build steps. The downside is that you will need to update the modified makefile any time a new version of the library is released.

For libraries that have complex configuration processes, use a non-standard compiler or have other special characteristics, it may be unfeasible to create a Mozilla-compliant makefile. In this case, I would recommend placing the entire library distribution inside the project or subproject that uses it. So if library acmelib is used inside the multifarious/ subproject in the above example, it would be placed as a subdirectory underneath that subproject (at the same level as public/ and src/).

Of course, this means that you will have to build acmelib manually on all platforms before launching the Mozilla build. But at least you can then refer to include files and import libraries from your component using relative paths.

Building for Multiple Platforms

TODO