Archive of obsolete content

JavaScript Client API

Overview

This page describes how to use the internal client-side Sync JavaScript API. This API is available in Mozilla-based products that use Sync, such as Firefox desktop. This document is somewhat outdated, and the API isn't well-supported for use from add-ons; tread carefully.

The best, and most up-to-date, reference to Sync's internal APIs is the source code.

Please note that usage of the Sync APIs is governed by a terms of service:

By accessing or using the Firefox Sync APIs in connection with the development of your own client software to access the Firefox Sync services (a “Third Party Client”), you acknowledge that you will need to install and use a local version of the Firefox Sync server for multiple account testing and that any use of Mozilla’s hosted Firefox Sync services is subject to Mozilla’s Firefox Sync Terms of Service at https://services.mozilla.com/tos/.   Further, you agree (a) to maintain and link to (including on websites from which your Third Party Client may be downloaded) a separate, conspicuous, and reasonably detailed privacy policy detailing how data collected or transmitted by your Third Party Client is managed and protected; (b) that your Third Party Client will only store data in encrypted form on the Firefox Sync servers operated by Mozilla; (c) that you and your Third Party Client will use the Firefox Sync APIs solely for their intended purpose; (d) that your Third Party Client will not hide or mask its identity as it uses the Services and/or Firefox Sync APIs, including by failing to follow required identification conventions; and (e) that you and your Third Party Client will not use the Firefox Sync APIs for any application or service that replicates or attempts to replicate the Services or Firefox Sync experience unless such use is non-confusing (by non-confusing, we mean that people should always know with whom they are dealing and where the information or software they are downloading came from). You may not imply, either directly or by omission, that your Third Party Client is produced or endorsed by Mozilla. By providing access to the Firefox Sync APIs, Mozilla is not granting you a license to any of our trademarks.

Before Starting

Before you start learning the JavaScript API, you should spend some time on http://docs.services.mozilla.com/ reading about how the Sync service operates. This will help fundamental understanding significantly.

API Location

The Sync JavaScript client API is defined in files in the services/sync/ directory of mozilla-central or similar repository.

Terminology

SyncEngine
The main Sync client object. It's what starts all the magic.
Engine
Governs the synchronizing of a specific set of information. e.g. there is an engine for bookmarks, tabs, history, preferences, etc.
Record
Piece of information that gets synchronized. Records are transformed to WBO's, uploaded to a collection in a Sync server and eventually downloaded by other Sync clients.
Tracker
Entity that tracks changes to data. Each engine has a corresponding tracker which listens for changes to data or events it is interested in. When something of interest occurs, it interacts with the engine to let it know something has happened and it might want to take action
Store
Entity that serves as a data store/proxy for information in an engine. The name store is somewhat of a misnomer, as stores don't actually persistently store anything, but rather serve as short-lived data stores during the course of a single sync and act as proxies to other data stores within the application (like the places database) outside of Sync.

Creating a Custom Engine

The Sync client ships with a number of engines out of the box (history, bookmarks, etc). It is possible for other components, including 3rd party extensions, to supplement the set of engines and synchronize their own data.

A word of warning: the JavaScript Client API is still a young component and is continuing to evolve. Therefore, developers outside of the Sync core should know that their code could require significant refactoring in future releases. Before developing against the JavaScript API, it is recommended to speak to developers of the API. See https://wiki.mozilla.org/Services/Sync for their contact information.

To create a custom Sync engine to synchronize a new data you, you'll need to define an engine object that extends the base SyncEngine object. This involves writing types that extend the Store, Tracker, and Record types as well.

It will be very helpful to look at an existing sync engine for guidance. Have a look at one of the following files:

  • services/sync/modules/engines/bookmarks.js
  • services/sync/modules/engines/history.js

Setting up a JS Module

The code for your custom sync engine should live in a JS module. First off we're going to import the files mentioned above at the top the file. We're also going to import ''util.js'' as it contains many useful helpers.

const Cu = Components.utils;
Cu.import("resource://services-sync/engines.js");
Cu.import("resource://services-sync/record.js");
Cu.import("resource://services-sync/util.js");

The Record Object

The record object is a wrapper around a single instance of whatever data it is that you are syncing -- a single bookmark, history URL, password or form data entry. For some sync engines, it makes sense to bundle all data into one record. The tabs and preferences engines work that way. For other engines, it makes sense to have multiple records per sync. It all depends on the data model. If in doubt, consult the Sync developers.

The base record objects are defined in services/sync/modules/record.js. There's a base Record object, which defines the basic record API. But, you want to use a derived class, CryptoWrapper, which seamlessly encrypts and decrypts records on the client. This makes your synchronized data more secure and unreadable by the server operators. All you have to do is write an object that extends CryptoWrapper and maintains a property called cleartext. cleartext must be a JSON-able object. Put into it all values that you want to have encrypted, stored on the server, decrypted, and synced up.

You may find it useful to write getters and setters for various properties of your record implementation.

The skeleton of a sample record implementation:

function FooRecord(collection, id) {
  CryptoWrapper.call(this, collection, id);
}
FooRecord.prototype = {
  __proto__: CryptoWrapper.prototype,
  _logName: "Record.Foo",
  ttl: FOO_TTL,  // optional
  get bar() this.cleartext.bar,
  set bar(value) {
    this.cleartext.bar = value;
  },
  get baz() this.cleartext.baz,
  set baz(value) {
    this.cleartext.baz = value;
  }
};

To save all that typing for declaring the getters and setters, you can also use Utils.deferGetSet:

function FooRecord(collection, id) {
  CryptoWrapper.call(this, collection, id);
}
FooRecord.prototype = {
  __proto__: CryptoWrapper.prototype,
  _logName: "Record.Foo",
  ttl: FOO_TTL  // optional
};
Utils.deferGetSet(FooRec, "cleartext", ["bar", "baz"]);

The Store Object

The Store object (which extends Store, as defined in services/sync/modules/engines.js) has the job of creating and maintaining a set of Record objects from the underlying data.  The store must also make updates to the underlying data itself based on incoming Record objects. The majority of the code you write for syncing a new data type will most likely be in the Store implementation.

Each Record that the Store keeps track of must be identified by a unique ID (unique among all records in the store) called GUID. Depending of what type of data you're working with, you might already have GUIDs built into your data that you can make use of (note that GUIDs must be allowed to change and should be URL friendly).  If not, you may have to invent your own mapping from data objects to GUIDs as long as it's consistent.  In this case, it is highly recommended to use the Utils.makeGUID() helper to generate new GUIDs:

let newGuid = Utils.makeGUID();

Your Store object must implement the following methods:

  • itemExists(id)
  • createRecord(id, collection)
  • changeItemID(oldId, newId)
  • getAllIDs()
  • wipe()
  • create(record)
  • update(record)
  • remove(record)

You may also find it useful to override other methods of the base implementation, for example applyIncomingBatch if the underlying storage for your data supports batch operations.

createRecord
The createRecord( id, collection ) method gets called by the engine to request a new record for an item with a given GUID. It's your Store's responsibility to instantiate a Record object, assign the given GUID, and return it.
itemExists(guid)
Should simply return true if an item with the given guid exists in the store, false otherwise.
changeItemID(oldId, newId)
Must find the stored item currently associated with the guid oldId and change it to be associated with the guid newId.
getAllIDs()
Must return an iterable list containing the GUIDs of every item being stored on the local system. This can be a dictionary-type object where the keys are the GUIDs and the values are whatever; or it can simply be a list of GUIDs. This is the method that the engine will call the first time it syncs, in order to get a complete inventory of what data there is that will need to be uploaded to the server. Unless the items you're syncing have their own inherent GUIDs already defined, you'll need to invent GUIDs for all your items at this time. When one of these GUIDs is later passed as an argument to createRecord(), you need to return a record based on the matching data. Therefore, it's the responsibility of this method to define the guid -> item mapping, and to store it for later reference by other methods.
wipe()
When this method is called, your Store needs to completely wipe all of its stored data from the local application. That doesn't mean just whatever caches of the data the Store happens to be maintaining; it means the underlying data itself. For instance, BookmarkStore.wipe() deletes all bookmarks from the current Firefox profile. How to do this for your particular data type is up to you to figure out.

The next three methods are called by the sync algorithm when it determines that the state of the local application needs to be changed to keep it up-to-date with the user's remote activities. The sync algorithm will call your Store object with a record to be created, updated, or removed, and it is your Store object's responsibility to apply this change to the local application using whatever methods are neccessary.

create(record)
Not to be confused with createRecord(), this method (which should probably be renamed for clarity soon) tells your Store to create a new item in the local application, based on the data in record. (So for example, BookmarkStore.create() adds a new bookmark to the Firefox profile).
update(record)
The argument is a record which has been remotely modified; your Store should locate the matching local item (presumably using the GUID, which is available in record.id) and update it to the new values provided in record.
remove(record)
The argument is a record which has been remotely deleted; your Store should locate the matching local item and delete it. (Don't rely on the record that is passed in to this method having any of its attributes set correctly except for .id. The .id should be al you need, anyway.)
applyIncomingBatch :
TODO

Example Store Skeleton

// Maintains the store of all your Foo-type items and their GUIDs.
function FooStore(name) {
  Store.call(this, name);
}
FooStore.prototype = {
  __proto__: Store.prototype,
  itemExists: function(guid) {
    // Return true if an item with given guid exists in the store.
  },
  createRecord: function(id, collection) {
    record = new FooRecord(uri);
    // Look up the data corresponding to this guid, by the mapping
    // you've defined.
    // Set the data and the guid on the new record:
    record.bar = 17; // or whatever
    record.id = guid;
    // return the record
    return record;
  },
  changeItemID: function(oldId, newId) {
    // Find the item with guid = oldId and change its guid to newId.
  },
  getAllIDs: function() {
    // Return a list of the GUIDs of all items.  Invent GUIDs for any items
    // that don't have them already, and remember the mapping for later use.
  },
  wipe: function() {
    // Delete everything!
  },
  create: function(record) {
    // Create a new item based on the values in record
  },
  update: function(record) {
    // look up the stored record with id = record.id, then set
    // its values to those of new record
  },
  remove: function(record) {
    // look up the stored record with id = record.id, then delete it.
  }
};

Writing a Tracker Class

Your tracker class must inherit from Tracker, which is defined in services/sync/modules/trackers.js. Its purpose in life is to track changes to whatever data type you are syncing. It must maintain a list of GUIDs for objects that have been changed and therefore require syncing. It also has to maintain a "score", which is a number from 0 to 100 which represents "how badly does this data type need to be synced right now". Getting your tracker to track changes in the underlying data is probably easiest to do if you register your tracker as an observer, so that it can receive notifications from one or more Mozilla components. What events you listen for is entirely up to you. You can find out more about registering as an observer here: [link].

The Score

The '''score''' is stored in this.score, a variable defined by the base Tracker class. Set your score by assigning a value to this.score. The score number is used by the scheduler to decide when your engine will be synced. Setting it to zero means "I have no need to sync". The higher you set this number, the more urgently the scheduler treats your request. Setting it to 100 means "Sync me immediately". It's up to you to figure out the best way to assign a score based on the current state of whatever data type you're tracking. Your tracker score is automatically reset to zero after each time your engine syncs.

The changed GUID list

The base class maintains a list of GUIDs that need syncing. When your tracker detects that an item has changed, you should add it to this list by calling: this.addChangedID(guid); These GUIDs correspond to the .id fields of your Record objects; see the section on the Store class for more about defining and maintaining the mapping between GUIDs and Records.

Example

Here's the skeleton of a sample Tracker class.

function FooTracker(name) {
 Tracker.call(this, name);
 // Register yourself as event listener or observer for whatever
 // you want to track. Note that this may unnecessarily slow down
 // things for users who don't use Sync. It's a bit smarter to have
 // yourself notified when to start and stop tracking therefore:
 Svc.Obs.add("weave:engine:start-tracking", this);
 Svc.Obs.add("weave:engine:stop-tracking", this);
}
FooTracker.prototype = {
  __proto__: Tracker.prototype,
  _enabled: false,
  observe: function observe(subject, topic, data) {
    switch (topic) {
      case "weave:engine:start-tracking":
        if (!this._enabled) {
          // register event handler or observer here
          ...
          this._enabled = true;
        }
        break;
      case "weave:engine:stop-tracking":
        if (this._enabled) {
          // remove event handler or observer here
          ...
          this._enabled = false;
        }
        break;
   },
   onEvent: function onEvent() {
     /* Here is where you'd handle the event.  See the documentation for
        whatever service you are observing to find out what to call this
        method, what arguments to expect, and how to interpret them. */
     var guid = 0;
     /* Here is where you'd include code to figure out the GUID of the item
        that has changed... */
     this.addChangedID(guid);
     // Update the score as you see fit:
     this.score += 10;
   }
 };

Writing an Engine class

Your Engine class serves to tie together your Store, Tracker, and Record classes into a bundle that the Sync service can instantiate and use. You're probably sick of writing subclasses by this point, but don't worry: this one is very easy. I saved it for last because it requires the least code. Your class must derive from the SyncEngine class, defined in services-sync/modules/engines.js. SyncEngine contains a lot of code which handles logic for the core sync algorithm, but your subclass won't need to call any of this directly, unless you are overriding part of the sync algorithm to provide custom sync behavior (an advanced technique outside the scope of this article).

A sample Engine class:

function FooEngine() {
  Weave.SyncEngine.call(this, "Foo");
}
FooEngine.prototype = {
  __proto__: Weave.SyncEngine.prototype,
  _recordObj: FooRecord,
  _storeObj: FooStore,
  _trackerObj: FooTracker
};

As you can see, there isn't actually any new code here at all; the prototype simply defines some metadata such as the Store and Tracker classes to use, and the human-readable name that will be used in the log files to identify errors and status messages coming from this engine. (Don't forget that you'll need to import Weave. So the top of your file should include:)

const Cu = Components.utils; // etc...
Cu.import("resource://services-sync/main.js");

Installing your classes into Sync

You can register your engine in an "weave:service:ready" observer using the Weave.Engines.register(FooEngine) function. However, "weave:service:ready" is issued after the "weave:engine:start-tracking" observer notification. If your tracker observes "weave:engine:start-tracking", it's best to have the Sync service register your engine. Stick your engine class on to the Weave object:

Weave.FooEngine = FooEngine;

and add "Foo" to the services.sync.registerEngines preference (it's a comma separated list of engine names). You can also set up a component that registers a weave:service:ready observer and at that time, imports main.js as well as your engine. Also, for me, doing Weave.Engines.register(FooEngine) worked better then doing Weave.FooEngine = FooEngine. Reason for this is, Weave will iterate through the registerEngines preference and try to instantiate each engine it has there before you the line mentioned above ever has a chance to execute.This means the constructor to your engine is not ready when Weave tries to instantiate it.

Testing and Debugging your Engine

Set observers in the Tracker to tell when your datatype changes, after which the score needs to be set. You can also add api's to the engine, get a handle of the engine by going Weave.Engines.get("Foo") and call it directly.

Document Tags and Contributors

Tags: 
 Contributors to this page: rnewman, teoli, gps
 Last updated by: rnewman,