Chapter 21 MySQL NDB Cluster 7.5

The NDB Cluster NDB storage engine is implemented using a distributed, shared-nothing architecture, which causes it to behave differently from InnoDB in a number of ways. For those unaccustomed to working with NDB, unexpected behaviors can arise due to its distributed nature with regard to transactions, foreign keys, table limits, and other characteristics. These are shown in the following table:

NDB Cluster has a range of unique attributes that make it ideal to serve applications requiring high availability, fast failover, high throughput, and low latency. Due to its distributed architecture and multi-node implementation, NDB Cluster also has specific constraints that may keep some workloads from performing well. A number of major differences in behavior between the NDB and InnoDB storage engines with regard to some common types of database-driven application workloads are shown in the following table::

When comparing application feature requirements to the capabilities of InnoDB with NDB, some are clearly more compatible with one storage engine than the other.

The following table lists supported application features according to the storage engine to which each feature is typically better suited.

Some SQL statements relating to certain MySQL features produce errors when used with NDB tables, as described in the following list:

In this section, we list limits found in NDB Cluster that either differ from limits found in, or that are not found in, standard MySQL.

Memory usage and recovery.  Memory consumed when data is inserted into an NDB table is not automatically recovered when deleted, as it is with other storage engines. Instead, the following rules hold true:

A number of limitations exist in NDB Cluster with regard to the handling of transactions. These include the following:

Starting, stopping, or restarting a node may give rise to temporary errors causing some transactions to fail. These include the following cases:

In either of these cases, any errors that are generated must be handled within the application. This should be done by retrying the transaction.

See also Section 21.1.6.2, “Limits and Differences of NDB Cluster from Standard MySQL Limits”.

Some database objects such as tables and indexes have different limitations when using the NDBCLUSTER storage engine:

A number of features supported by other storage engines are not supported for NDB tables. Trying to use any of these features in NDB Cluster does not cause errors in or of itself; however, errors may occur in applications that expects the features to be supported or enforced. Statements referencing such features, even if effectively ignored by NDB, must be syntactically and otherwise valid.

The following performance issues are specific to or especially pronounced in NDB Cluster :

The following are limitations specific to the NDB storage engine:

See also Section 21.1.6.10, “Limitations Relating to Multiple NDB Cluster Nodes”.

Disk Data object maximums and minimums.  Disk data objects are subject to the following maximums and minimums:

In addition, when working with NDB Disk Data tables, you should be aware of the following issues regarding data files and extents:

Disk Data tables and diskless mode.  Use of Disk Data tables is not supported when running the cluster in diskless mode.

Multiple SQL nodes.  The following are issues relating to the use of multiple MySQL servers as NDB Cluster SQL nodes, and are specific to the NDBCLUSTER storage engine:

Multiple management nodes.  When using multiple management servers:

Multiple network addresses.  Multiple network addresses per data node are not supported. Use of these is liable to cause problems: In the event of a data node failure, an SQL node waits for confirmation that the data node went down but never receives it because another route to that data node remains open. This can effectively make the cluster inoperable.

This section provides information on supported operating platforms and software, required software, and other prerequisites for running the NDB Cluster Auto-Installer.

Supported platforms.  The NDB Cluster Auto-Installer is available with most NDB 7.5.2 and later NDB Cluster distributions for recent versions of Linux, Windows, Solaris, and MacOS X. For more detailed information about platform support for NDB Cluster and the NDB Cluster Auto-Installer, see http://www.mysql.com/support/supportedplatforms/cluster.html.

The NDB Cluster Auto-Installer is not supported with NDB 7.5.0 or 7.5.1 (Bug #79853, Bug #22502247).

Supported Web browsers.  The Web-based installer is supported with recent versions of Firefox and Microsoft Internet Explorer. It should also work with recent versions of Opera, Safari, and Chrome, although we have not thoroughly tested for compability with these browsers.

Required software—setup host.  The following software must be installed on the host where the Auto-Installer is run:

All of the software in the preceding list is included in the Windows version of the configuration tool, and does not need to be installed separately.

The Paramiko and Pycrypto libraries are required only if you intend to deploy NDB Cluster nodes on remote hosts, and are not needed if all nodes are on the same host where the installer is run.

Required software—remote hosts.  The only software required for remote hosts where you wish to deploy NDB Cluster nodes is the SSH server, which is usually installed by default on Linux and Solaris systems. Several alternatives are available for Windows; for an overview of these, see http://en.wikipedia.org/wiki/Comparison_of_SSH_servers.

An additional requirement when using multiple hosts is that it is possible to authenticate to any of the remote hosts using SSH and the proper keys or user credentials, as discussed in the next few paragraphs:

Authentication and security.  Three basic security or authentication mechanisms for remote access are available to the Auto-Installer, which we list and describe here:

There is no need or requirement for secure authentication when the client browser is running on the same host as the Auto-Installer back end.

See also Section 21.5.12, “NDB Cluster Security Issues”, which discusses security considerations to take into account when deploying NDB Cluster , as well as Chapter 7, Security, for more general MySQL security information.

The NDB Cluster Auto-Installer is made up of two components. The front end is a GUI client implemented as a Web page that loads and runs in a standard Web browser such as Firefox or Microsoft Internet Explorer (see Section 21.2.1.1, “NDB Cluster Auto-Installer Requirements”). The back end is a server process (ndb_setup.py) that runs on the local machine or on another host to which you have access.

These two components (client and server) communicate with each other using standard HTTP requests and responses. The back end can manage NDB Cluster software programs on any host where the back end user has granted access. If the NDB Cluster software is on a different host, the back end relies on SSH for access, using the Paramiko library for executing commands remotely (see Section 21.2.1.1, “NDB Cluster Auto-Installer Requirements”).

The remainder of this section is concerned primarily with the Web client. For more information about using the command-line tool, see Section 21.4.23, “ndb_setup.py — Start browser-based Auto-Installer for NDB Cluster ”.

NDB Cluster Auto-Installer Interface.  This section describes the layout and navigation of the NDB Cluster Auto-Installer, whose Welcome screen looks similar to what is shown here when it is first opened in the Web browser:

Figure 21.5 Welcome Screen For NDB Cluster Auto-Installer

You can access the installer UI by selecting either of the options Create New NDB Cluster or Continue Previous Cluster Configuration. A typical screen in the Auto-Installer includes the following elements:

The locations of the elements just described are shown here in a typical page in the Auto-Installer; the numbers superimposed thereupon correspond to those used in the preceding list.

Figure 21.6 Layout of the NDB Cluster Auto-Installer GUI

All of these elements except for the display panel are described in greater detail in the remainder of this section. Section 21.2.1.3, “Using the NDB Cluster Auto-Installer”, describes the panels shown in the display area as well as the functionality of each panel and the controls it contains.

Arbitrary and sequential navigation.  The Auto-Installer can display any of a number of pages covering different stages in the setup and configuration of an NDB Cluster deployment. You can navigate between pages in either of two ways. The first of these is the breadcrumb trail navigation toolbar displaying the titles of the various pages (in which the title of the current page is highlighted and disabled). From these, any desired page, in any desired order, can be reached by selecting the title of the corresponding page. This toolbar is shown here:

Figure 21.7 Detail of NDB Cluster Auto-Installer breadcrumb navigation, showing page titles/links

The second navigation mechanism provided by the Auto-Installer consists of the Next, Previous, and Finish sequential navigation buttons at the bottom right of the page. These can be used to move to the next or previous page in predetermined order, or to go to the very last page. The buttons are enabled and disabled as needed, so that you cannot, for example, advance beyond the last page.

Settings and Help menus.  These menus are positioned adjacent to one another in the top right corner of the GUI, as shown earlier in this section. The Settings menu is shown here in more detail:

Figure 21.8 NDB Cluster Auto-Installer Settings menu detail

The entries in the Settings menu are described here, in the following list:

As with the installer's navigation elements, one or more of the entries in the Settings menu may be disabled due to choices you have made previously.

The Help menu is shown here, as it appears when expanded:

Figure 21.9 The NDB Cluster Auto-Installer Help menu, expanded

The Help menu provides several options, described in the following list:

The Auto-Installer also provides context-sensitive help in the form of tooltips for most input widgets. One of these tooltips is displayed when the mouse hovers over a widget or the small question mark which can sometimes appear next to a widget label.

In addition, the names of NDB Cluster configuration parameters are linked to their descriptions in the online NDB Cluster documentation, so that if you click on the name of a given parameter, the documentation for that parameter is shown in a separate window.

The NDB Cluster Auto-Installer consists of several pages, each corresponding to a step in the process used to configure and deploy an NDB Cluster , and listed here:

The following sections describe in greater detail the purpose and function of each of these pages, in the order just listed.

21.2.1.3.2 NDB Cluster Auto-Installer Welcome Screen

The Welcome screen is loaded in the default browser when ndb_setup.py is invoked, as shown here:

Figure 21.11 The NDB Cluster Auto-Installer Welcome screen (Closeup)

This screen provides the following two choices for entering the installer, one of which must be selected to continue:

1. Create New NDB Cluster : Start the Auto-Installer with a completely new cluster to be set up and deployed.

2. Continue Previous Cluster Configuration: Start the Auto-Installer at the same point where the previous session ended, with all previous settings preserved.

The second option requires that the browser be able to access its cookies from the previous session, as these provide the mechanism by which configuration and other information generated during a session is stored. In other words, to continue the previous session with the Auto-Installer, you must use the same web browser running on the same host as you did for the previous session.

21.2.1.3.3 NDB Cluster Auto-Installer Define Cluster Screen

The Define Cluster screen is the first screen to appear following the choice made in the Welcome screen, and is used for setting general properties of the cluster. The layout of the Define Cluster screen is shown here:

Figure 21.12 The NDB Cluster Auto-Installer Define Cluster screen

The Define Cluster screen allows you to set a number of general properties for the cluster, as described in this list:

• Cluster name: A name that identifies the cluster. The default is MyCluster.

• Host list: A comma-delimited list of one or more hosts where cluster processes should run. By default, this is 127.0.0.1. If you add remote hosts to the list, you must be able to connect to them using the SSH Credentials supplied.

• Application type: Choose one of the following:

  1. Simple testing: Minimal resource usage for small-scale testing. This the default. Not intended for production environments.

  2. Web: Maximize performance for the given hardware.

  3. Real-time: Maximize performance while maximizing sensitivity to timeouts in order to minimize the time needed to detect failed cluster processes.

• Write load: Choose a level for the anticipated number of writes for the cluster as a whole. You can choose any one of the following levels:

  1. Low: The expected load includes fewer than 100 write transactions for second.

  2. Medium: The expected load includes 100 to 1000 write transactions per second.

  3. High: The expected load includes more than 1000 write transactions per second.

• SSH Credentials: Choose Key-Based SSH or enter User and Password credentials. The SSH key or a user name with password is required for connecting to any remote hosts specified in the Host list. By default, Key-Based SSH is selected, and the User and Password fileds are blank.

21.2.1.3.4 NDB Cluster Auto-Installer Define Hosts Screen

The Define Hosts screen, shown here, provides a means of viewing and specifying several key properties of each cluster host:

Figure 21.13 NDB Cluster Define Hosts screen

The hosts currently entered are displayed in the grid with various pieces of information. You can add hosts by clicking the Add hosts button and entering a list of one or more comma-separated host names, IP addresses, or both (as when editing the host list on the Define Cluster screen).

Similarly, you can remove one or more hosts using the button labelled Remove selected host(s). When you remove a host in this fashion, any process which was configured for that host is also removed.

If Automatically get resource information for new hosts is checked in the Settings menu, the Auto-Installer attempts to retrieve the platform name, amount of memory, and number of CPU cores and to fill these in automatically. The status of this is displayed in the Resource info column. Fetching the information from remote hosts is not instantaneous and may take some time, particularly from remote hosts running Windows.

If the SSH user credentials on the Define Cluster screen are changed, the tool tries to refresh the hardware information from any hosts for which information is missing. However, if a given field has already been edited, the user-supplied information is not overwritten by any value fetched from that host.

The hardware resource information, platform name, installation directory, and data directory can be edited by the user by clicking the corresponding cell in the grid, by selecting one or more hosts and clicking the button labelled Edit selected host(s). This causes a dialog box to appear, in which these fields can be edited, as shown here:

Figure 21.14 NDB Cluster Auto-Installer Edit Hosts dialog

When more than one host is selected, any edited values are applied to all selected hosts.

21.2.1.3.5 NDB Cluster Auto-Installer Define Processes Screen

The Define Processes screen, shown here, provides a way to assign NDB Cluster processes (nodes) to cluster hosts:

Figure 21.15 NDB Cluster Auto-Installer Define Processes dialog

The left-hand portion of this screen contains a process tree showing cluster hosts and processes set up to run on each one. On the right is a panel which displays information about the item currently selected in the tree.

When this screen is accessed for the first time for a given cluster, a default set of processes is defined for you, based on the number of hosts. If you later return to the Define Hosts screen, remove all hosts, and add new hosts, this also causes a new default set of processes to be defined.

NDB Cluster processes are of the following types:

• Management node.  Performs administrative tasks such as stopping individual data nodes, querying node and cluster status, and making backups. Executable: ndb_mgmd.

• Single-threaded data node.  Stores data and executes queries. Executable: ndbd.

• Multi threaded data node.  Stores data and executes queries with multiple worker threads executing in parallel. Executable: ndbmtd.

• SQL node.  MySQL server for executing SQL queries against NDB. Executable: mysqld.

• API node.  A client accessing data in NDB by means of the NDB API or other low-level client API, rather than by using SQL. See MySQL NDB Cluster API Developer Guide, for more information.

For more information about process (node) types, see Section 21.1.1, “NDB Cluster Core Concepts”.

Processes shown in the tree are numbered sequentially by type, for each host—for example, SQL node 1, SQL node 2, and so on—to simplify identification.

Each management node, data node, or SQL process must be assigned to a specific host, and is not allowed to run on any other host. An API node may be assigned to a single host, but this is not required. Instead, you can assign it to the special Any host entry which the tree also contains in addition to any other hosts, and which acts as a placeholder for processes that are allowed to run on any host. Only API processes may use this Any host entry.

Adding processes.  To add a new process to a given host, either right-click that host's entry in the tree, then select the Add process popup when it appears, or select a host in the process tree, and press the Add process button below the process tree. Performing either of these actions opens the add process dialog, as shown here:

Figure 21.16 NDB Cluster Auto-Installer Add Process Dialog

Here you can select from among the available process types described earlier this section; you can also enter an arbitrary process name to take the place of the suggested value, if desired.

Removing processes.  To delete a process, right-click on a process in the tree and select delete process from the pop up menu that appears, or select a process, then use the delete process button below the process tree.

When a process is selected in the process tree, information about that process is displayed in the panel to the right of the tree, where you can change the process name and possibly its type. Important: Currently, you can change a single-threaded data node (ndbd) to a multi-threaded data node (ndbmtd), or the reverse, only; no other process type changes are allowed. If you want to make a change between any other process types, you must delete the original process first, then add a new process of the desired type.

21.2.1.3.6 NDB Cluster Auto-Installer Define Attributes Screen

This screen has a layout similar to that of the Define Processes screen, with a process tree at the left. Unlike that screen's tree, the Define Attributes process tree is organized by process or node type, with single-threaded and multi-threaded data nodes considered to be of the same type for this purpose, in groups labelled Management Layer, Data Layer, SQL Layer, and API Layer. A panel to the right of this tree displays information regarding the item currently selected. The Define Attributes screen is shown here:

Figure 21.17 NDB Cluster Auto-Installer Define Attributes screen

A checkbox labelled Show advanced configuration is located below the process tree. Checking this box makes advanced options visible in the information pane. These options are set and used whether or not they are visible.

You can edit attributes for a single process by selecting that process from the tree, or for all processes of the same type in the cluster by selecting one of the Layer folders. A per-process value set for a given attribute overrides any per-group setting for that attribute that would otherwise apply to the process in question. An example of such an information panel (for an SQL process) is shown here:

Figure 21.18 Define Attributes Detail With SQL Process Attributes

For some of the attributes shown in the information panel, a button bearing a plus sign is displayed to the right, which means that the value of this attribute can be overridden. This + button activates an input widget for the attribute, enabling you to change its value. When the value has been overridden, this button changes into a button showing an X, as shown here:

Figure 21.19 Define Attributes Detail, Overriding Attribute Default Value

Clicking the X button next to an attribute undoes any changes made to it; it immediately reverts to the predefined value.

All configuration attributes have predefined values calculated by the installer, based such factors as host name, node ID, node type, and so on. In most cases, these values may be left as they are. If you are not familiar with it already, it is highly recommended that you read the applicable documentation before making changes to any of the attribute values. To make finding this information easier, each attribute name shown in the information panel is linked to its description in the online NDB Cluster documentation.

21.2.1.3.7 NDB Cluster Auto-Installer Deploy Cluster Screen

This screen allows you to perform the following tasks:

• Review process startup commands and configuration files to be applied

• Distribute configuration files by creating any necessary files and directories on all cluster hosts—that is, deploy the cluster as presently configured

• Start and stop the cluster

The Deploy Cluster screen is shown here:

Figure 21.20 NDB Cluster Auto-Installer Deploy Cluster screen

Like the Define Attributes screen, this screens features a process tree, organized by process type, on the left hand side. Next to each process is a status icon whose color indicates the current status of the process: green if it is running; yellow if it is starting or stopping; red if the process is stopped.

To the right of the process tree are two information panels, the upper panel showing the startup command or commands needed to start the selected process. (For some processes, more than one command may be required—for example, if initialization is necessary.) The lower panel shows the contents of the configuration file, if any, for the given process; currently, the management node process is only type of process having a configuration file. Other process types are configured using command-line parameters when starting the process, or by obtaining configuration information from the management nodes as needed in real time.

Three buttons are located immediately below the process tree. These are labelled as and perform the functions described in the following list:

• Deploy cluster: Verify that the configuration is valid. Create any directories required on the cluster hosts, and distribute the configuration files onto the hosts. A progress bar shows how far the deployment has proceeded.

• Start cluster: The cluster is deployed as with Deploy cluster, after which all cluster processes are started in the correct order.

  Starting these processes may take some time. If the estimated time to completion is too large, the installer provides an opportunity to cancel or to continue of the startup procedure. A progress bar indicates the current status of the startup procedure, as shown here:

  Figure 21.21 Progress Bar With Status of Node Startup Process

  
  
  

  The process status icons adjoining the process tree mentioned previously also update with the status of each process.

• Stop cluster: After the cluster has been started, you can stop it using the this. As with starting the cluster, cluster shutdown is not instantaneous, and may require some time complete. A progress bar, similar to that displayed during cluster startup, shows the approximate current status of the cluster shutdown procedure, as do the process status icons adjoining the process tree.

The Auto-Installer generates a my.cnf file containing the appropriate options for each mysqld process in the cluster.

This section covers the steps necessary to install the correct executables for each type of Cluster node from precompiled binaries supplied by Oracle.

For setting up a cluster using precompiled binaries, the first step in the installation process for each cluster host is to download the latest NDB Cluster 7.5 binary archive (mysql-cluster-gpl-7.5.7-linux-i686-glibc23.tar.gz from the NDB Cluster downloads area. We assume that you have placed this file in each machine's /var/tmp directory. (If you do require a custom binary, see Section 2.9.3, “Installing MySQL Using a Development Source Tree”.)

SQL nodes.  On each of the machines designated to host SQL nodes, perform the following steps as the system root user:

Remember that the preceding steps must be repeated on each machine where an SQL node is to reside.

Data nodes.  Installation of the data nodes does not require the mysqld binary. Only the NDB Cluster data node executable ndbd (single-threaded) or ndbmtd (multi-threaded) is required. These binaries can also be found in the .tar.gz archive. Again, we assume that you have placed this archive in /var/tmp.

As system root (that is, after using sudo, su root, or your system's equivalent for temporarily assuming the system administrator account's privileges), perform the following steps to install the data node binaries on the data node hosts:

The preceding steps should be repeated on each data node host.

Although only one of the data node executables is required to run an NDB Cluster data node, we have shown you how to install both ndbd and ndbmtd in the preceding instructions. We recommend that you do this when installing or upgrading NDB Cluster , even if you plan to use only one of them, since this will save time and trouble in the event that you later decide to change from one to the other.

Management nodes.  Installation of the management node does not require the mysqld binary. Only the NDB Cluster management server (ndb_mgmd) is required; you most likely want to install the management client (ndb_mgm) as well. Both of these binaries also be found in the .tar.gz archive. Again, we assume that you have placed this archive in /var/tmp.

As system root, perform the following steps to install ndb_mgmd and ndb_mgm on the management node host:

In Section 21.2.4, “Initial Configuration of NDB Cluster”, we create configuration files for all of the nodes in our example NDB Cluster .

This section covers the steps necessary to install the correct executables for each type of NDB Cluster node using RPM packages supplied by Oracle beginning with NDB 7.5.4. For information about RPMs for previous versions of NDB Cluster , see Installation using old-style RPMs (NDB 7.5.3 and earlier).

RPMs are available for both 32-bit and 64-bit Linux platforms. The filenames for these RPMs use the following pattern:

mysql-cluster-license-component-ver-rev.distro.arch.rpm
  
    license:= {commercial | community}
    
    component: {management-server | data-node | server | client | other—see text}
    
    ver: major.minor.release
    
    rev: major.minor
    
    distro: {el5 | el6 | el7 | sles12}
    
    arch: {i686 | x86_64}

license indicates whether the RPM is part of a Commercial or Community release of NDB Cluster . In the remainder of this section, we assume for the examples that you are installing a Community release.

Possible values for component, with descriptions, can be found in the following table:

A single bundle (.tar file) of all NDB Cluster RPMs for a given platform and architecture is also available. The name of this file follows the pattern shown here:

mysql-cluster-license-ver-rev.distro.arch.rpm-tar

You can extract the individual RPM files from this file using tar or your preferred tool for extracting archives.

The components required to install the three major types of NDB Cluster nodes are given in the following list:

In addition, the client RPM should be installed to provide the ndb_mgm management client on at least one management node. You may also wish to install it on SQL nodes, to have mysql and other MySQL client programs available on these. We discuss installation of nodes by type later in this section.

ver represents the three-part NDB storage engine version number in 7.5.x format, shown as 7.5.7 in the examples. rev provides the RPM revision number in major.minor format. In the examples shown in this section, we use 1.1 for this value.

The distro (Linux distribution) is one of rhel5 (Oracle Linux 5, Red Hat Enterprise Linux 4 and 5), el6 (Oracle Linux 6, Red Hat Enterprise Linux 6), el7 (Oracle Linux 7, Red Hat Enterprise Linux 7), or sles12 (SUSE Enterprise Linux 12). For the examples in this section, we assume that the host runs Oracle Linux 7, Red Hat Enterprise Linux 7, or the equivalent (el7).

arch is i686 for 32-bit RPMs and x86_64 for 64-bit versions. In the examples shown here, we assume a 64-bit platform.

The NDB Cluster version number in the RPM file names (shown here as 7.5.7) can vary according to the version which you are actually using. It is very important that all of the Cluster RPMs to be installed have the same version number. The architecture should also be appropriate to the machine on which the RPM is to be installed; in particular, you should keep in mind that 64-bit RPMs (x86_64) cannot be used with 32-bit operating systems (use i686 for the latter).

Data nodes.  On a computer that is to host an NDB Cluster data node it is necessary to install only the data-node RPM. To do so, copy this RPM to the data node host, and run the following command as the system root user, replacing the name shown for the RPM as necessary to match that of the RPM downloaded from the MySQL web site:

shell> rpm -Uhv mysql-cluster-community-data-node-7.5.7-1.1.el7.x86_64.rpm

This installs the ndbd and ndbmtd data node binaries in /usr/sbin. Either of these can be used to run a data node process on this host.

SQL nodes.  Copy the server and common RPMs to each machine to be used for hosting an NDB Cluster SQL node (server requires common). Install the server RPM by executing the following command as the system root user, replacing the name shown for the RPM as necessary to match the name of the RPM downloaded from the MySQL web site:

shell> rpm -Uhv mysql-cluster-community-server-7.5.7-1.1.el7.x86_64.rpm

This installs the MySQL server binary (mysqld), with NDB storage engine support, in the /usr/sbin directory. It also installs all needed MySQL Server support files and useful MySQL server programs, including the mysql.server and mysqld_safe startup scripts (in /usr/share/mysql and /usr/bin, respectively). The RPM installer should take care of general configuration issues (such as creating the mysql user and group, if needed) automatically.

To administer the SQL node (MySQL server), you should also install the client RPM, as shown here:

shell> rpm -Uhv mysql-cluster-community-client-7.5.7-1.1.el7.x86_64.rpm

This installs the mysql client and other MySQL client programs, such as mysqladmin and mysqldump, to /usr/bin.

Management nodes.  To install the NDB Cluster management server, it is necessary only to use the management-server RPM. Copy this RPM to the computer intended to host the management node, and then install it by running the following command as the system root user (replace the name shown for the RPM as necessary to match that of the management-server RPM downloaded from the MySQL web site):

shell> rpm -Uhv mysql-cluster-commercial-management-server-7.5.7-1.1.el7.x86_64.rpm

This RPM installs the management server binary ndb_mgmd in the /usr/sbin directory. While this is the only program actually required for running a management node, it is also a good idea to have the ndb_mgm NDB Cluster management client available as well. You can obtain this program, as well as other NDB client programs such as ndb_desc and ndb_config, by installing the client RPM as described previously.

See Section 2.5.5, “Installing MySQL on Linux Using RPM Packages from Oracle”, for general information about installing MySQL using RPMs supplied by Oracle.

After installing from RPM, you still need to configure the cluster; see Section 21.2.4, “Initial Configuration of NDB Cluster”, for the relevant information.

Installation using old-style RPMs (NDB 7.5.3 and earlier).  The information in the remainder of this section applies only to NDB 7.5.3 and earlier, and provides the steps necessary to install the correct executables for each type of NDB Cluster node using “old-style” RPM packages as supplied by Oracle prior to NDB 7.5.4. The filenames for these “old-style” RPMs use the following pattern:

MySQL-Cluster-component-producttype-ndbversion-revision.distribution.architecture.rpm

component:= {server | client [| other]}

producttype:= {gpl | advanced}

ndbversion:= major.minor.release

distribution:= {sles11 | rhel5 | el6}

architecture:= {i386 | x86_64}

The component can be server or client. (Other values are possible, but since only the server and client components are required for a working NDB Cluster installation, we do not discuss them here.) The producttype for Community RPMs downloaded from http://dev.mysql.com/downloads/cluster/ is always gpl; advanced is used to indicate commercial releases. ndbversion represents the three-part NDB storage engine version number in 7.5.x format; we use 7.5.3 throughout the rest of this section. The RPM revision is shown as 1 in the examples following. The distribution can be one of sles11 (SUSE Enterprise Linux 11), rhel5 (Oracle Linux 5, Red Hat Enterprise Linux 4 and 5), or el6 (Oracle Linux 6, Red Hat Enterprise Linux 6). The architecture is i386 for 32-bit RPMs and x86_64 for 64-bit versions.

For an NDB Cluster , one and possibly two RPMs are required:

It is very important that all of the Cluster RPMs to be installed have the same version number. The architecture designation should also be appropriate to the machine on which the RPM is to be installed; in particular, you should keep in mind that 64-bit RPMs cannot be used with 32-bit operating systems.

Data nodes.  On a computer that is to host a cluster data node it is necessary to install only the server RPM. To do so, copy this RPM to the data node host, and run the following command as the system root user, replacing the name shown for the RPM as necessary to match that of the RPM downloaded from the MySQL web site:

shell> rpm -Uhv MySQL-Cluster-server-gpl-7.5.3-1.sles11.i386.rpm

Although this installs all NDB Cluster binaries, only the program ndbd or ndbmtd (both in /usr/sbin) is actually needed to run an NDB Cluster data node.

SQL nodes.  On each machine to be used for hosting a cluster SQL node, install the server RPM by executing the following command as the system root user, replacing the name shown for the RPM as necessary to match the name of the RPM downloaded from the MySQL web site:

shell> rpm -Uhv MySQL-Cluster-server-gpl-7.5.3-1.sles11.i386.rpm

This installs the MySQL server binary (mysqld) with NDB storage engine support in the /usr/sbin directory, as well as all needed MySQL Server support files. It also installs the mysql.server and mysqld_safe startup scripts (in /usr/share/mysql and /usr/bin, respectively). The RPM installer should take care of general configuration issues (such as creating the mysql user and group, if needed) automatically.

To administer the SQL node (MySQL server), you should also install the client RPM, as shown here:

shell> rpm -Uhv MySQL-Cluster-client-gpl-7.5.3-1.sles11.i386.rpm

This installs the mysql client program.

Management nodes.  To install the NDB Cluster management server, it is necessary only to use the server RPM. Copy this RPM to the computer intended to host the management node, and then install it by running the following command as the system root user (replace the name shown for the RPM as necessary to match that of the server RPM downloaded from the MySQL web site):

shell> rpm -Uhv MySQL-Cluster-server-gpl-7.5.3-1.sles11.i386.rpm

Although this RPM installs many other files, only the management server binary ndb_mgmd (in the /usr/sbin directory) is actually required for running a management node. The server RPM also installs ndb_mgm, the NDB management client.

See Section 2.5.5, “Installing MySQL on Linux Using RPM Packages from Oracle”, for general information about installing MySQL using RPMs supplied by Oracle. See Section 21.2.4, “Initial Configuration of NDB Cluster”, for information about required post-installation configuration.

The section provides information about installing NDB Cluster on Debian and related Linux distributions such Ubuntu using the .deb files supplied by Oracle for this purpose.

Oracle provides .deb installer files for NDB Cluster 7.5 for 32-bit and 64-bit platforms. For a Debian-based system, only a single installer file is necessary. This file is named using the pattern shown here, according to the applicable NDB Cluster version, Debian version, and architecture:

mysql-cluster-gpl-ndbver-debiandebianver-arch.deb

Here, ndbver is the 3-part NDB engine version number, debianver is the major version of Debian (6.0 or 7), and arch is one of i686 or x86_64. In the examples that follow, we assume you wish to install NDB 7.5.7 on a 64-bit Debian 7 system; in this case, the installer file is named mysql-cluster-gpl-7.5.7-debian7-x86_64.deb.

Once you have downloaded the appropriate .deb file, you can install it from the command line using dpkg, like this:

shell> dpkg -i mysql-cluster-gpl-7.5.7-debian7-i686.deb

You can also remove it using dpkg as shown here:

shell> dpkg -r mysql

The installer file should also be compatible with most graphical package managers that work with .deb files, such as GDebi for the Gnome desktop.

The .deb file installs NDB Cluster under /opt/mysql/server-version/, where version is the 2-part release series version for the included MySQL server. For NDB 7.5, this is always 5.7. The directory layout is the same as that for the generic Linux binary distribution (see Table 2.3, “MySQL Installation Layout for Generic Unix/Linux Binary Package”), with the exception that startup scripts and configuration files are found in support-files instead of share. All NDB Cluster executables, such as ndb_mgm, ndbd, and ndb_mgmd, are placed in the bin directory.

This section provides information about compiling NDB Cluster on Linux and other Unix-like platforms. Building NDB Cluster from source is similar to building the standard MySQL Server, although it differs in a few key respects discussed here. For general information about building MySQL from source, see Section 2.9, “Installing MySQL from Source”. For information about compiling NDB Cluster on Windows platforms, see Section 21.2.3.2, “Compiling and Installing NDB Cluster from Source on Windows”.

Building NDB Cluster requires using the NDB Cluster sources. These are available from the NDB Cluster downloads page at http://dev.mysql.com/downloads/cluster/. The archived source file should have a name similar to mysql-cluster-gpl-7.5.7.tar.gz. You can also obtain MySQL development sources from launchpad.net. Building NDB Cluster from standard MySQL Server 5.7 sources is not supported.

The WITH_NDBCLUSTER_STORAGE_ENGINE option for CMake causes the binaries for the management nodes, data nodes, and other NDB Cluster programs to be built; it also causes mysqld to be compiled with NDB storage engine support. This option (or its alias WITH_NDBCLUSTER) is required when building NDB Cluster .

For more information about CMake options specific to building NDB Cluster , see Options for Compiling NDB Cluster .

After you have run make && make install (or your system's equivalent), the result is similar to what is obtained by unpacking a precompiled binary to the same location.

Management nodes.  When building from source and running the default make install, the management server and management client binaries (ndb_mgmd and ndb_mgm) can be found in /usr/local/mysql/bin. Only ndb_mgmd is required to be present on a management node host; however, it is also a good idea to have ndb_mgm present on the same host machine. Neither of these executables requires a specific location on the host machine's file system.

Data nodes.  The only executable required on a data node host is the data node binary ndbd or ndbmtd. (mysqld, for example, does not have to be present on the host machine.) By default, when building from source, this file is placed in the directory /usr/local/mysql/bin. For installing on multiple data node hosts, only ndbd or ndbmtd need be copied to the other host machine or machines. (This assumes that all data node hosts use the same architecture and operating system; otherwise you may need to compile separately for each different platform.) The data node binary need not be in any particular location on the host's file system, as long as the location is known.

When compiling NDB Cluster from source, no special options are required for building multi-threaded data node binaries. Configuring the build with NDB storage engine support causes ndbmtd to be built automatically; make install places the ndbmtd binary in the installation bin directory along with mysqld, ndbd, and ndb_mgm.

SQL nodes.  If you compile MySQL with clustering support, and perform the default installation (using make install as the system root user), mysqld is placed in /usr/local/mysql/bin. Follow the steps given in Section 2.9, “Installing MySQL from Source” to make mysqld ready for use. If you want to run multiple SQL nodes, you can use a copy of the same mysqld executable and its associated support files on several machines. The easiest way to do this is to copy the entire /usr/local/mysql directory and all directories and files contained within it to the other SQL node host or hosts, then repeat the steps from Section 2.9, “Installing MySQL from Source” on each machine. If you configure the build with a nondefault PREFIX option, you must adjust the directory accordingly.

In Section 21.2.4, “Initial Configuration of NDB Cluster”, we create configuration files for all of the nodes in our example NDB Cluster .

This section describes a basic installation of NDB Cluster on Windows using a binary no-install NDB Cluster release provided by Oracle, using the same 4-node setup outlined in the beginning of this section (see Section 21.2, “NDB Cluster Installation”), as shown in the following table:

As on other platforms, the NDB Cluster host computer running an SQL node must have installed on it a MySQL Server binary (mysqld.exe). You should also have the MySQL client (mysql.exe) on this host. For management nodes and data nodes, it is not necessary to install the MySQL Server binary; however, each management node requires the management server daemon (ndb_mgmd.exe); each data node requires the data node daemon (ndbd.exe or ndbmtd.exe). For this example, we refer to ndbd.exe as the data node executable, but you can install ndbmtd.exe, the multi-threaded version of this program, instead, in exactly the same way. You should also install the management client (ndb_mgm.exe) on the management server host. This section covers the steps necessary to install the correct Windows binaries for each type of NDB Cluster node.

For setting up an NDB Cluster using Oracles's no-install binaries, the first step in the installation process is to download the latest NDB Cluster Windows binary archive from http://dev.mysql.com/downloads/cluster/. This archive has a filename of the form mysql-cluster-gpl-noinstall-ver-winarch.zip, where ver is the NDB storage engine version (such as 7.5.7), and arch is the architecture (32 for 32-bit binaries, and 64 for 64-bit binaries). For example, the NDB Cluster 7.5.7 no-install archive for 32-bit Windows systems is named mysql-cluster-gpl-noinstall-7.5.7-win32.zip.

You can run 32-bit NDB Cluster binaries on both 32-bit and 64-bit versions of Windows; however, 64-bit NDB Cluster binaries can be used only on 64-bit versions of Windows. If you are using a 32-bit version of Windows on a computer that has a 64-bit CPU, then you must use the 32-bit NDB Cluster binaries.

To minimize the number of files that need to be downloaded from the Internet or copied between machines, we start with the computer where you intend to run the SQL node.

SQL node.  We assume that you have placed a copy of the no-install archive in the directory C:\Documents and Settings\username\My Documents\Downloads on the computer having the IP address 192.168.0.20, where username is the name of the current user. (You can obtain this name using ECHO %USERNAME% on the command line.) To install and run NDB Cluster executables as Windows services, this user should be a member of the Administrators group.

Extract all the files from the archive. The Extraction Wizard integrated with Windows Explorer is adequate for this task. (If you use a different archive program, be sure that it extracts all files and directories from the archive, and that it preserves the archive's directory structure.) When you are asked for a destination directory, enter C:\, which causes the Extraction Wizard to extract the archive to the directory C:\mysql-cluster-gpl-noinstall-ver-winarch. Rename this directory to C:\mysql.

It is possible to install the NDB Cluster binaries to directories other than C:\mysql\bin; however, if you do so, you must modify the paths shown in this procedure accordingly. In particular, if the MySQL Server (SQL node) binary is installed to a location other than C:\mysql or C:\Program Files\MySQL\MySQL Server 5.7, or if the SQL node's data directory is in a location other than C:\mysql\data or C:\Program Files\MySQL\MySQL Server 5.7\data, extra configuration options must be used on the command line or added to the my.ini or my.cnf file when starting the SQL node. For more information about configuring a MySQL Server to run in a nonstandard location, see Section 2.3.5, “Installing MySQL on Microsoft Windows Using a noinstall Zip Archive”.

For a MySQL Server with NDB Cluster support to run as part of an NDB Cluster , it must be started with the options --ndbcluster and --ndb-connectstring. While you can specify these options on the command line, it is usually more convenient to place them in an option file. To do this, create a new text file in Notepad or another text editor. Enter the following configuration information into this file:

[mysqld]
# Options for mysqld process:
ndbcluster                      # run NDB storage engine
ndb-connectstring=192.168.0.10  # location of management server

You can add other options used by this MySQL Server if desired (see Section 2.3.5.2, “Creating an Option File”), but the file must contain the options shown, at a minimum. Save this file as C:\mysql\my.ini. This completes the installation and setup for the SQL node.

Data nodes.  An NDB Cluster data node on a Windows host requires only a single executable, one of either ndbd.exe or ndbmtd.exe. For this example, we assume that you are using ndbd.exe, but the same instructions apply when using ndbmtd.exe. On each computer where you wish to run a data node (the computers having the IP addresses 192.168.0.30 and 192.168.0.40), create the directories C:\mysql, C:\mysql\bin, and C:\mysql\cluster-data; then, on the computer where you downloaded and extracted the no-install archive, locate ndbd.exe in the C:\mysql\bin directory. Copy this file to the C:\mysql\bin directory on each of the two data node hosts.

To function as part of an NDB Cluster , each data node must be given the address or hostname of the management server. You can supply this information on the command line using the --ndb-connectstring or -c option when starting each data node process. However, it is usually preferable to put this information in an option file. To do this, create a new text file in Notepad or another text editor and enter the following text:

[mysql_cluster]
# Options for data node process:
ndb-connectstring=192.168.0.10  # location of management server

Save this file as C:\mysql\my.ini on the data node host. Create another text file containing the same information and save it on as C:mysql\my.ini on the other data node host, or copy the my.ini file from the first data node host to the second one, making sure to place the copy in the second data node's C:\mysql directory. Both data node hosts are now ready to be used in the NDB Cluster , which leaves only the management node to be installed and configured.

Management node.  The only executable program required on a computer used for hosting an NDB Cluster management node is the management server program ndb_mgmd.exe. However, in order to administer the NDB Cluster once it has been started, you should also install the NDB Cluster management client program ndb_mgm.exe on the same machine as the management server. Locate these two programs on the machine where you downloaded and extracted the no-install archive; this should be the directory C:\mysql\bin on the SQL node host. Create the directory C:\mysql\bin on the computer having the IP address 192.168.0.10, then copy both programs to this directory.

You should now create two configuration files for use by ndb_mgmd.exe:

DataDir=C:\mysql\bin\cluster-logs

DataDir=C:\\mysql\\bin\\cluster-logs  # Escaped backslashes

DataDir=C:/mysql/bin/cluster-logs     # Forward slashes

Oracle provides precompiled NDB Cluster binaries for Windows which should be adequate for most users. However, if you wish, it is also possible to compile NDB Cluster for Windows from source code. The procedure for doing this is almost identical to the procedure used to compile the standard MySQL Server binaries for Windows, and uses the same tools. However, there are two major differences:

For more information about CMake options specific to building NDB Cluster , see Options for Compiling NDB Cluster .

Once the build process is complete, you can create a Zip archive containing the compiled binaries; Section 2.9.2, “Installing MySQL Using a Standard Source Distribution” provides the commands needed to perform this task on Windows systems. The NDB Cluster binaries can be found in the bin directory of the resulting archive, which is equivalent to the no-install archive, and which can be installed and configured in the same manner. For more information, see Section 21.2.3.1, “Installing NDB Cluster on Windows from a Binary Release”.

Once the NDB Cluster executables and needed configuration files are in place, performing an initial start of the cluster is simply a matter of starting the NDB Cluster executables for all nodes in the cluster. Each cluster node process must be started separately, and on the host computer where it resides. The management node should be started first, followed by the data nodes, and then finally by any SQL nodes.

You should now be ready to work with database objects and data using NDB Cluster 's NDBCLUSTER storage engine. See Section 21.2.6, “NDB Cluster Example with Tables and Data”, for more information and examples.

You can also install ndb_mgmd.exe, ndbd.exe, and ndbmtd.exe as Windows services. For information on how to do this, see Section 21.2.3.4, “Installing NDB Cluster Processes as Windows Services”).

Once you are satisfied that NDB Cluster is running as desired, you can install the management nodes and data nodes as Windows services, so that these processes are started and stopped automatically whenever Windows is started or stopped. This also makes it possible to control these processes from the command line with the appropriate NET START or NET STOP command, or using the Windows graphical Services utility.

Installing programs as Windows services usually must be done using an account that has Administrator rights on the system.

To install the management node as a service on Windows, invoke ndb_mgmd.exe from the command line on the machine hosting the management node, using the --install option, as shown here:

C:\> C:\mysql\bin\ndb_mgmd.exe --install
Installing service 'NDB Cluster  Management Server'
  as '"C:\mysql\bin\ndbd.exe" "--service=ndb_mgmd"'
Service successfully installed.

The --install option must be used first, ahead of any other options that might be specified for ndb_mgmd.exe. However, it is preferable to specify such options in an options file instead. If your options file is not in one of the default locations as shown in the output of ndb_mgmd.exe --help, you can specify the location using the --config-file option.

Now you should be able to start and stop the management server like this:

C:\> NET START ndb_mgmd
The NDB Cluster  Management Server service is starting.
The NDB Cluster  Management Server service was started successfully.

C:\> NET STOP ndb_mgmd
The NDB Cluster  Management Server service is stopping..
The NDB Cluster  Management Server service was stopped successfully.

You can also start or stop the management server as a Windows service using the descriptive name, as shown here:

C:\> NET START 'NDB Cluster  Management Server'
The NDB Cluster  Management Server service is starting.
The NDB Cluster  Management Server service was started successfully.

C:\> NET STOP  'NDB Cluster  Management Server'
The NDB Cluster  Management Server service is stopping..
The NDB Cluster  Management Server service was stopped successfully.

However, it is usually simpler to specify a short service name or to permit the default service name to be used when installing the service, and then reference that name when starting or stopping the service. To specify a service name other than ndb_mgmd, append it to the --install option, as shown in this example:

C:\> C:\mysql\bin\ndb_mgmd.exe --install=mgmd1
Installing service 'NDB Cluster  Management Server'
  as '"C:\mysql\bin\ndb_mgmd.exe" "--service=mgmd1"'
Service successfully installed.

Now you should be able to start or stop the service using the name you have specified, like this:

C:\> NET START mgmd1
The NDB Cluster  Management Server service is starting.
The NDB Cluster  Management Server service was started successfully.

C:\> NET STOP mgmd1
The NDB Cluster  Management Server service is stopping..
The NDB Cluster  Management Server service was stopped successfully.

To remove the management node service, invoke ndb_mgmd.exe with the --remove option, as shown here:

C:\> C:\mysql\bin\ndb_mgmd.exe --remove
Removing service 'NDB Cluster  Management Server'
Service successfully removed.

If you installed the service using a service name other than the default, you can remove the service by passing this name as the value of the --remove option, like this:

C:\> C:\mysql\bin\ndb_mgmd.exe --remove=mgmd1
Removing service 'mgmd1'
Service successfully removed.

Installation of an NDB Cluster data node process as a Windows service can be done in a similar fashion, using the --install option for ndbd.exe (or ndbmtd.exe), as shown here:

C:\> C:\mysql\bin\ndbd.exe --install
Installing service 'NDB Cluster  Data Node Daemon' as '"C:\mysql\bin\ndbd.exe" "--service=ndbd"'
Service successfully installed.

Now you can start or stop the data node using either the default service name or the descriptive name with net start or net stop, as shown in the following example:

C:\> NET START ndbd
The NDB Cluster  Data Node Daemon service is starting.
The NDB Cluster  Data Node Daemon service was started successfully.

C:\> NET STOP ndbd
The NDB Cluster  Data Node Daemon service is stopping..
The NDB Cluster  Data Node Daemon service was stopped successfully.

C:\> NET START 'NDB Cluster  Data Node Daemon'
The NDB Cluster  Data Node Daemon service is starting.
The NDB Cluster  Data Node Daemon service was started successfully.

C:\> NET STOP 'NDB Cluster  Data Node Daemon'
The NDB Cluster  Data Node Daemon service is stopping..
The NDB Cluster  Data Node Daemon service was stopped successfully.

To remove the data node service, invoke ndbd.exe with the --remove option, as shown here:

C:\> C:\mysql\bin\ndbd.exe --remove
Removing service 'NDB Cluster  Data Node Daemon'
Service successfully removed.

As with ndb_mgmd.exe (and mysqld.exe), when installing ndbd.exe as a Windows service, you can also specify a name for the service as the value of --install, and then use it when starting or stopping the service, like this:

C:\> C:\mysql\bin\ndbd.exe --install=dnode1
Installing service 'dnode1' as '"C:\mysql\bin\ndbd.exe" "--service=dnode1"'
Service successfully installed.

C:\> NET START dnode1
The NDB Cluster  Data Node Daemon service is starting.
The NDB Cluster  Data Node Daemon service was started successfully.

C:\> NET STOP dnode1
The NDB Cluster  Data Node Daemon service is stopping..
The NDB Cluster  Data Node Daemon service was stopped successfully.

If you specified a service name when installing the data node service, you can use this name when removing it as well, by passing it as the value of the --remove option, as shown here:

C:\> C:\mysql\bin\ndbd.exe --remove=dnode1
Removing service 'dnode1'
Service successfully removed.

Installation of the SQL node as a Windows service, starting the service, stopping the service, and removing the service are done in a similar fashion, using mysqld --install, NET START, NET STOP, and mysqld --remove. For additional information, see Section 2.3.5.8, “Starting MySQL as a Windows Service”.

The summary table in this section provides information about parameters used in the [ndbd] or [ndbd default] sections of a config.ini file for configuring NDB Cluster data nodes. For detailed descriptions and other additional information about each of these parameters, see Section 21.3.3.6, “Defining NDB Cluster Data Nodes”.

These parameters also apply to ndbmtd, the multi-threaded version of ndbd. For more information, see Section 21.4.3, “ndbmtd — The NDB Cluster Data Node Daemon (Multi-Threaded)”.

Restart types.  Changes in NDB Cluster configuration parameters do not take effect until the cluster is restarted. The type of restart required to change a given parameter is indicated in the summary table as follows:

For more information about restart types, see Section 21.3.2, “Overview of NDB Cluster Configuration Parameters, Options, and Variables”.

NDB Cluster also supports the addition of new data node groups online, to a running cluster. For more information, see Section 21.5.14, “Adding NDB Cluster Data Nodes Online”.

The summary table in this section provides information about parameters used in the [ndb_mgmd] or [mgm] sections of a config.ini file for configuring NDB Cluster management nodes. For detailed descriptions and other additional information about each of these parameters, see Section 21.3.3.5, “Defining an NDB Cluster Management Server”.

Restart types.  Changes in NDB Cluster configuration parameters do not take effect until the cluster is restarted. The type of restart required to change a given parameter is indicated in the summary table as follows:

For more information about restart types, see Section 21.3.2, “Overview of NDB Cluster Configuration Parameters, Options, and Variables”.