Chapter 21 MySQL NDB Cluster 7.5 and NDB Cluster 7.6

ndbinfo Enhancements.  A number of changes are made in the ndbinfo database, chief of which is that it now provides detailed information about NDB Cluster node configuration parameters.

The config_params table has been made read-only, and has been enhanced with additional columns providing information about each configuration parameter, including the parameter's type, default value, maximum and minimum values (where applicable), a brief description of the parameter, and whether the parameter is required. This table also provides each parameter with a unique param_number.

A row in the config_values table shows the current value of a given parameter on the node having a specified ID. The parameter is identified by the value of the config_param column, which maps to the config_params table's param_number.

Using this relationship you can write a join on these two tables to obtain the default, maximum, minimum, and current values for one or more NDB Cluster configuration parameters by name. An example SQL statement using such a join is shown here:

SELECT  p.param_name AS Name,
        v.node_id AS Node,
        p.param_type AS Type,
        p.param_default AS 'Default',
        p.param_min AS Minimum,
        p.param_max AS Maximum,
        CASE p.param_mandatory WHEN 1 THEN 'Y' ELSE 'N' END AS 'Required',
        v.config_value AS Current
FROM    config_params p
JOIN    config_values v
ON      p.param_number = v.config_param
WHERE   p. param_name IN ('NodeId', 'HostName','DataMemory', 'IndexMemory');

For more information about these changes, see Section 21.5.10.8, “The ndbinfo config_params Table”. See Section 21.5.10.9, “The ndbinfo config_values Table”, for further information and examples.

In addition, the ndbinfo database no longer depends on the MyISAM storage engine. All ndbinfo tables and views now use NDB (shown as NDBINFO).

Several new ndbinfo tables were introduced in NDB 7.5.4. These tables are listed here, with brief descriptions:

See the descriptions of the individual tables for more information.

Default row and column format changes.  Starting with NDB 7.5.1, the default value for both the ROW_FORMAT option and the COLUMN_FORMAT option for CREATE TABLE can be set to DYNAMIC rather than FIXED, using a new MySQL server variable ndb_default_column_format is added as part of this change; set this to FIXED or DYNAMIC (or start mysqld with the equivalent option --ndb-default-column-format=FIXED) to force this value to be used for COLUMN_FORMAT and ROW_FORMAT. Prior to NDB 7.5.4, the default for this variable was DYNAMIC; in this and later versions, the default is FIXED, which provides backwards compatibility with prior releases (Bug #24487363).

The row format and column format used by existing table columns are unaffected by this change. New columns added to such tables use the new defaults for these (possibly overridden by ndb_default_column_format), and existing columns are changed to use these as well, provided that the ALTER TABLE statement performing this operation specifies ALGORITHM=COPY.

ndb_binlog_index no longer dependent on MyISAM.  As of NDB 7.5.2, the ndb_binlog_index table employed in NDB Cluster Replication now uses the InnoDB storage engine instead of MyISAM. When upgrading, you can run mysql_upgrade with --force --upgrade-system-tables to cause it to execute ALTER TABLE ... ENGINE=INNODB on this table. Use of MyISAM for this table remains supported for backward compatibility.

A benefit of this change is that it makes it possible to depend on transactional behavior and lock-free reads for this table, which can help alleviate concurrency issues during purge operations and log rotation, and improve the availability of this table.

ALTER TABLE changes.  NDB Cluster formerly supported an alternative syntax for online ALTER TABLE. This is no longer supported in NDB Cluster 7.5, which makes exclusive use of ALGORITHM = DEFAULT|COPY|INPLACE for table DDL, as in the standard MySQL Server.

Another change affecting the use of this statement is that ALTER TABLE ... ALGORITHM=INPLACE RENAME may now contain DDL operations in addition to the renaming.

ExecuteOnComputer parameter deprecated.  The ExecuteOnComputer configuration parameter for management nodes, data nodes, and API nodes has been deprecated and is now subject to removal in a future release of NDB Cluster. You should use the equivalent HostName parameter for all three types of nodes.

records-per-key optimization.  The NDB handler now uses the records-per-key interface for index statistics implemented for the optimizer in MySQL 5.7.5. Some of the benefits from this change include those listed here:

Connection pool node IDs.  NDB 7.5.0 adds the mysqld --ndb-cluster-connection-pool-nodeids option, which allows a set of node IDs to be set for the connection pool. This setting overrides --ndb-nodeid, which means that it also overrides both the --ndb-connectstring option and the NDB_CONNECTSTRING environment variable.

create_old_temporals removed.  The create_old_temporals system variable was deprecated in NDB Cluster 7.4, and has now been removed.

ndb_mgm Client PROMPT command.  NDB Cluster 7.5 adds a new command for setting the client's command-line prompt. The following example illustrates the use of the PROMPT command:

ndb_mgm> PROMPT mgm#1:
mgm#1: SHOW
Cluster Configuration
---------------------
[ndbd(NDB)]     4 node(s)
id=5    @10.100.1.1  (mysql-5.7.29-ndb-7.5.17, Nodegroup: 0, *)
id=6    @10.100.1.3  (mysql-5.7.29-ndb-7.5.17, Nodegroup: 0)
id=7    @10.100.1.9  (mysql-5.7.29-ndb-7.5.17, Nodegroup: 1)
id=8    @10.100.1.11  (mysql-5.7.29-ndb-7.5.17, Nodegroup: 1)

[ndb_mgmd(MGM)] 1 node(s)
id=50   @10.100.1.8  (mysql-5.7.29-ndb-7.5.17)

[mysqld(API)]   2 node(s)
id=100  @10.100.1.8  (5.7.29-ndb-7.5.17)
id=101  @10.100.1.10  (5.7.29-ndb-7.5.17)

mgm#1: PROMPT
ndb_mgm> EXIT
jon@valhaj:/usr/local/mysql/bin>

For additional information and examples, see Section 21.5.2, “Commands in the NDB Cluster Management Client”.

Increased FIXED column storage per fragment.  NDB Cluster 7.5 and later supports a maximum of 128 TB per fragment of data in FIXED columns. In NDB Cluster 7.4 and earlier, this was 16 GB per fragment.

Deprecated parameters removed.  The following NDB Cluster data node configuration parameters were deprecated in previous releases of NDB Cluster, and were removed in NDB 7.5.0:

The archaic and unused (and for this reason also previously undocumented) ByteOrder computer configuration parameter was also removed in NDB 7.5.0.

The parameters just described are not supported in NDB 7.5. Attempting to use any of these parameters in an NDB Cluster configuration file now results in an error.

DBTC scan enhancements.  Scans have been improved by reducing the number of signals used for communication between the DBTC and DBDIH kernel blocks in NDB, enabling higher scalability of data nodes when used for scan operations by decreasing the use of CPU resources for scan operations, in some cases by an estimated five percent.

Also as result of these changes response times should be greatly improved, which could help prevent issues with overload of the main threads. In addition, scans made in the BACKUP kernel block have also been improved and made more efficient than in previous releases.

JSON column support.  NDB 7.5.2 and later supports the JSON column type for NDB tables and the JSON functions found in the MySQL Server, subject to the limitation that an NDB table can have at most 3 JSON columns.

Read from any replica; specify number of hashmap partition fragments.  Previously, all reads were directed towards the primary replica except for simple reads. (A simple read is a read that locks the row while reading it.) Beginning with NDB 7.5.2, it is possible to enable reads from any replica. This is disabled by default but can be enabled for a given SQL node using the ndb_read_backup system variable added in this release.

Previously, it was possible to define tables with only one type of partition mapping, with one primary partition on each LDM in each node, but in NDB 7.5.2 it becomes possible to be more flexible about the assignment of partitions by setting a partition balance (fragment count type). Possible balance schemes are one per node, one per node group, one per LDM per node, and one per LDM per node group.

This setting can be controlled for individual tables by means of a PARTITION_BALANCE option (renamed from FRAGMENT_COUNT_TYPE in NDB 7.5.4) embedded in NDB_TABLE comments in CREATE TABLE or ALTER TABLE statements. Settings for table-level READ_BACKUP are also supported using this syntax. For more information and examples, see Section 13.1.18.10, “Setting NDB_TABLE Options”.

In NDB API applications, a table's partition balance can also be get and set using methods supplied for this purpose; see Table::getPartitionBalance(), and Table::setPartitionBalance(), as well as Object::PartitionBalance, for more information about these.

As part of this work, NDB 7.5.2 also introduces the ndb_data_node_neighbour system variable. This is intended for use, in transaction hinting, to provide a “nearby” data node to this SQL node.

In addition, when restoring table schemas, ndb_restore --restore-meta now uses the target cluster's default partitioning, rather than using the same number of partitions as the original cluster from which the backup was taken. See Section 21.4.24.1.2, “Restoring to More Nodes Than the Original”, for more information and an example.

NDB 7.5.3 adds a further enhancement to READ_BACKUP: In this and later versions, it is possible to set READ_BACKUP for a given table online as part of ALTER TABLE ... ALGORITHM=INPLACE ....

ThreadConfig improvements.  A number of enhancements and feature additions are implemented in NDB 7.5.2 for the ThreadConfig multithreaded data node (ndbmtd) configuration parameter, including support for an increased number of platforms. These changes are described in the next few paragraphs.

Non-exclusive CPU locking is now supported on FreeBSD and Windows, using cpubind and cpuset. Exclusive CPU locking is now supported on Solaris (only) using the cpubind_exclusive and cpuset_exclusive parameters which are introduced in this release.

Thread prioritzation is now available, controlled by the new thread_prio parameter. thread_prio is supported on Linux, FreeBSD, Windows, and Solaris, and varies somewhat by platform. For more information, see the description of ThreadConfig.

The realtime parameter is now supported on Windows platforms.

Partitions larger than 16 GB.  Due to an improvement in the hash index implementation used by NDB Cluster data nodes, partitions of NDB tables may now contain more than 16 GB of data for fixed columns, and the maximum partition size for fixed columns is now raised to 128 TB. The previous limitation was due to the fact that the DBACC block in the NDB kernel used only 32-bit references to the fixed-size part of a row in the DBTUP block, although 45-bit references to this data are used in DBTUP itself and elsewhere in the kernel outside DBACC; all such references in to the data handled in the DBACC block now use 45 bits instead.

Print SQL statements from ndb_restore.  NDB 7.5.4 adds the --print-sql-log option for the ndb_restore utility provided with the NDB Cluster distribution. This option enables SQL logging to stdout. Important: Every table to be restored using this option must have an explicitly defined primary key.

See Section 21.4.24, “ndb_restore — Restore an NDB Cluster Backup”, for more information.

Organization of RPM packages.  Beginning with NDB 7.5.4, the naming and organization of RPM packages provided for NDB Cluster align more closely with those released for the MySQL server. The names of all NDB Cluster RPMs are now prefixed with mysql-cluster. Data nodes are now installed using the data-node package; management nodes are now installed from the management-server package; and SQL nodes require the server and common packages. MySQL and NDB client programs, including the mysql client and the ndb_mgm management client, are now included in the client RPM.

For a detailed listing of NDB Cluster RPMs and other information, see Section 21.2.3.2, “Installing NDB Cluster from RPM”.

ndbinfo processes and config_nodes tables.  NDB 7.5.7 adds two tables to the ndbinfo information database to provide information about cluster nodes; these tables are listed here:

System name.  The system name of an NDB cluster can be used to identify a specific cluster. Beginning with NDB 7.5.7, the MySQL Server shows this name as the value of the Ndb_system_name status variable; NDB API applications can use the Ndb_cluster_connection::get_system_name() method which is added in the same release.

A system name based on the time the management server was started is generated automatically; you can override this value by adding a [system] section to the cluster's configuration file and setting the Name parameter to a value of your choice in this section, prior to starting the management server.

ndb_restore options.  Beginning with NDB 7.5.13, the --nodeid and --backupid options are both required when invoking ndb_restore.

ndb_blob_tool enhancements.  Beginning with NDB 7.5.18, the ndb_blob_tool utility can detect missing blob parts for which inline parts exist and replace these with placeholder blob parts (consisting of space characters) of the correct length. To check whether there are missing blob parts, use the --check-missing option with this program. To replace any missing blob parts with placeholders, use the --add-missing option.

For more information, see Section 21.4.6, “ndb_blob_tool — Check and Repair BLOB and TEXT columns of NDB Cluster Tables”.

New Disk Data table file format.  A new file format was introduced in NDB 7.6.2 for NDB Disk Data tables, which makes it possible for each Disk Data table to be uniquely identified without reusing any table IDs. The format was improved further in NDB 7.6.4. This should help resolve issues with page and extent handling that were visible to the user as problems with rapid creating and dropping of Disk Data tables, and for which the old format did not provide a ready means to fix.

The new format is now used whenever new undo log file groups and tablespace data files are created. Files relating to existing Disk Data tables continue to use the old format until their tablespaces and undo log file groups are re-created.

To avoid problems relating to the changes in format, you should re-create any existing tablespaces and undo log file groups when upgrading to NDB 7.6.2 or NDB 7.6.4. You can do this by performing an initial restart of each data node (that is, using the --initial option) as part of the upgrade process. You can expect this step to be made mandatory as part of upgrading from NDB 7.5 or an earlier release series to NDB 7.6 or later.

If you are using Disk Data tables, a downgrade from any NDB 7.6 release—without regard to release status—to any NDB 7.5 or earlier release requires that you restart all data nodes with --initial as part of the downgrade process. This is because NDB 7.5 and earlier release series are not able to read the new Disk Data file format.

For more information, see Section 21.2.9, “Upgrading and Downgrading NDB Cluster”.

Data memory pooling and dynamic index memory.  Memory required for indexes on NDB table columns is now allocated dynamically from that allocated for DataMemory. For this reason, the IndexMemory configuration parameter is now deprecated, and subject to removal in a future release series.

The pooling together of index memory with data memory simplifies the configuration of NDB; a further benefit of these changes is that scaling up by increasing the number of LDM threads is no longer limited by having set an insufficiently large value for IndexMemory.This is because index memory is no longer a static quantity which is allocated only once (when the cluster starts), but can now be allocated and deallocated as required. Previously, it was sometimes the case that increasing the number of LDM threads could lead to index memory exhaustion while large amounts of DataMemory remained available.

As part of this work, a number of instances of DataMemory usage not directly related to storage of table data now use transaction memory instead.

For this reason, it may be necessary on some systems to increase SharedGlobalMemory to allow transaction memory to increase when needed, such as when using NDB Cluster Replication, which requires a great deal of buffering on the data nodes. On systems performing initial bulk loads of data, it may be necessary to break up very large transactions into smaller parts.

In addition, data nodes now generate MemoryUsage events (see Section 21.5.6.2, “NDB Cluster Log Events”) and write appropriate messages in the cluster log when resource usage reaches 99%, as well as when it reaches 80%, 90%, or 100%, as before.

Other related changes are listed here:

ndbinfo processes and config_nodes tables.  NDB 7.6.2 adds two tables to the ndbinfo information database to provide information about cluster nodes; these tables are listed here:

System name.  The system name of an NDB cluster can be used to identify a specific cluster. Beginning with NDB 7.6.2, the MySQL Server shows this name as the value of the Ndb_system_name status variable; NDB API applications can use the Ndb_cluster_connection::get_system_name() method which is added in the same release.

A system name based on the time the management server was started is generated automatically>; you can override this value by adding a [system] section to the cluster's configuration file and setting the Name parameter to a value of your choice in this section, prior to starting the management server.

Improved GUI installer.  The NDB Cluster Auto-Installer has been enhanced in a number of respects, as described in the following list:

For more details and usage information, see Section 21.2.2, “The NDB Cluster Auto-Installer (NDB 7.6)”.

ndb_import CSV import tool.  ndb_import, added in NDB Cluster 7.6.2, loads CSV-formatted data directly into an NDB table using the NDB API (a MySQL server is needed only to create the table and database in which it is located). ndb_import can be regarded as an analog of mysqlimport or the LOAD DATA SQL statement, and supports many of the same or similar options for formatting of the data.

Assuming that the database and target NDB table exist, ndb_import needs only a connection to the cluster's management server (ndb_mgmd) to perform the importation; for this reason, there must be an [api] slot available to the tool in the cluster's config.ini file purpose.

See Section 21.4.14, “ndb_import — Import CSV Data Into NDB”, for more information.

ndb_top monitoring tool.  Added the ndb_top utility, which shows CPU load and usage information for an NDB data node in real time. This information can be displayed in text format, as an ASCII graph, or both. The graph can be shown in color, or using grayscale.

ndb_top connects to an NDB Cluster SQL node (that is, a MySQL Server). For this reason, the program must be able to connect as a MySQL user having the SELECT privilege on tables in the ndbinfo database.

ndb_top is available for Linux, Solaris, and macOS platforms beginning with NDB 7.6.3. It is not currently available for Windows platforms.

For more information, see Section 21.4.30, “ndb_top — View CPU usage information for NDB threads”.

Code cleanup.  A significant number of debugging statements and printouts not necessary for normal operations have been moved into code used only when testing or debugging NDB, or dispensed with altogether. This removal of overhead should result in a noticeable improvement in the performance of LDM and TC threads on the order of 10% in many cases.

LDM thread and LCP improvements.  Previously, when a local data management thread experienced I/O lag, it wrote to local checkpoints more slowly. This could happen, for example, during a disk overload condition. Problems could occur because other LDM threads did not always observe this state, or do likewise. NDB now tracks I/O lag mode globally, so that this state is reported as soon as at least one thread is writing in I/O lag mode; it then makes sure that the reduced write speed for this LCP is enforced for all LDM threads for the duration of the slowdown condition. Because the reduction in write speed is now observed by other LDM instances, overall capacity is increased; this enables the disk overload (or other condition inducing I/O lag) to be overcome more quickly in such cases than it was previously.

NDB error identification.  Error messages and information can be obtained using the mysql client in NDB 7.6.4 and later from a new error_messages table in the ndbinfo information database. In addition, the 7.6.4 release introduces a command-line client ndb_perror for obtaining information from NDB error codes; this replaces using perror with --ndb, which is now deprecated and subject to removal in a future release.

For more information, see Section 21.5.10.21, “The ndbinfo error_messages Table”, and Section 21.4.17, “ndb_perror — Obtain NDB Error Message Information”.

SPJ improvements.  When executing a scan as a pushed join (that is, the root of the query is a scan), the DBTC block sends an SPJ request to a DBSPJ instance on the same node as the fragment to be scanned. Formerly, one such request was sent for each of the node's fragments. As the number of DBTC and DBSPJ instances is normally set less than the number of LDM instances, this means that all SPJ instances were involved in the execution of a single query, and, in fact, some SPJ instances could (and did) receive multiple requests from the same query. In NDB 7.6.4, it becomes possible for a single SPJ request to handle a set of root fragments to be scanned, so that only a single SPJ request (SCAN_FRAGREQ) needs to be sent to any given SPJ instance (DBSPJ block) on each node.

Since DBSPJ consumes a relatively small amount of the total CPU used when evaluating a pushed join, unlike the LDM block (which is repsonsible for the majority of the CPU usage), introducing multiple SPJ blocks adds some parallelism, but the additional overhead also increases. By enabling a single SPJ request to handle a set of root fragments to be scanned, such that only a single SPJ request is sent to each DBSPJ instance on each node and batch sizes are allocated per fragment, the multi-fragment scan can obtain a larger total batch size, allowing for some scheduling optimizations to be done within the SPJ block, which can scan a single fragment at a time (giving it the total batch size allocation), scan all fragments in parallel using smaller sub-batches, or some combination of the two.

This work is expected to increase performance of pushed-down joins for the following reasons:

Improved O_DIRECT handling for redo logs.  NDB 7.6.4 implements a new data node configuration parameter ODirectSyncFlag which causes completed redo log writes using O_DIRECT to be handled as fsync calls. ODirectSyncFlag is disabled by default; to enable it, set it to true.

You should bear in mind that the setting for this parameter is ignored when at least one of the following conditions is true:

Locking of CPUs to offline index build threads.  In NDB 7.6.4 and later, offline index builds by default use all cores available to ndbmtd, instead of being limited to the single core reserved for the I/O thread. It also becomes possible to specify a desired set of cores to be used for I/O threads performing offline multithreaded builds of ordered indexes. This can improve restart and restore times and performance, as well as availability.

This improvement involves several related changes. The first of these is to change the default value for the BuildIndexThreads configuration parameter (from 0 to 128), means that offline ordered index builds are now multithreaded by default. The default value for the TwoPassInitialNodeRestartCopy is also changed (from false to true), so that an initial node restart first copies all data without any creation of indexes from a “live” node to the node which is being started, builds the ordered indexes offline after the data has been copied, then again synchronizes with the live node; this can significantly reduce the time required for building indexes. In addition, to facilitate explicit locking of offline index build threads to specific CPUs, a new thread type (idxbld) is defined for the ThreadConfig configuration parameter.

As part of this work, NDB can now distinguish between execution thread types and other types of threads, and between types of threads which are permanently assigned to specific tasks, and those whose assignments are merely temporary.

NDB 7.6.4 also introduces the nosend parameter for ThreadCOnfig. By setting this to 1, you can keep a main, ldm, rep, or tc thread from assisting the send threads. This parameter is 0 by default, and cannot be used with I/O threads, send threads, index build threads, or watchdog threads.

For additonal information, see the descriptions of the parameters.

Variable batch sizes for DDL bulk data operations.  As part of work ongoing to optimize bulk DDL performance by ndbmtd, it is now possible to obtain performance improvements by increasing the batch size for the bulk data parts of DDL operations processing data using scans. Batch sizes are now made configurable for unique index builds, foreign key builds, and online reorganization, by setting the respective data node configuration parameters listed here:

For each of the parameters just listed, the default value is 64, the minimum is 16, and the maximum is 512.

Increasing the appropriate batch size or sizes can help amortize inter-thread and inter-node latencies and make use of more parallel resources (local and remote) to help scale DDL performance. In each case there can be a tradeoff with ongoing traffic.

Partial LCPs.  NDB 7.6.4 implements partial local checkpoints. Formerly, an LCP always made a copy of the entire database. When working with terabytes of data this process could require a great deal of time, with an adverse impact on node and cluster restarts especially, as well as more space for the redo logs. It is now no longer strictly necessary for LCPs to do this—instead, an LCP now by default saves only a number of records that is based on the quantity of data changed since the previous LCP. This can vary between a full checkpoint and a checkpoint that changes nothing at all. In the event that the checkpoint reflects any changes, the minimum is to write one part of the 2048 making up a local LCP.

As part of this change, two new data node configuration parameters are inroduced in this release: EnablePartialLcp (default true, or enabled) enables partial LCPs. RecoveryWork controls the percentage of space given over to LCPs; it increases with the amount of work which must be performed on LCPs during restarts as opposed to that performed during normal operations. Raising this value causes LCPs during normal operations to require writing fewer records and so decreases the usual workload. Raising this value also means that restarts can take longer.

You must disable partial LCPs explicitly by setting EnablePartialLcp=false. This uses the least amount of disk, but also tends to maximize the write load for LCPs. To optimize for the lowest workload on LCPs during normal operation, use EnablePartialLcp=true and RecoveryWork=100. To use the least disk space for partial LCPs, but with bounded writes, use EnablePartialLcp=true and RecoveryWork=25, which is the minimum for RecoveryWork. The default is EnablePartialLcp=true with RecoveryWork=50, which means LCP files require approximately 1.5 times DataMemory; using CompressedLcp=1, this can be further reduced by half. Recovery times using the default settings should also be much faster than when EnablePartialLcp is set to false.

In addition the data node configuration parameters BackupDataBufferSize, BackupWriteSize, and BackupMaxWriteSize are all now deprecated, and subject to removal in a future release of MySQL NDB Cluster.

As part of this enhancement, work has been done to correct several issues with node restarts wherein it was possible to run out of undo log in various situations, most often when restoring a node that had been down for a long time during a period of intensive write activity.

Additional work was done to improve data node survival of long periods of synchronization without timing out, by updating the LCP watchdog during this process, and keeping better track of the progress of disk data synchronization. Previously, there was the possibility of spurious warnings or even node failures if synchronization took longer than the LCP watchdog timeout.

Parallel undo log record processing.  Formerly, the data node LGMAN kernel block processed undo log records serially; now this is done in parallel. The rep thread, which hands off undo records to LDM threads, waited for an LDM to finish applying a record before fetching the next one; now the rep thread no longer waits, but proceeds immediately to the next record and LDM.

A count of the number of outstanding log records for each LDM in LGMAN is kept, and decremented whenever an LDM has completed the execution of a record. All the records belonging to a page are sent to the same LDM thread but are not guaranteed to be processed in order, so a hash map of pages that have outstanding records maintains a queue for each of these pages. When the page is available in the page cache, all records pending in the queue are applied in order.

A few types of records continue to be processed serially: UNDO_LCP, UNDO_LCP_FIRST, UNDO_LOCAL_LCP, UNDO_LOCAL_LCP_FIRST, UNDO_DROP, and UNDO_END.

There are no user-visible changes in functionality directly associated with this performance enhancement; it is part of the work being done to improve undo long handling in support of partial local checkpoints in NDB Cluster 7.6.4.

Reading table and fragment IDs from extent for undo log applier.  When applying an undo log, it is necessary to obtain the table ID and fragment ID from the page ID. This was done previously by reading the page from the PGMAN kernel block using an extra PGMAN worker thread, but when applying the undo log it was necessary to read the page again.

when using O_DIRECT this was very inefficient since the page was not cached in the OS kernel. To correct this issue, mapping from page ID to table ID and fragment ID is now done using information from the extent header the table IDs and fragment IDs for the pages used within a given extent. The extent pages are always present in the page cache, so no extra reads from disk are required for performing the mapping. In addition, the information can already be read, using existing TSMAN kernel block data structures.

See the description of the ODirect data node configuration parameter, for more information.

NDB Cluster Auto-Installer improvements.  In NDB 7.6.4, node configuration parameters, their default values, and their documentation as found in the Auto-Installer have been better aligned with those found in NDB Cluster software releases. SSH support and configuration have also been improved. In addition, HTTPS is now used by default for Web connections, and cookies are no longer employed as a persistent data storage mechanism. More information about these and other changes in the Auto-Installer is given in the next several paragraphs.

The Auto-Installer now implements a mechanism for setting configuration parameters that take discrete values. For example, the data node parameter Arbitration must now be set to one of its allowed values Default, Disabled, or WaitExternal.

The Auto-Installer also now gets and shows the amount of disk space available per host to the cluster (as DiskFree), using this information to obtain realistic values for configuration parameters that depend on it.

Secure connection support in the MySQL NDB Cluster Auto-Installer has been updated or improved in NDB Cluster 7.6.4 as follows:

Other enhancements relating to data security that are implemented in NDB 7.6.4 include the following:

See Section 21.2.1, “The NDB Cluster Auto-Installer (NDB 7.5)”, for more information.

Shared memory transporter.  User-defined shared memory (SHM) connections between a data node and an API node on the same host computer are supported in NDB 7.6.6 and later, and are no longer considered experimental. You can enable an explicit shared memory connection by setting the UseShm configuration parameter to 1 for the relevant data node. When explicitly defining shared memory as the connection method, it is also necessary that both the data node and the API node are identified by HostName.

Performance of SHM connections can be enhanced through setting parameters such as ShmSize, ShmSpintime, and SendBufferMemory in an [shm] or [shm default] section of the cluster configuration file (config.ini). Configuration of SHM is otherwise similar to that of the TCP transporter.

The SigNum parameter is not used in the new SHM implementation, and any settings made for it are now ignored. Section 21.3.3.12, “NDB Cluster Shared Memory Connections”, provides more information about these parameters. In addition, as part of this work, NDB code relating to the old SCI transporter has been removed.

For more information, see Section 21.3.3.12, “NDB Cluster Shared Memory Connections”.

SPJ block inner join optimization.  In NDB 7.6.6 and later, the SPJ kernel block can take into account when it is evaluating a join request in which at least some of the tables are INNER-joined. This means that it can eliminate requests for row, ranges, or both as soon as it becomes known that one or more of the preceding requests did not return any results for a parent row. This saves both the data nodes and the SPJ block from having to handle requests and result rows which never take part in an INNER-joined result row.

Consider this join query, where pk is the primary key on tables t2, t3, and t4, and columns x, y, and z are nonindexed columns:

SELECT * FROM t1
  JOIN t2 ON t2.pk = t1.x
  JOIN t3 ON t3.pk = t1.y
  JOIN t4 ON t4.pk = t1.z;

Previously, this resulted in an SPJ request including a scan on table t1, and lookups on each of the tables t2, t3, and t4; these were evaluated for every row returned from t1. For these, SPJ created LQHKEYREQ requests for tables t2, t3, and t4. Now SPJ takes into consideration the requirement that, to produce any result rows, an inner join must find a match in all tables joined; as soon as no matches are found for one of the tables, any further requests to tables having the same parent or tables are now skipped.

NDB wakeup thread.  NDB uses a poll receiver to read from sockets, to execute messages from the sockets, and to wake up other threads. When making only intermittent use of a receive thread, poll ownership is given up before starting to wake up other threads, which provides some degree of parallelism in the receive thread, but, when making constant use of the receive thread, the thread can be overburdened by tasks including wakeup of other threads.

NDB 7.6.6 and later supports offloading by the receiver thread of the task of waking up other threads to a new thread that wakes up other threads on request (and otherwise simply sleeps), making it possible to improve the capacity of a single cluster connection by roughly ten to twenty percent.

Adaptive LCP control.  NDB 7.6.7 implements an adaptive LCP control mechanism which acts in response to changes in redo log space usage. By controlling LCP disk write speed, you can help protect against a number of resource-related issues, including the following:

This work includes the following changes relating to NDB configuration parameters:

This work implements control of LCP speed chiefly to minimize the risk of running out of redo log. This is done in adapative fashion, based on the amount of redo log space used, using the alert levels, with the responses taken when these levels are attained, shown here:

Raising the level also has the effect of increasing the calculated target checkpoint speed.

LCP control has the following benefits for NDB installations:

ndb_restore options.  Beginning with NDB 7.6.9, the --nodeid and --backupid options are both required when invoking ndb_restore.

Restoring by slices.  Beginning with NDB 7.6.13, it is possible to divide a backup into roughly equal portions (slices) and to restore these slices in parallel using two new options implemented for ndb_restore:

This makes it possible to employ multiple instances of ndb_restore to restore subsets of the backup in parallel, potentially reducing the amount of time required to perform the restore operation.

For more information, see the description of the ndb_restore --num-slices option.

ndb_blob_tool enhancements.  Beginning with NDB 7.6.14, the ndb_blob_tool utility can detect missing blob parts for which inline parts exist and replace these with placeholder blob parts (consisting of space characters) of the correct length. To check whether there are missing blob parts, use the --check-missing option with this program. To replace any missing blob parts with placeholders, use the --add-missing option.

For more information, see Section 21.4.6, “ndb_blob_tool — Check and Repair BLOB and TEXT columns of NDB Cluster Tables”.

Node Configuration Parameters Introduced in NDB 7.5

Node Configuration Parameters Deprecated in NDB 7.5

Node Configuration Parameters Removed in NDB 7.5

MySQL Server Options and Variables Introduced in NDB 7.5

MySQL Server Options and Variables Deprecated in NDB 7.5

MySQL Server Options and Variables Removed in NDB 7.5

Node Configuration Parameters Introduced in NDB 7.6

Node Configuration Parameters Deprecated in NDB 7.6

Node Configuration Parameters Removed in NDB 7.6

MySQL Server Options and Variables Introduced in NDB 7.6

MySQL Server Options and Variables Deprecated in NDB 7.6

MySQL Server Options and Variables Removed in NDB 7.6

Temporary tables.  Temporary tables are not supported. Trying either to create a temporary table that uses the NDB storage engine or to alter an existing temporary table to use NDB fails with the error Table storage engine 'ndbcluster' does not support the create option 'TEMPORARY'.

Indexes and keys in NDB tables.  Keys and indexes on NDB Cluster tables are subject to the following limitations:

Restrictions on foreign keys.  Support for foreign key constraints in NDB 7.5 is comparable to that provided by InnoDB, subject to the following restrictions:

For more information, see Section 13.1.18.6, “FOREIGN KEY Constraints”, and Section 1.8.3.2, “FOREIGN KEY Constraints”.

NDB Cluster and geometry data types.  Geometry data types (WKT and WKB) are supported for NDB tables. However, spatial indexes are not supported.

Character sets and binary log files.  Currently, the ndb_apply_status and ndb_binlog_index tables are created using the latin1 (ASCII) character set. Because names of binary logs are recorded in this table, binary log files named using non-Latin characters are not referenced correctly in these tables. This is a known issue, which we are working to fix. (Bug #50226)

To work around this problem, use only Latin-1 characters when naming binary log files or setting any the --basedir, --log-bin, or --log-bin-index options.

Creating NDB tables with user-defined partitioning.  Support for user-defined partitioning in NDB Cluster is restricted to [LINEAR] KEY partitioning. Using any other partitioning type with ENGINE=NDB or ENGINE=NDBCLUSTER in a CREATE TABLE statement results in an error.

It is possible to override this restriction, but doing so is not supported for use in production settings. For details, see User-defined partitioning and the NDB storage engine (NDB Cluster).

Default partitioning scheme.  All NDB Cluster tables are by default partitioned by KEY using the table's primary key as the partitioning key. If no primary key is explicitly set for the table, the “hidden” primary key automatically created by the NDB storage engine is used instead. For additional discussion of these and related issues, see Section 22.2.5, “KEY Partitioning”.

CREATE TABLE and ALTER TABLE statements that would cause a user-partitioned NDBCLUSTER table not to meet either or both of the following two requirements are not permitted, and fail with an error:

Exception.  If a user-partitioned NDBCLUSTER table is created using an empty column-list (that is, using PARTITION BY [LINEAR] KEY()), then no explicit primary key is required.

Maximum number of partitions for NDBCLUSTER tables.  The maximum number of partitions that can defined for a NDBCLUSTER table when employing user-defined partitioning is 8 per node group. (See Section 21.1.2, “NDB Cluster Nodes, Node Groups, Replicas, and Partitions”, for more information about NDB Cluster node groups.

DROP PARTITION not supported.  It is not possible to drop partitions from NDB tables using ALTER TABLE ... DROP PARTITION. The other partitioning extensions to ALTER TABLE—ADD PARTITION, REORGANIZE PARTITION, and COALESCE PARTITION—are supported for NDB tables, but use copying and so are not optimized. See Section 22.3.1, “Management of RANGE and LIST Partitions” and Section 13.1.8, “ALTER TABLE Statement”.

Row-based replication.  When using row-based replication with NDB Cluster, binary logging cannot be disabled. That is, the NDB storage engine ignores the value of sql_log_bin.

JSON data type.  The MySQL JSON data type is supported for NDB tables in the mysqld supplied with NDB 7.5.2 and later.

An NDB table can have a maximum of 3 JSON columns.

The NDB API has no special provision for working with JSON data, which it views simply as BLOB data. Handling data as JSON must be performed by the application.

CPU and thread info ndbinfo tables.  NDB 7.5.2 adds several new tables to the ndbinfo information database providing information about CPU and thread activity by node, thread ID, and thread type. The tables are listed here:

For more information about these tables, see Section 21.5.10, “ndbinfo: The NDB Cluster Information Database”.

Lock info ndbinfo tables.  NDB 7.5.3 adds new tables to the ndbinfo information database providing information about locks and lock attempts in a running NDB Cluster. These tables are listed here:

A DELETE statement on an NDB table makes the memory formerly used by the deleted rows available for re-use by inserts on the same table only. However, this memory can be made available for general re-use by performing OPTIMIZE TABLE.

A rolling restart of the cluster also frees any memory used by deleted rows. See Section 21.5.5, “Performing a Rolling Restart of an NDB Cluster”.

A DROP TABLE or TRUNCATE TABLE operation on an NDB table frees the memory that was used by this table for re-use by any NDB table, either by the same table or by another NDB table.

Limits imposed by the cluster's configuration.  A number of hard limits exist which are configurable, but available main memory in the cluster sets limits. See the complete list of configuration parameters in Section 21.3.3, “NDB Cluster Configuration Files”. Most configuration parameters can be upgraded online. These hard limits include:

Node and data object maximums.  The following limits apply to numbers of cluster nodes and metadata objects:

See Previous NDB Cluster Issues Resolved in NDB Cluster 7.3, for more information.

Transaction isolation level.  The NDBCLUSTER storage engine supports only the READ COMMITTED transaction isolation level. (InnoDB, for example, supports READ COMMITTED, READ UNCOMMITTED, REPEATABLE READ, and SERIALIZABLE.) You should keep in mind that NDB implements READ COMMITTED on a per-row basis; when a read request arrives at the data node storing the row, what is returned is the last committed version of the row at that time.

Uncommitted data is never returned, but when a transaction modifying a number of rows commits concurrently with a transaction reading the same rows, the transaction performing the read can observe “before” values, “after” values, or both, for different rows among these, due to the fact that a given row read request can be processed either before or after the commit of the other transaction.

To ensure that a given transaction reads only before or after values, you can impose row locks using SELECT ... LOCK IN SHARE MODE. In such cases, the lock is held until the owning transaction is committed. Using row locks can also cause the following issues:

NDB uses READ COMMITTED for all reads unless a modifier such as LOCK IN SHARE MODE or FOR UPDATE is used. LOCK IN SHARE MODE causes shared row locks to be used; FOR UPDATE causes exclusive row locks to be used. Unique key reads have their locks upgraded automatically by NDB to ensure a self-consistent read; BLOB reads also employ extra locking for consistency.

See Section 21.5.3.4, “NDB Cluster Backup Troubleshooting”, for information on how NDB Cluster's implementation of transaction isolation level can affect backup and restoration of NDB databases.

Transactions and BLOB or TEXT columns.  NDBCLUSTER stores only part of a column value that uses any of MySQL's BLOB or TEXT data types in the table visible to MySQL; the remainder of the BLOB or TEXT is stored in a separate internal table that is not accessible to MySQL. This gives rise to two related issues of which you should be aware whenever executing SELECT statements on tables that contain columns of these types:

Unique key lookups and transaction isolation.  Unique indexes are implemented in NDB using a hidden index table which is maintained internally. When a user-created NDB table is accessed using a unique index, the hidden index table is first read to find the primary key that is then used to read the user-created table. To avoid modification of the index during this double-read operation, the row found in the hidden index table is locked. When a row referenced by a unique index in the user-created NDB table is updated, the hidden index table is subject to an exclusive lock by the transaction in which the update is performed. This means that any read operation on the same (user-created) NDB table must wait for the update to complete. This is true even when the transaction level of the read operation is READ COMMITTED.

One workaround which can be used to bypass potentially blocking reads is to force the SQL node to ignore the unique index when performing the read. This can be done by using the IGNORE INDEX index hint as part of the SELECT statement reading the table (see Section 8.9.4, “Index Hints”). Because the MySQL server creates a shadowing ordered index for every unique index created in NDB, this lets the ordered index be read instead, and avoids unique index access locking. The resulting read is as consistent as a committed read by primary key, returning the last committed value at the time the row is read.

Reading via an ordered index makes less efficient use of resources in the cluster, and may have higher latency.

It is also possible to avoid using the unique index for access by querying for ranges rather than for unique values.

Rollbacks.  There are no partial transactions, and no partial rollbacks of transactions. A duplicate key or similar error causes the entire transaction to be rolled back.

This behavior differs from that of other transactional storage engines such as InnoDB that may roll back individual statements.

Transactions and memory usage.  As noted elsewhere in this chapter, NDB Cluster does not handle large transactions well; it is better to perform a number of small transactions with a few operations each than to attempt a single large transaction containing a great many operations. Among other considerations, large transactions require very large amounts of memory. Because of this, the transactional behavior of a number of MySQL statements is affected as described in the following list:

Transactions and the COUNT() function.  When using NDB Cluster Replication, it is not possible to guarantee the transactional consistency of the COUNT() function on the slave. In other words, when performing on the master a series of statements (INSERT, DELETE, or both) that changes the number of rows in a table within a single transaction, executing SELECT COUNT(*) FROM table queries on the slave may yield intermediate results. This is due to the fact that SELECT COUNT(...) may perform dirty reads, and is not a bug in the NDB storage engine. (See Bug #31321 for more information.)

Temporary errors.  When first starting a node, it is possible that you may see Error 1204 Temporary failure, distribution changed and similar temporary errors.

Errors due to node failure.  The stopping or failure of any data node can result in a number of different node failure errors. (However, there should be no aborted transactions when performing a planned shutdown of the cluster.)

Database and table names.  When using the NDB storage engine, the maximum allowed length both for database names and for table names is 63 characters. A statement using a database name or table name longer than this limit fails with an appropriate error.

Number of database objects.  The maximum number of all NDB database objects in a single NDB Cluster—including databases, tables, and indexes—is limited to 20320.

Attributes per table.  The maximum number of attributes (that is, columns and indexes) that can belong to a given table is 512.

Attributes per key.  The maximum number of attributes per key is 32.

Row size.  The maximum permitted size of any one row is 14000 bytes.

Each BLOB or TEXT column contributes 256 + 8 = 264 bytes to this total; this includes JSON columns. See String Type Storage Requirements, as well as JSON Storage Requirements, for more information relating to these types.

In addition, the maximum offset for a fixed-width column of an NDB table is 8188 bytes; attempting to create a table that violates this limitation fails with NDB error 851 Maximum offset for fixed-size columns exceeded. For memory-based columns, you can work around this limitation by using a variable-width column type such as VARCHAR or defining the column as COLUMN_FORMAT=DYNAMIC; this does not work with columns stored on disk. For disk-based columns, you may be able to do so by reordering one or more of the table's disk-based columns such that the combined width of all but the disk-based column defined last in the CREATE TABLE statement used to create the table does not exceed 8188 bytes, less any possible rounding performed for some data types such as CHAR or VARCHAR; otherwise it is necessary to use memory-based storage for one or more of the offending column or columns instead.

BIT column storage per table.  The maximum combined width for all BIT columns used in a given NDB table is 4096.

FIXED column storage.  NDB Cluster 7.5 and later supports a maximum of 128 TB per fragment of data in FIXED columns. (Previously, this was 16 GB.)

Index prefixes.  Prefixes on indexes are not supported for NDB tables. If a prefix is used as part of an index specification in a statement such as CREATE TABLE, ALTER TABLE, or CREATE INDEX, the prefix is not created by NDB.

A statement containing an index prefix, and creating or modifying an NDB table, must still be syntactically valid. For example, the following statement always fails with Error 1089 Incorrect prefix key; the used key part isn't a string, the used length is longer than the key part, or the storage engine doesn't support unique prefix keys, regardless of storage engine:

CREATE TABLE t1 (
    c1 INT NOT NULL,
    c2 VARCHAR(100),
    INDEX i1 (c2(500))
);

This happens on account of the SQL syntax rule that no index may have a prefix larger than itself.

Savepoints and rollbacks.  Savepoints and rollbacks to savepoints are ignored as in MyISAM.

Durability of commits.  There are no durable commits on disk. Commits are replicated, but there is no guarantee that logs are flushed to disk on commit.

Replication.  Statement-based replication is not supported. Use --binlog-format=ROW (or --binlog-format=MIXED) when setting up cluster replication. See Section 21.6, “NDB Cluster Replication”, for more information.

Replication using global transaction identifiers (GTIDs) is not compatible with NDB Cluster, and is not supported in NDB Cluster 7.5 or NDB CLuster 7.6. Do not enable GTIDs when using the NDB storage engine, as this is very likely to cause problems up to and including failure of NDB Cluster Replication.

Semisynchronous replication is not supported in NDB Cluster.

Generated columns.  The NDB storage engine does not support indexes on virtual generated columns.

As with other storage engines, you can create an index on a stored generated column, but you should bear in mind that NDB uses DataMemory for storage of the generated column as well as IndexMemory for the index. See JSON columns and indirect indexing in NDB Cluster, for an example.

NDB Cluster writes changes in stored generated columns to the binary log, but does log not those made to virtual columns. This should not effect NDB Cluster Replication or replication between NDB and other MySQL storage engines.

Range scans.  There are query performance issues due to sequential access to the NDB storage engine; it is also relatively more expensive to do many range scans than it is with either MyISAM or InnoDB.

Reliability of Records in range.  The Records in range statistic is available but is not completely tested or officially supported. This may result in nonoptimal query plans in some cases. If necessary, you can employ USE INDEX or FORCE INDEX to alter the execution plan. See Section 8.9.4, “Index Hints”, for more information on how to do this.

Unique hash indexes.  Unique hash indexes created with USING HASH cannot be used for accessing a table if NULL is given as part of the key.

Machine architecture.  All machines used in the cluster must have the same architecture. That is, all machines hosting nodes must be either big-endian or little-endian, and you cannot use a mixture of both. For example, you cannot have a management node running on a PowerPC which directs a data node that is running on an x86 machine. This restriction does not apply to machines simply running mysql or other clients that may be accessing the cluster's SQL nodes.

Binary logging.  NDB Cluster has the following limitations or restrictions with regard to binary logging:

Schema operations.  Schema operations (DDL statements) are rejected while any data node restarts. Schema operations are also not supported while performing an online upgrade or downgrade.

Number of replicas.  The number of replicas, as determined by the NoOfReplicas data node configuration parameter, is the number of copies of all data stored by NDB Cluster. Setting this parameter to 1 means there is only a single copy; in this case, no redundancy is provided, and the loss of a data node entails loss of data. To guarantee redundancy, and thus preservation of data even if a data node fails, set this parameter to 2, which is the default and recommended value in production.

Setting NoOfReplicas to a value greater than 2 is possible (to a maximum of 4) but unnecessary to guard against loss of data. In addition, values greater than 2 for this parameter are not supported in production.

Maximum number of tablespaces: 2³² (4294967296)

Maximum number of data files per tablespace: 2¹⁶ (65536)

The minimum and maximum possible sizes of extents for tablespace data files are 32K and 2G, respectively. See Section 13.1.19, “CREATE TABLESPACE Statement”, for more information.

Data files use DataMemory. Usage is the same as for in-memory data.

Data files use file descriptors. It is important to keep in mind that data files are always open, which means the file descriptors are always in use and cannot be re-used for other system tasks.

Extents require sufficient DiskPageBufferMemory; you must reserve enough for this parameter to account for all memory used by all extents (number of extents times size of extents).

No distributed table locks.  A LOCK TABLES works only for the SQL node on which the lock is issued; no other SQL node in the cluster “sees” this lock. This is also true for a lock issued by any statement that locks tables as part of its operations. (See next item for an example.)

ALTER TABLE operations.  ALTER TABLE is not fully locking when running multiple MySQL servers (SQL nodes). (As discussed in the previous item, NDB Cluster does not support distributed table locks.)

If any of the management servers are running on the same host, you must give nodes explicit IDs in connection strings because automatic allocation of node IDs does not work across multiple management servers on the same host. This is not required if every management server resides on a different host.

When a management server starts, it first checks for any other management server in the same NDB Cluster, and upon successful connection to the other management server uses its configuration data. This means that the management server --reload and --initial startup options are ignored unless the management server is the only one running. It also means that, when performing a rolling restart of an NDB Cluster with multiple management nodes, the management server reads its own configuration file if (and only if) it is the only management server running in this NDB Cluster. See Section 21.5.5, “Performing a Rolling Restart of an NDB Cluster”, for more information.

Python 2.6 or higher.  The Auto-Installer requires the Python interpreter and standard libraries. If these are not already installed on the system, you may be able to add them using the system's package manager. Otherwise, they can be downloaded from http://python.org/download/.

Paramiko 1.7.7.1 or higher.  You can download this from http://www.lag.net/paramiko/ if it is not available from your system's package manager.

Pycrypto version 1.9 or higher.  This cryptography module is required by Paramiko. If it is not available using your system's package manage, you can download it from https://www.dlitz.net/software/pycrypto/.

SSH.  A secure shell connection is used to enable the back end to perform actions on remote hosts. For this reason, an SSH server must be running on the remote host. In addition, the operating system user running the installer must have access to the remote server, either with a user name and password, or by using public and private keys.

HTTPS.  Remote communication between the Web browser front end and the back end is not encrypted by default, which means that information such as the user's SSH password is transmitted as cleartext that is readable to anyone. For communication from a remote client to be encrypted, the back end must have a certificate, and the front end must communicate with the back end using HTTPS rather than HTTP. Enabling HTTPS is accomplished most easily through issuing a self-signed certificate. Once the certificate is issued, you must make sure that it is used. You can do this by starting ndb_setup.py from the command line with the --use-https and --cert-file options.

Certificate-based authentication.  The back end ndb_setup.py process can execute commands on the local host as well as remote hosts. This means that anyone connecting to the back end can take charge of how commands are executed. To reject unwanted connections to the back end, a certificate may be required for authentication of the client. In this case, a certificate must be issued by the user, installed in the browser, and made available to the back end for authentication purposes. You can enact this requirement (together with or in place of password or key authentication) by starting ndb_setup.py with the --ca-certs-file option.

Welcome: Begin using the Auto-Installer by choosing either to configure a new NDB Cluster, or to continue configuring an existing one.

Define Cluster: Set basic information about the cluster as a whole, such as name, hosts, and load type. Here you can also set the SSH authentication type for accessing remote hosts, if needed.

Define Hosts: Identify the hosts where you intend to run NDB Cluster processes.

Define Processes: Assign one or more processes of a given type or types to each cluster host.

Define Attributes: Set configuration attributes for processes or types of processes.

Deploy Cluster: Deploy the cluster with the configuration set previously; start and stop the deployed cluster.

Python 2.6 or higher.  The Auto-Installer requires the Python interpreter and standard libraries. If these are not already installed on the system, you may be able to add them using the system's package manager. Otherwise, you can download them from http://python.org/download/.

Paramiko 2 or higher.  This is required to communicate with remote hosts using SSH. You can download it from http://www.lag.net/paramiko/. Paramiko may also be available from your system's package manager.

Pycrypto version 2.6 or higher.  This cryptography module is required by Paramiko, and can be iunstalled using pip install cryptography. If pip is not installed, and the module is not available using your system's package manage, you can download it from https://www.dlitz.net/software/pycrypto/.

SSH.  A secure shell connection is used to enable the back end to perform actions on remote hosts. For this reason, an SSH server must be running on the remote host. In addition, the operating system user running the installer must have access to the remote server, either with a user name and password, or by using public and private keys.

HTTPS.  Remote communication between the Web browser front end and the back end is not encrypted by default, which means that information such as the user's SSH password is transmitted as cleartext that is readable to anyone. For communication from a remote client to be encrypted, the back end must have a certificate, and the front end must communicate with the back end using HTTPS rather than HTTP. Enabling HTTPS is accomplished most easily through issuing a self-signed certificate. Once the certificate is issued, you must make sure that it is used. You can do this by starting ndb_setup.py from the command line with the --use-https (-S) and --cert-file (-c) options.

A sample certificate file cfg.pem is included and is used by default. This file is located in the mcc directory under the installation share directory; on Linux, the full path to the file is normally /usr/share/mysql/mcc/cfg.pem. On Windows systems, this is usually C:\Program Files\MySQL\MySQL Server 5.7\share\mcc\cfg.pem. Letting the default be used means that, for testing purposes, you can simply start the installer with the -S option to use an HTTPS connection between the browser and the back end.

The Auto-Installer saves the configuration file for a given cluster mycluster01 as mycluster01.mcc in the home directory of the user invoking the ndb_setup.py executable. This file is encrypted with a passphrase supplied by the user (using Fernet); because HTTP transmits the passphrase in the clear, it is strongly recommended that you always use an HTTPS connection to access the Auto-Installer on a remote host.

Certificate-based authentication.  The back end ndb_setup.py process can execute commands on the local host as well as remote hosts. This means that anyone connecting to the back end can take charge of how commands are executed. To reject unwanted connections to the back end, a certificate may be required for authentication of the client. In this case, a certificate must be issued by the user, installed in the browser, and made available to the back end for authentication purposes. You can enact this requirement (together with or in place of password or key authentication) by starting ndb_setup.py with the --ca-certs-file (-a) option.

Welcome: Begin using the Auto-Installer by choosing either to configure a new NDB Cluster, or to continue configuring an existing one.

Define Cluster: Set basic information about the cluster as a whole, such as name, hosts, and load type. Here you can also set the SSH authentication type for accessing remote hosts, if needed.

Define Hosts: Identify the hosts where you intend to run NDB Cluster processes.

Define Processes: Assign one or more processes of a given type or types to each cluster host.

Define Parameters: Set configuration attributes for processes or types of processes.

Deploy Configuration: Deploy the cluster with the configuration set previously; start and stop the deployed cluster.

Automatically save configuration as cookies: Save your configuration information—such as host names, process data, and parameter values—as a cookie in the browser. When this option is chosen, all information except any SSH password is saved. This means that you can quit and restart the browser, and continue working on the same configuration from where you left off at the end of the previous session. This option is enabled by default.

The SSH password is never saved; if you use one, you must supply it at the beginning of each new session.

Show advanced configuration options: Shows by default advanced configuration parameters where available.

Once set, the advanced parameters continue to be used in the configuration file until they are explicitly changed or reset. This is regardless of whether the advanced parameters are currently visible in the installer; in other words, disabling the menu item does not reset the values of any of these parameters.

You can also toggle the display of advanced parameters for individual processes on the Define Parameters screen.

This option is disabled by default.

Automatically get resource information for new hosts: Query new hosts automatically for hardware resource information to pre-populate a number of configuration options and values. In this case, the suggested values are not mandatory, but they are used unless explicitly changed using the appropriate editing options in the installer.

This option is enabled by default.

Contents: Show the built-in user guide. This is opened in a separate browser window, so that it can be used simultaneously with the installer without interrupting workflow.

Current page: Open the built-in user guide to the section describing the page currently displayed in the installer.

About: open a dialog displaying the installer name and the version number of the NDB Cluster distribution with which it was supplied.

For distributions using yum or dnf, you can use the MySQL Yum Repository for NDB Cluster. See Installing MySQL NDB Cluster Using the Yum Repository, for instructions and additional information.

For SLES, you can use the MySQL SLES Repository for NDB Cluster. See Installing MySQL NDB Cluster Using the SLES Repository, for instructions and additional information.

Management node: management-server

Data node: data-node

SQL node: server and common

The server RPM (for example, MySQL-Cluster-server-gpl-7.5.3-1.sles11.i386.rpm), which supplies the core files needed to run a MySQL Server with NDBCLUSTER storage engine support (that is, as an NDB Cluster SQL node) as well as all NDB Cluster executables, including the management node, data node, and ndb_mgm client binaries. This RPM is always required for installing NDB Cluster.

If you do not have your own client application capable of administering a MySQL server, you should also obtain and install the client RPM (for example, MySQL-Cluster-client-gpl-7.5.3-1.sles11.i386.rpm), which supplies the mysql client

Building NDB Cluster requires using the NDB Cluster sources. These are available from the NDB Cluster downloads page at https://dev.mysql.com/downloads/cluster/. The archived source file should have a name similar to mysql-cluster-gpl-7.5.17.tar.gz. You can also obtain NDB Cluster sources from GitHub at https://github.com/mysql/mysql-server/tree/cluster-7.5 (NDB 7.5) and https://github.com/mysql/mysql-server/tree/cluster-7.6 (NDB 7.6). Building NDB Cluster 7.5 or 7.6 from standard MySQL Server 5.7 sources is not supported.

You must configure the build using the WITH_NDBCLUSTER_STORAGE_ENGINE or WITH_NDBCLUSTER option in addition to any other build options you wish to use with CMake. (WITH_NDBCLUSTER is supported as an alias for WITH_NDBCLUSTER_STORAGE_ENGINE, and works in exactly the same way.)

NDB Cluster 7.4 GA releases (7.4.4 and later)

NDB Cluster 7.3 GA releases (7.3.2 and later)

When upgrading from NDB 7.5.2 or 7.5.3 to a later version, the use of mysqld with the --initialize and --ndbcluster options together caused problems later running mysql_upgrade.

When run with --initialize, the server does not require NDB support; having NDB enabled at this time can cause problems with ndbinfo tables. To keep this from happening, the --initialize option now causes mysqld to ignore the --ndbcluster option if the latter is also specified.

A workaround for an upgrade that has failed for these reasons can be accomplished as follows:

(Bug #81689, Bug #82724, Bug #24521927, Bug #23518923)

During an online upgrade from an NDB Cluster 7.3 release to an NDB 7.4 (or later) release, the failures of several data nodes running the lower version during local checkpoints (LCPs), and just prior to upgrading these nodes, led to additional node failures following the upgrade. This was due to lingering elements of the EMPTY_LCP protocol initiated by the older nodes as part of an LCP-plus-restart sequence, and which is no longer used in NDB 7.4 and later due to LCP optimizations implemented in those versions. This issue was fixed in NDB 7.5.4. (Bug #23129433)

Beginning with NDB 7.5.2, the ndb_binlog_index table uses the InnoDB storage engine. (Use of the MyISAM storage engine for this table continues to be supported for backward compatibility.)

When upgrading a previous release to NDB 7.5.2 or later, you can use the --force --upgrade-system-tables options with mysql_upgrade so that it performs ALTER TABLE ... ENGINE=INNODB on the ndb_binlog_index table.

For more information, see Section 21.6.4, “NDB Cluster Replication Schema and Tables”.

Online upgrades from previous versions of NDB Cluster to NDB 7.5.1 were not possible due to missing entries in the matrix used to test upgrade compatibility between versions. (Bug #22024947)

Also in NDB 7.5.1, mysql_upgrade failed to upgrade the sys schema if a sys database directory existed but was empty. (Bug #81352, Bug #23249846, Bug #22875519)

NDB Cluster 7.5 GA releases (7.5.4 and later)

NDB Cluster 7.4 GA releases (7.4.4 and later)

NDB Cluster 7.3 GA releases (7.3.2 and later)

NDB 7.6.2

NDB 7.6.4