A rolling restart is required when you need to restart all of the nodes in a cluster for some change to take effect. Rather than bringing down the entire cluster, you can restart each single node one by one, in order to allow the cluster to continue serving requests during the changes.
Most operations can be done using this technique, such as changing config.ini settings (ie. DataMemory, MaxNoOfConcurrentTransactions, etc.). However, there is a main instance when you can not use this process – Changing an NDB cluster node layout.
When you are changing the topography of a cluster, such as changing NoOfReplicas or removing data nodes. In the scenario above, you will have to do a complete shutdown and a full restart of the cluster. In some scenarios, you may have to dump and re-import data as well.
In order to do a rolling restart the following steps should be taken:
- Setup the new config.ini or install the new version of software.
- Restart the management node using ndb_mgmd. This causes the management server to read in the new configuration file which will be retrieved in the later steps by the data and mysqld nodes. Don’t forget to add the –reload option for later versions.
- Restart each data node using ndbd (or ndbmtd), one at a time. This needs to be a full restart and it has to be done on each machine, not through the Management Server. This will cause the node to read in the remote configuration and to use the new settings and version of the software.
- Restart each SQL node (mysqld). Depending on how the applications are handling failover situations with the mysqld nodes, you may need to reconfigure them to reuse each one as it is brought up.
After you have done the above steps, you should have all of the new configuration information propagated to the nodes. Also, if you upgraded the cluster software, you should now be running the newer version. Throughout the steps above the entire cluster should never stop serving requests during the upgrade process.