The message passing interface (MPI) is a communication protocol that allows the transferring of data among nodes in a compute cluster. This enables users to distribute a large simulation among multiple machines rather than rely on the limited memory installed on a single machine. XFdtd's calculation engine, XFsolver, supports Open MPI for the 64-bit Red Hat Enterprise Linux 6 (RHEL6) and later versions, as well as operating systems that are comparable with one of these platforms.
Users can install and run XF when the MPI cluster configuration meets the following requirements:
- A shared file system is available to all nodes (e.g., /opt/).
- The XF simulation directory is in the shared network directory and each node has write permissions.
- The openmpi installation directory is installed in the same location on each node. Alternatively, it can be placed in a single shared network directory (e.g., /opt/openmpi/).
Install
Similar to the MPI installation, XF must be installed either in the same location on each node or in a shared network directory:
- Log on to the node(s) and follow the installation instructions for the operating system.
License
The distributed machines in a compute cluster must have access to a floating license server:- Follow the instructions to set up a floating license configuration.
- Define the environment variable REMCOMINC_LICENSE_FILE on each node (e.g., REMCOMINC_LICENSE_FILE=1711@node0).
Run
Once XF is installed and licensed, XFsolver can be run in a simulation directory (e.g. *.xf/Simulations/00001) by using an -openmpi command line, similar to the example below:
[remcom@head 000001]$ /opt/openmpi/bin/mpirun -np 4 --host node1,node2 /opt/remcom/bin/xfsolver -openmpi -v
System administrators typically provide instructional documentation about launching MPI programs on their clusters. Users should note and that the mpirun command for any particular cluster may differ from the command above.