Command-line execution
Before MUSCLE can be run from the command-line, it needs a few environment variables to be set. To do this, run
source [path/to/muscle]/etc/muscle.profile
The [path/to/muscle] should point to where MUSCLE is installed on your machine.
Local execution
Depending on the complexity of the code and the runtime requirements MUSCLE has different execution modes. In the simplest case, all the code can be run on a single machine, and no debugger or MPI is used. Use, for example, the SimpleExample.cxa.rb provided with MUSCLE. This will write data from submodel w to submodel r.
muscle2 --cxa $MUSCLE_HOME/share/muscle/examples/cxa/SimpleExample.cxa.rb --main --allkernels
Alternatively, the commands also have short notation, so
muscle2 -c $MUSCLE_HOME/share/muscle/examples/cxa/SimpleExample.cxa.rb -ma
When we dissect this command line, it starts the with the cxa file it needs, $MUSCLE_HOME/share/muscle/examples/cxa/SimpleExample.cxa.rb. Then --main indicates that the current execution is the main execution site, more on that later. Finally, --allkernels indicates that all kernels specified in the cxa file should be executed at the same time.
For execution on a local computer this command should be sufficient.
Execution on multiple machines
If multiple machines are needed to run the code, they need some way of locating each other. This is where the --main flag is used, it indicates that the simulation manager will be run on the current location. Suppose we are running SimpleExample.cxa.rb which is provided with MUSCLE, which has the submodels "w" and "r". Then if the machines all have access to one of the other machines, you can run
muscle2 --main --cxa $MUSCLE_HOME/share/muscle/examples/cxa/SimpleExample.cxa.rb w
which will run w on the current machine. On the command-line, it will show the following lines:
== Running MUSCLE2 Simulation Manager == [10:04:37 muscle] Started the connection handler, listening on 192.168.2.1:9000
The address at the end of the line can be used by the other MUSCLE executions. On a second machine, then run
muscle2 --manager 192.168.2.1:9000 --cxa $MUSCLE_HOME/share/muscle/examples/cxa/SimpleExample.cxa.rb r
This will start the simulation, contacting the simulation manager at 192.168.2.1:9000. For testing purposes, both commands can also be run on the same machine.
Running MPI
There are four ways of running a native submodel with MPI.
Starting MPI from MUSCLE
If MPI kernel uses the muscle.core.standalone.MPIKernel then only the CxA file needs to be adapted: "submodelName:mpiexec_command" and "submodelName:mpiexec_args" should be set. Then MUSCLE will create a new process with MPI, running with the same command-line arguments as usual.
Starting MPI with MUSCLE
If the MPI executable can be run by the user, first the variables "submodelName:mpiexec_command" and "submodelName:mpiexec_args" should be given in the CxA file. The most straightforward is running with the --native flag. Simply run
muscle2 --native --main --cxa path/to/cxa.rb submodelName
Of course, replace path/to/cxa.rb with your CxA file, and replace --main with --manager ip:port if another execution is already main. Running this way requires that only one submodel is run at a time.
Starting MPI directly
The second way, more direct, does not require the variable "submodelName:mpiexec_command" to be set. Then, the executable that was compiled with the MUSCLE dynamic library can be called as follows:
mpiexec -np 8 path/to/nativeSubmodel arg1 arg2 -- --main --cxa path/to/cxa.rb submodelName
Since you run mpiexec yourself, this does not require the mpiexec_command to be set. The command first specifies the native executable, and any arguments that the executable needs, and then it has two dashes "--" and then the same arguments as in the previous command.
Starting Java separately
Since the MUSCLE C++ API needs to communicate with Java code, in the previous examples a fork was performed to execute the Java core of MUSCLE. Some implementations of MPI do not allow this behavior, so there a different approach is needed.
First, the Java code should be started on a machine that can be contacted with TCP/IP from the machine running MPI. This is done with the command
muscle2 --main --cxa path/to/cxa.rb --native-tmp-file native.host.txt submodelName
Then open the file native.host.txt. It contains the hostname and port with which the native code should communicate. Then set environment variables on the machine where you will run MPI
export MUSCLE_GATEWAY_HOST=[hostname found in native.host.txt] export MUSCLE_GATEWAY_PORT=[port found in native.host.txt]
After this procedure, either of the MPI commands above can be used to start MUSCLE, and it will not perform a fork.
