= Introduction =
This page ilustrates MUSCLization of real astrophysics code [http://piernik.astri.umk.pl/ PIERNIK] ("Gingerbread").

= The orginal code =
- monolitic
- fortran
- MPI/HDF5 dependency
- high scalability of the main MHD module
- propotype particule simulation Monte Carlo module
- coupling done via global variables
= The goal =
- couple using MUSCLE, why?
 * the coupling is done one way (from MHD to MC), this brings a potential of introducing new level of parallelism (in curent code MHD and MC simulations are called one after another, sequentially)
 * the MC is in process of GPU-enabling, one may want to run both modules on different heterogenous resources (i.e. MC on GPU cluster, base MHD code on Intel Nehalem cluster)
= Step By Step Guide =
== linking with MUSCLE ==
PIERNIK uses own build system based on Python scripts, all set of flags used for compilation are stored in plain configuration files like this one:
{{{
PROG      = piernik
USE_GNUCPP = yes
F90       = mpif90
F90FLAGS  =  -ggdb -fdefault-real-8 -ffree-form -std=gnu -fimplicit-none -ffree-line-length-none 
F90FLAGS += -Ofast -funroll-loops
F90FLAGS += -I/software/local/libs/hdf5/1.8.9-pre1/gnu-4.7.2-ompi/include
LDFLAGS   = -Wl,--as-needed -Wl,-O1 -L/software/local/libs/hdf5/1.8.9-pre1/gnu-4.7.2-ompi/lib 
}}}
In order to link with MUSCLE 2.0 we have to alter the last lines of the file:
{{{
...
LDFLAGS   = -Wl,--as-needed -Wl,-O1 -L/software/local/libs/hdf5/1.8.9-pre1/gnu-4.7.2-ompi/lib -L/mnt/lustre/scratch/groups/plggmuscle/2.0/devel-debug/lib 
LIBS      = -lmuscle2
}}}
Then we build the code with the following command:
{{{
# load MUSCLE
module load muscle2/devel-debug
# load PIERN dependencies (HDF5, OpenMPI, newest GNU compiler)
module load plgrid/libs/hdf5/1.8.9-gnu-4.7.2-ompi
./setup mc_collisions_test -c gnufast -d HDF5,MUSCLE,MHD_KERNEL
}}}