Tip Function Object systemCall
Contents
1 Introduction to systemCall
The Function Object systemCall gives the user the power to execute a list of system calls in three types of instances during execution of a solver (some utilities might also use this function object):
- executeCalls
- List of commands (system calls to the command shell) to be executed just before the new time iteration is simulated.
- endCalls
- List of commands to be executed before the simulation is over and it is called before the last writeCall.
Note: this overrides executeCalls for after the final iteration. In other words, if you need to run the same list from executeCalls after the last iteration, then you will have to repeat it here as well. - writeCalls
- List of commands to be executed after a time iteration snapshot is written to disk.
What's this Function Object good for?
- Interoperability with shell scripts for moving files from a cluster to the workstation during run time, to save user quota.
- Interoperability with external processes for one-way communication with OpenFOAM simulation cases, for example, with finite elements solvers that take as input the forces made by the fluid onto the geometry in question. This exceptionally good for when it's not desirable to hard code the FEM solver to be used into the modified OpenFOAM based solver.
- Also good for two-way communication for solid deformation iterated with an independent FEM solver. Keep in mind that on OpenFOAM's side a dynamic mesh aware solver would be needed. This could lead to a very interesting damBreak type of tutorial, with fractures and actual pieces of the dam breaking off...
2 Usage Description
A Function Object is a library that is included by defining in the system/controlDict file in the simulation case folder, how to load it and how to use it. For systemCall this is the base structure:
functions { sysCall { type systemCall; functionObjectLibs ( "libsystemCall.so" ); executeCalls 0(); endCalls 0(); writeCalls 0(); outputControl outputTime; outputInterval 1; } }
Description of details:
- sysCall is just the name given for this function on the functions list and it could be any other name.
- The type systemCall is a must, since its what defines this function object to be used.
- functionObjectLibs defines in which library the desired function object is in.
- The lists of strings 0() have to be defined as the number of items on the list; the list itself is inside the parenthesis, each item separated with spaces (see the Example section).
- outputControl can have one of two internal name values: outputTime or timeStep.
- outputInterval is an integer/scalar value of the same type as controlDict's writeInterval. This parameter is only used when outputControl is set to timeStep.
2.1 Additional tuning
With OpenFOAM 2.0 a new security flag was created to avoid the OpenFOAM applications from launching mischievous scripts and commands via shell and even to avoid executing insecure code. So, to allow the function object systemCall to work, one must do the following steps:
- Create the folder ~/.OpenFOAM/$WM_PROJECT_VERSION like so:
mkdir -p ~/.OpenFOAM/$WM_PROJECT_VERSION
- Copy the global controlDict file to the new folder like this:
cp $WM_PROJECT_DIR/etc/controlDict ~/.OpenFOAM/$WM_PROJECT_VERSION/
- Edit the file ~/.OpenFOAM/$WM_PROJECT_VERSION/controlDict.
- Find the block that has the name InfoSwitches.
- Change the variable allowSystemOperations' to 1. It should now look something like this:
InfoSwitches { writePrecision 6; writeJobInfo 0; // Allow case-supplied C++ code (#codeStream, codedFixedValue) allowSystemOperations 1; }
- Save the file and close.
You should now be ready to use the function object systemCall!
3 Example
Add this near the end of the system/controlDict file for any OpenFOAM case:
functions { sysCall { type systemCall; functionObjectLibs ( "libsystemCall.so" ); executeCalls 1("echo Execute system call before time iteration is done"); endCalls 1("echo Finishing up with a system call, which is seems to be before the write call..."); writeCalls 2("echo Writing to file call" "ls -l"); outputControl outputTime; } }
Notice how the list of calls should be made, with the numbered list separated by spaces between each item.
After saving the file, run the solver for the chosen case. For example, if you picked the cavity case for tutorials/incompressible/icoFoam, run it like this while inside the cavity case:
blockMesh #just in case, generate the mesh, if you haven't done it yet icoFoam | tee log.icoFoam 2>&1
The last command allows you to both store the output to the file log.icoFoam and see the output during execution. When this output is analyzed, the instances can be inferred for when each system call is made.
3.1 Running in Parallel
When using systemCall in a parallel run, calls are made for all parallel processes that were launched. However, in this function object there is no way for OpenFOAM to tell the command shell which caller is the master process; but the MPI library used by OpenFOAM does set environment variables indicating the assigned ID number to each parallel process.
Since each MPI toolbox has it's own way to manage these environment variables, we have to first discover which ones to use. For that, set in one of the calls this command:
export > env$$.log
The two $ will expand in the shell to the PID number of the running shell.
When the run is complete (or after you abort), you'll have several log files. To compare them, use the command:
diff -Nur env1234.log env3245.log
The numbers are random here, but the idea is to compare the logs that were generated.
In the comparisons, you should be able to see the variable that has stored the number of the parallel process assigned by the MPI. Based on that number, you will know which one is the master process, which is either 0 or 1, depending on the number assignment scheme used by the MPI toolbox being used.
Then it's just a matter of using shell scripting. For example, for having only the master process do its work (MPI_PROCESS_CORE is a fake name here):
if [ $MPI_PROCESS_CORE = 0 ]; then echo "Do what the master needs to do." fi
For more information about shell scripting, look for it online or start right away by reading this tutorial.