Ichnos overview

Ichnos is a general purpose particle tracking code. As such, Ichnos can trace virtual particles in velocity fields. The development of the code is heavily influenced by the projects that Ichnos was developed for, which is primarily particle tracking in groundwater flow fields. In particular, in groundwater flow fields calculated from finite element groundwater models. Yet, groundwater velocity fields from finite difference and finite volume codes can also be used. A key consideration during the development of the code is to handle simulations with multi-million velocity nodes. Therefore the code can be used with velocitiy fields that can be split across many processors.

A brief demonstration of the capabilities of this code can be found at our paper

Kourakos G., Harter T., Dahlke H.E. (2024) Ichnos: A universal parallel particle tracking tool for groundwater flow simulations. SoftwareX 28,101893. 10.1016/j.softx.2024.10189300263-2/fulltext)

In particle tracking there are essentially two main functionalities. i) Tracing the particles in a velocity field. ii) Interpolating a velocity field. The code has been designed so that tracing functions are unaware of the details of the velocity field. To do so the code contains a velocity base class that communicates with the tracing class. Then each different type of field has to derive from the base velocity class and overwrite a few functions. The velocity field can be split into 2 components. The positional information of the points where the velocity is known and the velocity itself. Each of those components are defined in based classes. The positional base class class XYZ is responsible to calculate the weight contribution of each velocity point in the field for a given particle position. Then it passes the weights to the Velocity class which calculates the velocity value. Using this separation it is possible to combine different data structures in the code.

At the moment we provide two types of positional classes XYZ

CLOUD

The CLOUD type assumes that the velocity information is defined into a cloud of points without any particular relation other that proximity between the nodes.
MESH2D

The MESH2D type assumes that the velocity is organized via a 2D mesh that is extruded in the vertical direction. Within the extruded 2D mesh the velocity can be defined at the element barycenters, which assumes constant velocity within the elements, or at the mesh nodes, or at the Mesh faces. In the latter case the interpolation is carried out using the Raviart–Thomas basis functions.

The velocity can be defined as:

Transient or Steady state
Stochastic flow fields (This is in an experimental stage but a demonstration of the stochastic flow field idea can be found in our AGU 2020 poster and AGU 2023 poster)
Random Walk (This is in an experimental stage)

Outline of this repository

Here we breifly outline the Ichnos repository structure and what information each of these folders contain.

Bin is the folder which holds the windows executable of ichnos.
IchnosRwrksp contains R scripts related to ichnos. (Currently we are using primarily Matlab threfore there is no much R scripting there)
WikiExamples contains the configuration files for the examples presented at the wiki
docs was supposed to hold the doxygen documentation for the code. Currently it is empty as we use codeblocks
ichnos_code contains the c++ code.
ichnos_hou contains project files and data for various Houdini visualizations

Getting the code

The code relies on libraries that are available to all major operating OS.

Windons

Windows executables are available under the Bin folder. However, those executables may not contain the latest updates, fixes and new bugs.
Under the Bin folder there is a Dev folder which contains the latest build.

Put under the same folder the ichnos.exe and all the *.dlls. Make sure the antivirus is not blocking the execution. To verify that the code can run try the following under a powershell or command prompt.

ichnos.exe -v

Linux

For linux you have to compile the code. See the section Builinding Ichnos below.

Get started

The wiki is the best source to get you started.

Suggested Roadmap

Because the wiki contains alot of information and examples it can be a bit chaotic. Here we provide a roadmap on how to go through the wiki documentation

The main page of wiki is very important as it provides a list of the required information of the two required configuration files. It lists also all possible options. Therefore it is strongly suggested to go through this page to get an idea of the available options and requirements for a simulation run. This is also the page you would need to consult when setting up the two configuration files
Next it is recommended to go through, first the simple example and seconldy the sample face example excersizes as they go through all required file preparation that one would need to do for an Ichnos simulation.
Then it is recommended to go thought the tutorials that run on a single core. The single core examples can also be run as multicore by tracing particles concurrently.
Multiple core simulations where the domain is split into multiple subdomains require an extra effort and therefore one need to have a good understanding of the file requirements.
Besides the two configuration files ichnos requires a specific format about how to write any information. The File format should be used as a reference to the format of the files.

Building Ichnos

To build ichnos we use cmake. The code contains the required CMakeList.txt. However it is possible that it may need adjustments for each system.

Ichnos depends on a few libraries

MPI. All mpi frameworks should work.
Boost with mpi support. Any relatively new version of boost is good. In the CMakeList.txt we require 1.58 but the code works with lower versions.
CGAL Similarly any relatively recent version should work.
HighFive. This is optional but speeds up the reading of large files and it is highly recommended.

Bulding the dependencies

Getting the dependencies right sometimes can be quite tricky. To build Ichnos we have used three options for different systems.

vcpkg is probably the easiest method to build the dependencies. It has always worked under windows without issues. Under linux can work but not without issues.
Spack Using spack is probably the best option for linux, yet is a bit more involved compared to the vcpkg. There is more info about spack in the next section
Build everything That requires some experience and it is recommended on systems without administration privileges, e.g clusters.

Building with vcpkg

First install all the libraries that Ichnos depend on. Make sure you install the boost with mpi

Then pass the following argument to cmake

-DCMAKE_TOOLCHAIN_FILE=path\to\vcpkg\scripts\buildsystems\vcpkg.cmake -DUSEHF=True

If HighFive is not available then set -DUSEHF=False

Building with Spack

The following is just a guide which at times is not working and requires workarounds.

Prepare the environment

First create an environment that we name it ichnos

spack env create ichnos

and make it active

spack env activate ichnos

CMake

Next we are going to install cmake. Any version should work however I’m specifying here 3.9.4 because this has already been built in other spack environments.

 spack install cmake@3.9.4 target=x86_64

Now the command spack find should find that the cmake is installed and the cmake --version should return the version information.

MPI

Next we will install openmpi. The spack info openmpi will return the available openmpi versions, while at the top specifies which one is the prefered. At the time of writing this that was the 3.1.4 so we build it as

 spack install openmpi@3.1.4 target=x86_64

Usually clusters provide mpi frameworks and it is always a good practice to use these. To instruct Spack to use an external package one has to create a packages.yaml file with content similar to the following:

packages:
  mpi:
    buildable: False
  openmpi:
    modules:
      openmpi@4.0.1: openmpi/4.0.1
    buildable: False

It is important to set for the external packages the buildable flag as false.

Note: It is possible to build your own mpi via spack however there is the chance that you get errors during runtime that mpi is linked with a different version of mpi

Boost

The next piece ichnos needs is boost. I’ll install 1.69 as this is the one already exists from previous projects.

 spack install boost@1.69.0+mpi ^openmpi@3.1.4 target=x86_64

CGAL

CGAL is the next library that is required by ichnos. CGAL depends on boost so we will specify that we want to build cgal using the boost version we just installed

 spack install cgal@4.13 ^boost@1.69.0+mpi ^openmpi@3.1.4 target=x86_64

Sometimes cgal failed to get build because the mprf failed to apply a patch. One way to bypass this is by using a different version of mpfr

 spack install cgal@4.13 ^boost@1.69.0+mpi ^openmpi@3.1.4 ^mpfr@4.0.0

Highfive

The last library Highfive is optional.

 spack install highfive target=x86_64

Building all

Assuming that the current directory is ../ichnos_code/ where the CMakeLists.txt file is, create a folder, get into the folder and run cmake as

 mkdir build
 cd build
 cmake ..

If cmake finds the wrong boost version e.g the one under /usr/ for example then passing the following during cmake might help

cmake -DUSEHF=True \
      -DBoost_NO_BOOST_CMAKE=TRUE \
      -DBoost_NO_SYSTEM_PATHS=TRUE \
      -DBOOST_ROOT:PATHNAME=/path/to/spack/var/spack/environments/ichnos/.spack-env/view/ ..

Running Ichnos

More information about running Ichnos can he found in the wiki section.

Who do I talk to

It depends on what you want to say

The Discussion section is the best place to ask questions/clarifications regarding the use of Ichnos.

Any issues with the code should be reported at Issues section.

For other information you can reach us via email