HySoP (Hybrid Simulation with Particles) is a library dedicated to high performance direct numerical simulation of fluid related problems based on semi-lagrangian particle methods, for hybrid architectures providing multiple compute devices including CPUs, GPUs or MICs. See https://particle_methods.gricad-pages.univ-grenoble-alpes.fr/hysop-doc for more information.

Basics

Create and use a Python virtual environment to install python modules required to install and run hysop (to avoid conflits between 'apt install pyhton-xyz' and 'pip install xyz'). An example:

export VIRTUAL_ENV=/opt/venv
python3 -m venv $VIRTUAL_ENV
export PATH=${VIRTUAL_ENV}/bin:$PATH

Download lastest hysop:

git clone https://gricad-gitlab.univ-grenoble-alpes.fr/hysop.git
cd hysop

Install hysop using pip:

pip install .

Install hysop using meson:

export BUILD_DIR=builddir
meson setup         ${BUILD_DIR}
meson compile -v -C ${BUILD_DIR}
meson install    -C ${BUILD_DIR}

Install hysop using meson and force a directory of installation: In this case, meson option --python.install-env prefix is needed.

export BUILD_DIR=builddir
meson setup         ${BUILD_DIR} --python.install-env prefix --prefix=${BUILD_DIR}/install
meson compile -v -C ${BUILD_DIR}
meson install    -C ${BUILD_DIR}

By default, the python minimum required version is 3.10. Hysop installation is ready for Python3.12, but some hysop dependencies are not available for this python version!

Run Taylor-Green with 8 mpi processes:

mpirun -np 8 python3 hysop_examples/examples/taylor_green/taylor_green.py -impl fortran -V -cp fp64 -d 64

Check examples directory for complete simulation cases. Hysop is currently developped and tested with python3.10. It also works well with python3.8 and python3.9. You may want to export PYTHON_EXECUTABLE=python3.9 to run some of hysop scripts that look for python3 by default.

For each example you can get help with the -h option:

 python3 hysop_examples/examples/taylor_green/taylor_green.py -h

Dependencies:

To install hysop dependencies locally on an ubuntu machine (jammy-22.04 LTS):

sudo apt-get install -y expat unzip xz-utils pkg-config cmake rsync git ssh curl wget ca-certificates gcc g++ lsb-core cpio libnuma1 libpciaccess0 libreadline-dev libgcc-11-dev libcairo-dev libcairomm-1.0-dev python3.10-dev python3.10-venv openmpi-bin libopenmpi-dev hdf5-tools
export VIRTUAL_ENV=/opt/venv
python3 -m venv $VIRTUAL_ENV
export PATH=${VIRTUAL_ENV}/bin:$PATH
python3 -m pip install --upgrade -r ci/requirements_minimal.txt

Additionally you may want to provide a working OpenCL platform, HPTT, llvm/llvmlite/numba, clFFT/gpyFFT, flint/arb/python-flint and tbb/mklfft. See the docker files to get instructions on how to configure and install these packages (hysop/ci/docker_images/ubuntu_jammy/Dockerfile). Alternatively your can run hysop in an isolated environment by using either singularity or docker containers (see next sections).

Running in isolation with docker (recommended):

The docker images do not contain the hysop library. Please git clone hysop before to pull a docker image!

Docker images can be pulled (resp. pushed) with ./ci/utils/pull_docker_image.sh [imgname] and ./ci/utils/push_docker_image.sh [imgname]. The docker images can be run with ./ci/utils/run_docker_image.sh [imgname]. This script mounts your local hysop directory (read only) to /hysop inside the docker container and prompt a shell.

By default, [imgname] corresponds to the docker image used for gitlab continuous integration (currently ubuntu_jammy_meson, which corresponds to Ubuntu 22.04 running with python3.10). Docker images without the _nvidia postfix ship an intel OpenCL platform that is compatible with intel CPUs. Try out the ubuntu_jammy_nvidia docker image to run on the GPUs (requires host system driver compatible with cuda 12.0).

To quickly test and/or debug hysop inside the docker you can run /hysop/ci/utils/build_and_test.sh which will build and install hysop inside the container and submit tests.

Docker images can be build locally by using the ./ci/utils/build_docker_image.sh script. It is advised to build docker image only if the pull fails or if you want to add/update dependencies.

There are equivalent *.bat scripts to run on Windows. Please note that the nvidia enabled images currently do not work on Windows due to this bug.

Testing:

Hysop installation:

python3 -c "import hysop; print(hysop); print(dir(hysop))"
python3 -c "from hysop import f2hysop; print(f2hysop);print(dir(f2hysop))"

Tests: Set PYTHONPATH if needed!

./ci/scripts/test.sh

Using meson:

meson test    -C ${BUILD_DIR}

To know if Opencl is present?

clinfo

or

python3 -c 'import hysop; from hysop.testsenv import iter_clenv, __HAS_OPENCL_BACKEND__; print(next(iter(iter_clenv())) if __HAS_OPENCL_BACKEND__ else "");'