HySoP (Hybrid Simulation with Particles):

HySoP is a library dedicated to high performance direct numerical simulation of fluid related problems based on semi-lagrangian particle methods, for hybrid architectures providing multiple compute devices including CPUs and GPUs. HySoP is written in Python (main user interface and high level functionnalities) and Fortran.

See HySoP documentation for more information, screenshots and animations.

• HySoP (Hybrid Simulation with Particles):
• Try online at mybinder
• Documentation
• Quick start: use a Docker image (recommended)
• Install from sources
  • Step 1: get sources
  • Step 2: install prerequisites
    • Use a Docker image
    • Manual installation of dependencies
  • Step 3: install
    • Setup and build
    • Configuration options
    • Tests

Try online at mybinder

Click to open a jupyter notebook with Hysop installed and ready to use.

There you will find two detailed examples (2D advection problem and a flow past a cylinder) that will help you to understand hte simulation pipeline and familiarize yourself with Hysop.

Documentation

The documentation is available on Hysop web page with a user manual and the complete (auto-generated) API.

The documentation generation is handled in a separate Gitlab project hysop-doc.

Quick start: use a Docker image (recommended)

Docker Images (CPU and GPU) are provided with a complete installation of HySoP and its dependencies.

Docker Desktop must be installed on your computer! To find out how to install Docker Desktop click here.

HySoP "CPU_Intel" Docker Image can be retrieved and launched as a Docker Container. This image corresponds to Ubuntu 22.04 running with python3.12 and an Intel CPU OpenCL platform.

To start Hysop in a Jupyter notebook, run the following command in a terminal (or with you favorite Docker API)

docker run -p 8888:8888  --rm -ti \
  gricad-registry.univ-grenoble-alpes.fr/particle_methods/hysop/hysoplab-cpu-intel-master:latest

Or to start a terminal command-line session:

docker run -it --rm --entrypoint="/bin/bash" \
  gricad-registry.univ-grenoble-alpes.fr/particle_methods/hysop/hysoplab-cpu-intel-master:latest

If the host has Nvidia GPUs, replace hysoplab-cpu-intel-master with hysoplab-gpu-nvidia-master and run Docker as:

docker run -p 8888:8888 --runtime=nvidia --gpus all -it --rm --entrypoint="/bin/bash" \
  gricad-registry.univ-grenoble-alpes.fr/particle_methods/hysop/hysoplab-cpu-intel-master:latest

In all cases, you will end up with a fully-fonctionnal installation of Hysop software (in a Docker Container ) and will be able to quickly test HySoP using examples in the documentation.

Please note the following:

• If the HySoP Docker Image is not present on your host machine, it will be automatically downloaded from the HySoP gitlab container registry. This may take some time, depending on your Internet connection. HySoP Docker Image size is beetween 3 and 4 GB.

• Micromamba packaged manager is used as a virtual environment for HySoP.

• By default, the Docker container is completely isolated from the host machine's disk space! As a result, to be allowed to write or read data on your hard disk you need to explicitely share a directory between your host and the docker container. To do so, use the '-v' Docker option and update the command like this:

  docker run -v <HOST_DIRECTORY>:/home/hysop-user/shared -p 8888:8888  --rm -ti \
    gricad-registry.univ-grenoble-alpes.fr/particle_methods/hysop/hysoplab-gpu-nvidia-master

  Replace <HOST_DIRECTORY> (use full path!) with the path you want to shared between host and container. It will be available in the container in directory '/home/hysop-user/shared'.

Install from sources

Hysop sources are available in a Gitlab project (Apache v2 license)

Steps:

1. Download hysop sources on the host
2. Get all dependencies
3. Build and install Hysop

The main question here is for step 2: how to handle HySoP dependencies? Either you can use a Docker image (simpler) or install everything on your system (harder).

Step 1: get sources

Git clone HySoP and change directory to 'hysop':

git clone https://gricad-gitlab.univ-grenoble-alpes.fr/hysop.git

Step 2: install prerequisites

Use a Docker image

Docker images with all the required dependencies to build Hysop are available in Hysop Gitlab registries.

To start a container with these images, just run:

docker run -v $(pwd)/hysop:/home/hysop-user/hysop-shared --rm -ti \
  gricad-registry.univ-grenoble-alpes.fr/particle_methods/hysop/[imagename]

Your local directory 'hysop' is then shared inside the Docker Container in shared directory.

[imgname] is either ci_cpu_intel or ci_gpu_nvidia. These images are base image for the recommended usage and for Continuous Integration.

Manual installation of dependencies

Installing HySoP dependencies can be a lengthy process. That's why we prefer to use Docker Images to avoid this tedious installation phase. However, Docker utilities are rarely available on cluster computers, in which case all HySoP dependencies will have to be installed.

The quickest way to discover all the HySoP dependencies and how to install them is to explore the Dockerfile available in hysop sources hysop/ci/docker_images/ci_user/Dockerfile

NVIDIA driver: Please note that to run HySoP on NVIDIA GPUs, a Nvidia driver corresponding to your graphics card must be installed and running on your host system. Here you'll find the various NVIDIA drivers and how to install them.

You may handle all dependencies by yourself, with your favorite package manager, with venv Python and so on but we recommend to use Micromamba packaged manager. To create and activate a micromamba environment please use the yaml provided in Hysop sources, just try:

micromamba create -n hysop -f hysop/ci/hysopenv.yaml
micromamba activate hysop

User must complete the main dependencies with an OpenCL platform. Currently, we support only two:

• Intel CPU OpenCL platform

micromamba install -n base -y  intel-opencl-rt

• Nvidia OpenCL platform (note that the real path to libnvidia-opencl.so must be adapted).

echo "/usr/lib/x86_64-linux-gnu/libnvidia-opencl.so.1" > $CONDA_PREFIX/etc/OpenCL/vendors/nvidia.icd

Step 3: install

Once the container has started, you're connected has 'hysop-user' with all Hysop sources inside /home/hysop-user/hysop-shared (a directeory shared between the container and the host)

Next step consists in configuring, building and installing HySoP thanks to Meson build system inside the Docker container as described in installation instruction below.

Setup and build

We consider that hysop sources are available in directory '/path-to/hysop'.

Install HySoP using pip:

pip install --no-build-isolation --no-deps /path-to/hysop

Or install HySoP manually step by step using meson:

export BUILD_DIR=builddir # Choose any directory for builddir DIFFERENT from hysop sources. This will be a temporary dir.
meson setup  ${BUILD_DIR}  /path-to/hysop
meson compile -C ${BUILD_DIR}
meson install -C ${BUILD_DIR}

Then Hysop will be installed in the current active Python environment (micromamba, venv ...)

To choose another directory of installation, try option --python.install-env prefix

export BUILD_DIR=builddir
meson setup  --python.install-env prefix --prefix=${BUILD_DIR}/install  ${BUILD_DIR}  /path-to/hysop
meson compile -C ${BUILD_DIR}
meson install -C ${BUILD_DIR}

Note that in this case, an export of PYTHONPATH would be necessary.

export PYTHONPATH=${BUILD_DIR}/install/lib/python3.12/site-packages:$PYTHONPATH

Configuration options

Installation configuration may be customized by user overriding default configuration (for a complete list of these options, see meson_options.txt file in Hysop sources):

meson setup -DOPTION_NAME=VALUE  ${BUILD_DIR}  /path-to/hysop

OPTION_NAME being one of the options described below and ‘VALUE’ is either ‘ON’ or ‘OFF’.

Behavior options:

• DOUBLEPREC: Set default HySoP floating point precision. Default=ON, ie double precision else single precision will be used.
• VERBOSE: Enable verbose mode for HySoP (default=ON).
• PROFILE: Enable profiling mode for HySoP (default=OFF).
• DEBUG: Enable debug mode for HySoP (default=OFF).

Components options :

• WITH_FFTW: Link with fftw library (required for some HySoP solvers), (default=ON).
• WITH_EXTRAS: Link with some extra fortran libraries (like arnoldi solver), (default=OFF).
• WITH_GPU: Use of GPU (required for some HySoP solvers), (default=ON)
• WITH_SCALES: to compile an HySoP version including scales (default = ON).
• WITH_TESTS: enable testing (i.e. prepare target “meson test”, default = OFF).
• WITH_DOCUMENTATION: Build Documentation. (default=OFF)

Tests

When the installation process is over, first test that HySoP package is available and works properly. Try:

python3 -c "import hysop; print(hysop); print(dir(hysop))"
python3 -c "from hysop import f2hysop; print(f2hysop);print(dir(f2hysop))"

Then launch units tests shipped with the code

meson test    -C ${BUILD_DIR}

Finally one can try to launch all the provided examples from top directory of HySoP sources:

EXAMPLE_DIR=./hysop_examples/examples /path-to/hysop/ci/scripts/run_examples.sh