Docker containers and compiler support
How to use docker images ?
Docker images can be pulled (resp. pushed) with ./ci/utils/pull_docker_image.sh [imgname]
and ./ci/utils/push_docker_image.sh [imgname]
. The docker images do not contain de hysop library and can be run with ./ci/utils/run_docker_image.sh [imgname]
. This script mounts your local hysop directory (read only) to /hysop
inside the docker container.
To quickly test and/or debug hysop inside the docker you can run ./ci/utils/run_debug.sh [imgname]
which will build and install hysop inside the container and prompt a shell (read-only mount-bind /hysop
directory is copied to read-write /tmp/hysop
within the container). Alternatively if you just want to run tests inside docker, you can use the ./ci/utils/run_ci.sh [imgname]
script instead.
By default, [imgname]
corresponds to the docker image used for gitlab continuous integration (currently groovy
) but any other ubuntu image name may be used, hirsute
being the most up-to-date image. Docker images ship an intel opencl platform that is compatible with intel cpus. Cuda enabled images may be a bit out-of-date (see ./ci/docker_images/ubuntu/bionic_cuda
to get started).
Docker images can be build locally by using the ./ci/utils/build_docker_image.sh
script. It is advised to build docker image only if the pull fails or if you want to add/update dependencies. Each build takes around one hour (36 cores @ 2GHz). By default, the build script will use all of your cores. At least 16GB of RAM is recommended.
Known problems
-
OpenMPI complains about programs run as root: You can solve this by exporting defining two extra variables:
export OMPI_ALLOW_RUN_AS_ROOT=1 export OMPI_ALLOW_RUN_AS_ROOT_CONFIRM=1
These variables are already defined during tests and when running
./ci/utils/run_debug.sh
. -
OpenMPI complains about not having enough slots: This happens when you request more processes than you have physical cpu cores. To be able to run more processes you can define the following variables:
export OMPI_MCA_rmaps_base_oversubscribe=1
. -
The program crash with
Bus error (Non-existant physical address)
:By default, docker does not enable ptrace which makes the vader BTL crash (vader is openmpi shared memory interprocess communicator).
The solutions to this problem are:
- Enable ptrace when running docker (best solution): use
docker run --cap-add=SYS_PTRACE
to run an interactive container ordocker create --cap-add=SYS_PTRACE
to create a persistent container. This is already done in our scripts in./ci/utils/*
. - Disable vader CMA (Cross Memory Attach) with
OMPI_MCA_btl_vader_single_copy_mechanism=none
: works but interprocess communication performance drops ~50%. - Disable vader completely:
mpirun --mca btl ^vader
: works but leads to really poor performances
- Enable ptrace when running docker (best solution): use
Update some dockers images for the following distributions (Intel OpenCL):
-
ubuntu:jammy (22.04 LTS, April 2027):
Dockerfile, contains updated dependencies, actually used for CI on the gricad shared gitlab-ci runner (python 3.9). -
ubuntu:hirsute (21.04):
Dockerfile -
ubuntu:groovy (20.10):
Dockerfile -
ubuntu:focal (20.04 LTS, April 2025):
Dockerfile -
ubuntu:bionic (18.04 LTS, April 2023):
Dockerfile
Docker images that supports nvidia OpenCL and CUDA:
-
ubuntu:jammy (22.04 LTS) with CUDA 11.7
Dockerfile, actually not used for CI (no dedicated gitlab-ci runner with a gpu available). This runner depends on the host nvidia driver 515.65 and nvidia-docker2. -
ubuntu:bionic (18.04 LTS) with CUDA 9.2:
Dockerfile, actually used for CI on zephyr dedicated gitlab-ci runner with a gtx 980Ti. This runner depends on the host nvidia driver 396.45 and nvidia-docker2. Requires an update.
Docker images that have been deprecated due to end of life:
-
ubuntu:zesty (17.04, January 2018)
-
ubuntu:xenial (16.04 LTS, April 2021):
Dockerfile
Hysop build compiler status:
-
g++-4.4.7
f2py not working:unrecognized command line option "-Wdate-time"
-
g++-4.5.1
f2py not working:unrecognized command line option "-Wdate-time"
-
g++-4.6.4
f2py not working:unrecognized command line option "-Wdate-time"
-
g++-4.7.4
f2py not working:unrecognized command line option "-Wdate-time"
-
g++-4.8.5
f2py not working:unrecognized command line option "-Wdate-time"
-
g++-4.9.4
-
g++-5.4.1
-
g++-6.5.0
-
g++-7.5.0
-
g++-8.4.0
-
g++-9.3.0
-
g++-10.3.0
-
g++-11.1.0
-
clang++-3.5.2
-
clang++-3.6.2
-
clang++-3.7.1
-
clang++-3.8.0
-
clang++-3.9.1
-
clang++-4.0.0
-
clang++-5.0.1
-
clang++-6.0.0
-
clang++-10.0.0
-
clang++-11.0.0