Skip to content
Snippets Groups Projects
Commit 9dd5d10a authored by Jean-Baptiste Keck's avatar Jean-Baptiste Keck
Browse files

Update docker image documentation, remove all hardcoded python executables

parent c0e24d2e
No related branches found
No related tags found
1 merge request!40Add docker ubuntu 22.04 cuda 11.7 image
Pipeline #110529 passed
......@@ -18,7 +18,7 @@
# See https://pre-commit.com/hooks.html for more hooks
#
default_language_version:
python: python3.9
python: python3
repos:
- repo: https://github.com/johann-petrak/licenseheaders.git
......
......@@ -36,9 +36,9 @@ make -j8
make install
```
By default, cmake will try to find your most up to date Python3 installation. The minimum required version is Python3.8. You can force the python version by using the following trick during the cmake configuration step:
By default, cmake will try to find your most up to date Python3 installation. The minimum required version is Python3.8. You can force the python version by using the following trick during the cmake configuration step. If you want to force cmake to detect python3.9:
```
PYTHON_EXECUTABLE="$(which python3.8)"
PYTHON_EXECUTABLE="$(which python3.9)"
PYTHON_INCLUDE_DIR=$(${PYTHON_EXECUTABLE} -c "import sysconfig as sc; print(sc.get_paths()['include'])")
PYTHON_LIBRARY=$(${PYTHON_EXECUTABLE} -c "import sysconfig as sc, os; print(os.path.normpath(os.path.sep.join(sc.get_config_vars('LIBDIR', 'INSTSONAME'))))")
cmake -DPython3_EXECUTABLE="${PYTHON_EXECUTABLE}" -DPython3_INCLUDE_DIR="${PYTHON_INCLUDE_DIR}" -DPython3_LIBRARY="${PYTHON_LIBRARY}" ..
......@@ -46,13 +46,13 @@ cmake -DPython3_EXECUTABLE="${PYTHON_EXECUTABLE}" -DPython3_INCLUDE_DIR="${PYTHO
Run Taylor-Green with 8 mpi processes:
```
mpirun -np 8 python3.9 hysop_examples/examples/taylor_green/taylor_green.py -impl fortran -V -cp fp64 -d 64
mpirun -np 8 python3 hysop_examples/examples/taylor_green/taylor_green.py -impl fortran -V -cp fp64 -d 64
```
Check examples directory for complete simulation cases. Hysop is currently developped and tested with `python3.9`. It also works well with `python3.8`. You may want to `export PYTHON_EXECUTABLE=python3.8` to run some of hysop scripts that look for `python3.9` by default. Please note that `python3.10` has not been tested yet but should work as well.
Check examples directory for complete simulation cases. Hysop is currently developped and tested with `python3.10`. It also works well with `python3.8` and `python3.9`. You may want to `export PYTHON_EXECUTABLE=python3.9` to run some of hysop scripts that look for `python3` by default.
For each example you can get help with the `-h` option:
```
python3.9 hysop_examples/examples/taylor_green/taylor_green.py -h
python3 hysop_examples/examples/taylor_green/taylor_green.py -h
```
Dependencies:
......@@ -60,20 +60,22 @@ Dependencies:
To install hysop dependencies locally on an ubuntu machine (20.04 LTS):
```
sudo apt-get install -y expat unzip xz-utils automake libtool pkg-config cmake git ssh gcc g++ gfortran lsb-core cpio libnuma1 libpciaccess0 libreadline-dev libblas-dev liblapack-dev libgcc-10-dev libgfortran-10-dev libgmp-dev libmpfr-dev libmpc-dev python3.8-dev opencl-headers swig libgmp-dev libmpfr-dev libmpc-dev libcairo-dev libcairomm-1.0-dev python3.8-dev python3.8-tk libopenmpi-dev libhdf5-openmpi-dev libfftw3-dev libfftw3-mpi-dev
python3.9 -m pip install --upgrade numpy setuptools cffi wheel pytest pybind11 cython
python3.9 -m pip install --upgrade -r requirements.txt
python3 -m pip install --upgrade numpy setuptools cffi wheel pytest pybind11 cython
python3 -m pip install --upgrade -r requirements.txt
```
Additionally you may want to provide a working OpenCL platform, HPTT, llvm/llvmlite/numba, clFFT/gpyFFT, flint/arb/python-flint and tbb/mklfft. See the docker files to install those packages (`hysop/ci/docker_images/ubuntu`). Alternatively your can run hysop in an isolated environment by using docker containers (see next section).
# How to use docker images ?
Docker images can be pulled (resp. pushed) with `./ci/utils/pull_docker_image.sh [imgname]` and `./ci/utils/push_docker_image.sh [imgname]`. The docker images do not contain de hysop library and can be run with `./ci/utils/run_docker_image.sh [imgname]`. This script mounts your local hysop directory (read only) to `/hysop` inside the docker container and prompt a shell. Images have to be downloaded (pulled) prior to be run with this script.
By default, `[imgname]` corresponds to the docker image used for gitlab continuous integration (currently `jammy`, which corresponds to Ubuntu 22.04 running with python3.9). Docker images ship an intel OpenCL platform that is compatible with intel CPUs. Cuda enabled images may be a bit out-of-date (see `./ci/docker_images/ubuntu/bionic_cuda` to get started).
By default, `[imgname]` corresponds to the docker image used for gitlab continuous integration (currently `jammy`, which corresponds to Ubuntu 22.04 running with python3.10). Docker images without the `_cuda` postfix ship an intel OpenCL platform that is compatible with intel CPUs. Try out the `jammy_cuda` docker image to run on the GPUs (requires host system driver compatible with cuda 11.7).
To quickly test and/or debug hysop inside the docker you can run `./ci/utils/run_debug.sh [imgname]` which will build and install hysop inside the container and prompt a shell (read-only mount-bind `/hysop` directory is copied to read-write `/tmp/hysop` within the container). Alternatively if you just want to run tests inside docker, you can use the `./ci/utils/run_ci.sh [imgname]` script instead.
Docker images can be build locally by using the `./ci/utils/build_docker_image.sh` script. It is advised to build docker image only if the pull fails or if you want to add/update dependencies. Each build takes around one hour (36 cores @ 2GHz). By default, the build script will use all of your cores. At least 16GB of RAM is recommended.
There are equivalent `*.bat` scripts to run on Windows. Please note that the cuda enabled images currently do not work on Windows due to this [bug](https://github.com/microsoft/WSL/issues/6951).
## Known problems within docker containers
1. **OpenMPI complains about the program being run as root:**
You can solve this by defining two extra variables:
......
......@@ -15,7 +15,7 @@
## limitations under the License.
##
set -feu -o pipefail
PYTHON_EXECUTABLE=${PYTHON_EXECUTABLE:-python3.9}
PYTHON_EXECUTABLE=${PYTHON_EXECUTABLE:-python3}
MPIRUN_EXECUTABLE=${MPIRUN_EXECUTABLE:-mpirun}
MPIRUN_TASKS_OPTION='-np'
if [ "${MPIRUN_EXECUTABLE}" = "srun" ]; then MPIRUN_TASKS_OPTION='-n'; fi
......
......@@ -15,7 +15,7 @@
## limitations under the License.
##
set -x
PYTHON_EXECUTABLE=${PYTHON_EXECUTABLE:-python3.9}
PYTHON_EXECUTABLE=${PYTHON_EXECUTABLE:-python3}
MPIRUN_EXECUTABLE=${MPIRUN_EXECUTABLE:-mpirun}
MPIRUN_TASKS_OPTION='-np'
if [ "${MPIRUN_EXECUTABLE}" = "srun" ]; then MPIRUN_TASKS_OPTION='-n'; fi
......
......@@ -15,7 +15,7 @@
## limitations under the License.
##
set -feu -o pipefail
PYTHON_EXECUTABLE=${PYTHON_EXECUTABLE:-python3.9}
PYTHON_EXECUTABLE=${PYTHON_EXECUTABLE:-python3}
MPIRUN_EXECUTABLE=${MPIRUN_EXECUTABLE:-mpirun}
MPIRUN_TASKS_OPTION='-np'
if [ "${MPIRUN_EXECUTABLE}" = "srun" ]; then MPIRUN_TASKS_OPTION='-n'; fi
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment