README.md 4.16 KB
Newer Older
1

Geoffroy Lesur's avatar
Geoffroy Lesur committed
2

3
4
5
6
7
8
9
10
11
12
13
14
<!-- toc -->

- [Download:](#download)
- [Installation:](#installation)
- [Compile an example:](#compile-an-example)
- [Running](#running)
  * [serial (gpu/cpu), openMP (cpu)](#serial-gpucpu-openmp-cpu)
  * [With MPI (cpu)](#with-mpi-cpu)
  * [With MPI (gpu)](#with-mpi-gpu)
- [Profiling](#profiling)
- [Debugging](#debugging)
- [Running tests](#running-tests)
15
- [Contributing](#contributing)
16
17

<!-- tocstop -->
18

19
20
Download:
---------
Clément Robert's avatar
Clément Robert committed
21

Clément Robert's avatar
Clément Robert committed
22
using either ssh or https url as `<address>`
Clément Robert's avatar
Clément Robert committed
23
```shell
24
git clone <address> idefix
Clément Robert's avatar
Clément Robert committed
25
26
27
cd idefix
git submodule init
git submodule update
Clément Robert's avatar
Clément Robert committed
28
```
29

30
31
32
33
34
35
36
37
38
Installation:
-------------

Set the `IDEFIX_DIR` environment variable to the absolute path of the directory

```shell
export IDEFIX_DIR=<idefix main folder>
```

39
Add this line to `~/.<shell_rc_file>` for a permanent install.
40
41


42
43
Compile an example:
-------------------
Geoffroy Lesur's avatar
Geoffroy Lesur committed
44
45
Go to the example directory.
For instance:
46

Clément Robert's avatar
Clément Robert committed
47
48
49
```shell
cd test/HD/sod
```
50

Geoffroy Lesur's avatar
Geoffroy Lesur committed
51
Configure the code launching cmake (version >= 3.16) in the example directory:
Geoffroy Lesur's avatar
Geoffroy Lesur committed
52

Clément Robert's avatar
Clément Robert committed
53
```shell
Geoffroy Lesur's avatar
Geoffroy Lesur committed
54
cmake $IDEFIX_DIR
Clément Robert's avatar
Clément Robert committed
55
```
Geoffroy Lesur's avatar
Geoffroy Lesur committed
56

57
Several options can be enabled from the command line (a complete list is available with `cmake $IDEFIX_DIR -LH`). For instance: `-DIdefix_RECONSTRUCTION=Parabolic` (enable PPM reconstruction), `-DIdefix_MPI=ON` (enable mpi), `-DKokkos_ENABLE_OPENMP=ON` (enable openmp parallelisation), etc... For more complex target architectures, it is recommended to use cmake GUI launching `ccmake $IDEFIX_DIR` in place of `cmake` and then switching on the required options.
Geoffroy Lesur's avatar
Geoffroy Lesur committed
58
59
60

One can then compile the code:

Clément Robert's avatar
Clément Robert committed
61
```shell
Geoffroy Lesur's avatar
Geoffroy Lesur committed
62
make -j8
Clément Robert's avatar
Clément Robert committed
63
```
64
65
66

Running
-------------------
Geoffroy Lesur's avatar
Geoffroy Lesur committed
67
### serial (gpu/cpu), openMP (cpu)
68
69
simple launch the executable

Clément Robert's avatar
Clément Robert committed
70
71
72
```shell
./idefix
```
Geoffroy Lesur's avatar
Geoffroy Lesur committed
73
74

### With MPI (cpu)
75
76
77
78
`-dec` can be used to specify a domain decomposition manually.

It can be omitted for 1D problems, or if `NX`, `NY`, `NZ` and `nproc` are **all** powers of 2.
Otherwise, `-dec` is mandatory. For instance, in 2D, using a 2x2 domain decomposition:
Geoffroy Lesur's avatar
Geoffroy Lesur committed
79

Clément Robert's avatar
Clément Robert committed
80
81
82
```shell
mpirun -np 4 ./idefix -dec 2 2
```
Geoffroy Lesur's avatar
Geoffroy Lesur committed
83

84
or in 3D, using a 1x2x4 decomposition:
85

Clément Robert's avatar
Clément Robert committed
86
87
88
```shell
mpirun -np 8 ./idefix -dec 1 2 4
```
89

Geoffroy Lesur's avatar
Geoffroy Lesur committed
90
91
### With MPI (gpu)
The same rules for cpu domain decomposition applies for gpus. In addition, one should manually specify how many GPU devices one wants to use **per node**. Example, in a run with 2 nodes, 4 gpu per node, one would launch idefix with
92

Clément Robert's avatar
Clément Robert committed
93
```shell
94
mpirun -np 8 ./idefix -dec 1 2 4 --kokkos-num-devices=4
Clément Robert's avatar
Clément Robert committed
95
```
96
97
98
99

Profiling
-------------------
use the kokkos profiler tool, which can be downloaded from kokkos-tools (on github)
Geoffroy Lesur's avatar
Geoffroy Lesur committed
100
101
Then set the environement variable:

Clément Robert's avatar
Clément Robert committed
102
```shell
103
export KOKKOS_PROFILE_LIBRARY=kokkos-tools/src/tools/space-time-stack/kp_space_time_stack.so
Clément Robert's avatar
Clément Robert committed
104
````
Geoffroy Lesur's avatar
Geoffroy Lesur committed
105

106
107
and then simply run the code (no need to recompile)

Geoffroy Lesur's avatar
Geoffroy Lesur committed
108
109
Debugging
-------------------
110
111
Add `-DIdefix_DEBUG=ON` when calling cmake, or activate the `Idefix_DEBUG` option in ccmake, and recompile.
Note that this option triggers a lot of outputs and memory access checks which significantly slow down the code.
112

113
114
115
116
Running tests
-------------------
Tests require Python 3 along with some third party dependencies to be installed.
To install those deps, run
Clément Robert's avatar
Clément Robert committed
117
```shell
118
119
120
121
pip install -r test/python_requirements.txt
```

The test suite itself is then run with
Clément Robert's avatar
Clément Robert committed
122
```shell
123
124
bash test/checks.sh
```
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147

Contributing
-------------------
Idefix is developed with the help of the [pre-commit](https://pre-commit.com) framework.
We use [cpplint](https://en.wikipedia.org/wiki/Cpplint) to validate code style, mostly
following the Google standards for C++, and several pre-commit hooks to automatically fix
some coding bad practices.

It is recommended (though not mandatory) to install pre-commit by running the following
script from the top level of the repo
```shell
python3 -m pip install pre-commit
pre-commit install
```

Then, as one checks in their contribution with `git commit`, pre-commit hooks may perform
changes in situ. One then needs to re-add and enter the `git commit` command again for the
commit to be validated.
Note that an important hook that does _not_ perform auto-fixes is `cpplint`, so contributors
need to accomodate for this one by hand.

> Note that if for any reason you do not wish, or are unable to install pre-commit in your
> environment, formatting errors will be caught by our CI after you open a merge-request.