Skip to content
Snippets Groups Projects
Commit 5cdc0505 authored by EXT Jean-Matthieu Etancelin's avatar EXT Jean-Matthieu Etancelin
Browse files

Merge branch '14-development-documentation-2' into 'master'

Fixup documentation

Closes #14

See merge request !39
parents ac4e89bf d65741a0
No related branches found
No related tags found
1 merge request!39Fixup documentation
Pipeline #109911 passed
...@@ -29,7 +29,7 @@ macro(add_python_test test_name test_file) ...@@ -29,7 +29,7 @@ macro(add_python_test test_name test_file)
set_tests_properties(${test_name} PROPERTIES FAIL_REGULAR_EXPRESSION "FAILURE;Exception;[^x]failed;ERROR;Assertion") set_tests_properties(${test_name} PROPERTIES FAIL_REGULAR_EXPRESSION "FAILURE;Exception;[^x]failed;ERROR;Assertion")
set_tests_properties(${test_name} PROPERTIES WORKING_DIRECTORY ${CMAKE_BINARY_DIR}/dataForTests) set_tests_properties(${test_name} PROPERTIES WORKING_DIRECTORY ${CMAKE_BINARY_DIR}/dataForTests)
set_tests_properties(${test_name} PROPERTIES ENVIRONMENT "PYTHONPATH=$ENV{PYTHONPATH}:${HYSOP_BUILD_PYTHONPATH}") set_tests_properties(${test_name} PROPERTIES ENVIRONMENT "PYTHONPATH=$ENV{PYTHONPATH}:${HYSOP_BUILD_PYTHONPATH}")
set_tests_properties(${test_name} PROPERTIES ENVIRONMENT "PYTHONPATH=$ENV{PYTHONPATH}:${HYSOP_BUILD_PYTHONPATH},LD_LIBRARY_PATH=$ENV{LD_LIBRARY_PATH}:${HYSOP_BUILD_PYTHONPATH}") set_tests_properties(${test_name} PROPERTIES ENVIRONMENT "PYTHONPATH=$ENV{PYTHONPATH}:${HYSOP_BUILD_PYTHONPATH},LD_LIBRARY_PATH=$ENV{LD_LIBRARY_PATH}:${HYSOP_BUILD_PYTHONPATH},HYSOP_ENABLE_LONG_TESTS=${ENABLE_LONG_TESTS}")
set_tests_properties(${test_name} PROPERTIES TIMEOUT ${TEST_TIMEOUT}) set_tests_properties(${test_name} PROPERTIES TIMEOUT ${TEST_TIMEOUT})
if(WITH_MPI_TESTS) if(WITH_MPI_TESTS)
......
...@@ -37,15 +37,27 @@ extensions = [ ...@@ -37,15 +37,27 @@ extensions = [
'sphinx.ext.mathjax', 'sphinx.ext.mathjax',
'sphinx.ext.ifconfig', 'sphinx.ext.ifconfig',
'sphinx.ext.viewcode', 'sphinx.ext.viewcode',
'numpydoc', 'sphinx.ext.napoleon',
'sphinx.ext.intersphinx', 'sphinxcontrib.bibtex', 'sphinx.ext.intersphinx', 'sphinxcontrib.bibtex',
'sphinx.ext.inheritance_diagram', 'sphinx.ext.inheritance_diagram',
'sphinx.ext.todo', 'sphinx.ext.todo',
'sphinxcontrib.doxylink' 'sphinxcontrib.doxylink']
]
sys.path.insert(0, "@HYSOP_BUILD_PYTHONPATH@") sys.path.insert(0, "@HYSOP_BUILD_PYTHONPATH@")
numpydoc_class_members_toctree = True numpydoc_class_members_toctree = True
numpydoc_show_class_members = False
# Napoleon settings
napoleon_google_docstring = False
napoleon_numpy_docstring = True
napoleon_include_init_with_doc = False
napoleon_include_private_with_doc = False
napoleon_include_special_with_doc = True
napoleon_use_admonition_for_examples = False
napoleon_use_admonition_for_notes = False
napoleon_use_admonition_for_references = False
napoleon_use_ivar = False
napoleon_use_param = True
napoleon_use_rtype = True
# ----------------------------------------------------------------------------- # -----------------------------------------------------------------------------
# Autosummary # Autosummary
# ----------------------------------------------------------------------------- # -----------------------------------------------------------------------------
...@@ -59,7 +71,7 @@ doxylink = { ...@@ -59,7 +71,7 @@ doxylink = {
'doxygen/html'), 'doxygen/html'),
} }
autoclass_content = 'init' autoclass_content = 'both'
# Add any paths that contain templates here, relative to this directory. # Add any paths that contain templates here, relative to this directory.
templates_path = ['_templates'] templates_path = ['_templates']
...@@ -137,6 +149,7 @@ todo_include_todos = True ...@@ -137,6 +149,7 @@ todo_include_todos = True
# a list of builtin themes. # a list of builtin themes.
#html_theme = 'hysop' #html_theme = 'hysop'
html_theme = 'bootstrap' html_theme = 'bootstrap'
#html_theme = 'sphinx_rtd_theme'
# Add any paths that contain custom themes here, relative to this directory. # Add any paths that contain custom themes here, relative to this directory.
html_theme_path = ['@CMAKE_CURRENT_SOURCE_DIR@/sphinx/themes'] html_theme_path = ['@CMAKE_CURRENT_SOURCE_DIR@/sphinx/themes']
...@@ -264,6 +277,8 @@ latex_documents = [ ...@@ -264,6 +277,8 @@ latex_documents = [
# If false, no module index is generated. # If false, no module index is generated.
# latex_domain_indices = True # latex_domain_indices = True
# Bilbiography (bibtex)
bibtex_bibfiles = ['hysop.bib']
# -- Options for manual page output --------------------------------------- # -- Options for manual page output ---------------------------------------
...@@ -309,7 +324,6 @@ intersphinx_mapping = {'python': ('https://docs.python.org', None), ...@@ -309,7 +324,6 @@ intersphinx_mapping = {'python': ('https://docs.python.org', None),
'numpy': ('http://docs.scipy.org/doc/numpy/', None), 'numpy': ('http://docs.scipy.org/doc/numpy/', None),
'scipy': ('http://docs.scipy.org/doc/scipy/reference/', None), 'scipy': ('http://docs.scipy.org/doc/scipy/reference/', None),
'matplotlib': ('http://matplotlib.sourceforge.net/', None)} 'matplotlib': ('http://matplotlib.sourceforge.net/', None)}
numpydoc_show_class_members = False
#----------------------------------------------------------------------------- #-----------------------------------------------------------------------------
# Coverage checker # Coverage checker
# ----------------------------------------------------------------------------- # -----------------------------------------------------------------------------
......
#####################
HySoP manual contents
#####################
.. toctree::
user_guide/index
reference/index
examples/index
about
license
.. _about_codegen:
Code generation and autotuning
==============================
To Be done
see
:class:`~hysop.backend.device.autotunable_kernel.AutotunableKernel`
and hysop.symbolic
...@@ -3,8 +3,14 @@ ...@@ -3,8 +3,14 @@
How to document classes in HySoP How to document classes in HySoP
================================= =================================
We try to fit as much as possible with `numpydoc guide <https://numpydoc.readthedocs.io/en/latest/>` We try to fit as much as possible with `numpydoc guide <https://numpydoc.readthedocs.io/en/latest/>`_
guide standards. guide standards.
See the files fields/continuous.py and operator/drag_and_lift.py which are used as reference files for documentation See the files fields/continuous.py and operator/drag_and_lift.py which
and are used as reference files for documentation.
Mind the need of `raw strings litterals <https://docs.python.org/3/reference/lexical_analysis.html#string-and-bytes-literals>`_ in python docstrings for keeping the `\stuff` from math
environments `".. :math:"`.
Mind the 'space' character before colon character `":"` in parameters or
attributes definitions.
.. _about_work_arrays:
Temporary work arays in HySoP
=============================
TO BE DONE
See :class:`~hysop.core.memory.memory_request.MemoryRequest`
.. _coding_guidelines:
How to write proper code in HySoP
=================================
Developper should follow python recommandations for code style.
Lower-level functions should be tested in the appropriate unitary
tests folders.
Higher-level functionalities should be exposed in an example file.
.. _hysop_validation:
HySoP validation
================
To be done
...@@ -16,7 +16,5 @@ This part is for developers of Hysop and provides guidelines on the "proper" way ...@@ -16,7 +16,5 @@ This part is for developers of Hysop and provides guidelines on the "proper" way
about_operators_graph about_operators_graph
about_work_arrays about_work_arrays
about_codegen about_codegen
about_autotuning
about_symbolic
memo_sphinx memo_sphinx
hysop_validation hysop_validation
...@@ -5,8 +5,71 @@ ...@@ -5,8 +5,71 @@
%% Saved with string encoding Unicode (UTF-8) %% Saved with string encoding Unicode (UTF-8)
@article{etancelin:hal-02965451,
TITLE = {{Improvement of remeshed Lagrangian methods for the simulation of dissolution processes at pore-scale}},
AUTHOR = {Etancelin, Jean-Matthieu and Moonen, Peter and Poncet, Philippe},
URL = {https://hal.archives-ouvertes.fr/hal-02965451},
JOURNAL = {{Advances in Water Resources}},
PUBLISHER = {{Elsevier}},
VOLUME = {146},
PAGES = {103780},
YEAR = {2020},
MONTH = Oct,
DOI = {10.1016/j.advwatres.2020.103780},
KEYWORDS = {Three-dimensional flows ; Digital Rock Physics ; Reactive flows ; Dissolution process ; Lagrangian methods ; Porous media},
PDF = {https://hal.archives-ouvertes.fr/hal-02965451/file/10Final-AWR-HAL.pdf},
HAL_ID = {hal-02965451},
HAL_VERSION = {v1},
}
@phdthesis{keck:tel-02433509,
TITLE = {{Numerical modelling and High Performance Computing for sediment flows}},
AUTHOR = {Keck, Jean-Baptiste},
URL = {https://tel.archives-ouvertes.fr/tel-02433509},
NUMBER = {2019GREAM067},
SCHOOL = {{Universit{\'e} Grenoble Alpes}},
YEAR = {2019},
MONTH = Dec,
KEYWORDS = {Sediment transport ; Dual-Way coupling ; Particle methods ; Double diffusivity ; Hpc ; Gpu ; Transport s{\'e}dimentaire ; Couplage bidirectonnel ; M{\'e}thodes particulaires ; Double diffusivit{\'e} ; Hpc ; Gpu},
TYPE = {Theses},
PDF = {https://tel.archives-ouvertes.fr/tel-02433509v2/file/KECK_2019_diffusion.pdf},
HAL_ID = {tel-02433509},
HAL_VERSION = {v2},
}
@article{mimeau:hal-01483400,
TITLE = {{Passive control of the flow around a hemisphere using porous media}},
AUTHOR = {Mimeau, Chlo{\'e} and Mortazavi, Iraj and Cottet, Georges-Henri},
URL = {https://hal.archives-ouvertes.fr/hal-01483400},
JOURNAL = {{European Journal of Mechanics - B/Fluids}},
PUBLISHER = {{Elsevier}},
VOLUME = {65},
PAGES = {213--226},
YEAR = {2017},
MONTH = Sep,
DOI = {10.1016/j.euromechflu.2017.03.002},
KEYWORDS = {vortex methods ; flow past a hemisphere ; passive flow control ; porous media ; drag reduction ; Brinkman penalization method},
PDF = {https://hal.archives-ouvertes.fr/hal-01483400/file/paper_EJM.pdf},
HAL_ID = {hal-01483400},
HAL_VERSION = {v1},
}
@article{keck:hal-02982080,
TITLE = {{Double-diffusive sedimentation at high Schmidt numbers: Semi-Lagrangian simulations}},
AUTHOR = {Keck, Jean-Baptiste and Cottet, Georges-Henri and Meiburg, Eckart and Mortazavi, Iraj and Picard, Christophe},
URL = {https://hal.archives-ouvertes.fr/hal-02982080},
JOURNAL = {{Physical Review Fluids}},
PUBLISHER = {{American Physical Society}},
VOLUME = {6},
NUMBER = {2},
PAGES = {L022301},
YEAR = {2021},
MONTH = Feb,
DOI = {10.1103/PhysRevFluids.6.L022301},
PDF = {https://hal.archives-ouvertes.fr/hal-02982080v2/file/paper_rev.pdf},
HAL_ID = {hal-02982080},
HAL_VERSION = {v2},
}
@phdthesis{Chloe-thesis, @phdthesis{Chloe-thesis,
Author = {Chlo{\'e} Mimeau}, Author = {Chlo{\'e} Mimeau},
......
...@@ -9,9 +9,13 @@ HySoP User Guide ...@@ -9,9 +9,13 @@ HySoP User Guide
.. toctree:: .. toctree::
:maxdepth: 8 :maxdepth: 8
introduction
space space
fields fields
operators operators
operators_io
topologies
numerics numerics
tools tools
tasks
biblio biblio
hysop
=======
.. toctree::
:maxdepth: 4
.. _tasks:
About tasks management in HySoP
===============================
First note that tasks meaning in HySoP is not for well known "task
parallelism".
Whithout tasks, all MPI processes are participating in all operations
in the simulation. Introducing tasks enable to subdivise the set of
processes for some operators in the graph.
There are three main use case:
- One of the operators has only a sequential implementation while
others can benefit from MPI parallelism
- Some operators are using only GPUs and others are using CPU cores
and user wants to use all the resources of a compute note without
sharing a GPU among several MPI processes.
- On-the-fly post-processing of the data on dedicated resources
Setup tasks in HySoP is a 3 steps process:
1. specify a `proc_tasks` array to hysop.domain.domain object
creation.
2. create the hysop.MPI_Params using task specification
`task_id` **and** `on_task`.
3. create the operators with the appropriate MPI_Params. We strongly
advise to check the produced graph on the processes output (using
HYSOP_VERBOSE) especially for the automatically inserted InterTasksRedistributes
Details and limitations
-----------------------
1. A tasks is identified by an integer (`HYSOP_DEFAULT_TASK_ID` by
default). Each process is involved in one or several tasks. For
instance with 4 processes and 2 tasks (1 and 2):
* One task per rank (disjoint tasks)
proc_tasks = (1,2,2,2)
proc_tasks = (2,1,2,2)
proc_tasks = (1,2,1,2)
* rank0 work on both tasks. In this case all items must be
iterable. For the moment only nested tasks with the same root
process is possible with largest tasks first
proc_tasks = ((2,1),(2,),(2,),(2,))
proc_tasks = ((2,1),(2,),(2,1),(2,))
The following are not handled yet:
proc_tasks = ((1,2),(2,),(2,),(2,)) # largest task not in front for rank0
proc_tasks = ((2,),(2,1),(2,),(2,)) # root process is not the same for all tasks
proc_tasks = ((2,1),(2,),(1,),(2,)) # non-nested tasks
proc_tasks = ((1,),(1,),(1,2),(2,)) # non-nested tasks
In case of nested tasks, the `domain.current_task()` is always the
first task in the tasks list of a process.
**Note** configurations with more than 2 tasks have not been tested.
2. In case of disjoint tasks, a single `MPI_Params` still can be used:
box = Box(proc_tasks=proc_tasks)
mpi_params = MPIParams(comm=box.task_comm, task_id=box.current_task(), on_task=True)
Otherwise one `MPI_Params` per task is recommanded:
for t in box.all_tasks:
mpi_params[t] = MPIParams(comm=box.get_task_comm(t), task_id=t, on_task=box.is_on_task(t))
3. Automatic Inter-Tasks communications is done on graph
building. Consider the following situation with 4 operators (op1,
op2, op3 and endA) working on 3 fields (A, B anc C) and 2 tasks
(taskA and taskB)
# TaskA | TaskB
# op1 A->A |
# op2 A->B |
# `--B--, // Inter-task communication step
# | op3 B,C->A
# ,--A--' // Inter-task communication step
# endA A->A |
Note : without endA operator, the second communication is not
automatically inserted because A field is an output for task A
and cannot be invalidated by op3 on other task. Inter-task
invalidation in graph building is not yet implemented
Note this algorithm is not working with overlapping tasks
This is due to the fact that A is both input and output in taskA.
A workaround is to use an other field A' and endA is then a copy
Automatic communications are inserted on task change through operator
list. A search for appropriate field is done iterating trough previous
and next operators.
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment