Commit 544d558e authored by paugier's avatar paugier
Browse files

Add IO bounded in parallel

parent e160c46a
......@@ -40,6 +40,53 @@
"- IO bounded (wait for IO)"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"## Different parallel strategies"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "-"
}
},
"source": [
"### IO bounded: one process + `async`/`await`\n",
"\n",
"<p class=\"small\"><br></p>\n",
"\n",
"Cooperative concurrency\n",
"\n",
"<p class=\"small\"><br></p>\n",
"\n",
"Functions able to pause\n",
"\n",
"<p class=\"small\"><br></p>\n",
"\n",
"`asyncio`, [`trio`](https://trio.readthedocs.io)"
]
},
{
"cell_type": "raw",
"metadata": {
"slideshow": {
"slide_type": "-"
}
},
"source": [
"<div align=\"middle\">\n",
"<img src=\"./fig/logo_trio.png\" style=\"width: 15%\">\n",
"</div>"
]
},
{
"cell_type": "markdown",
"metadata": {
......@@ -164,7 +211,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.7.2"
"version": "3.7.3"
}
},
"nbformat": 4,
......
%% Cell type:markdown id: tags:
# Parallel computing (CPU bounded)
Pierre Augier (LEGI), Cyrille Bonamy (LEGI), Eric Maldonado (Irstea), Franck Thollard (ISTerre), Christophe Picard (LJK), Loïc Huder (ISTerre)
%% Cell type:markdown id: tags:
# Parallelism
## Hardware:
- Multicore CPU
- Multi nodes super computers (MPI)
- GPU (Nvidia: Cuda, Cupy) / Intel Xeon Phi
<p class="small"><br></p>
## Different problems
- CPU bounded (need to use cores at the same time)
- IO bounded (wait for IO)
%% Cell type:markdown id: tags:
## Different parallel strategies
%% Cell type:markdown id: tags:
### IO bounded: one process + `async`/`await`
<p class="small"><br></p>
Cooperative concurrency
<p class="small"><br></p>
Functions able to pause
<p class="small"><br></p>
`asyncio`, [`trio`](https://trio.readthedocs.io)
%% Cell type:raw id: tags:
<div align="middle">
<img src="./fig/logo_trio.png" style="width: 15%">
</div>
%% Cell type:markdown id: tags:
## Different parallel strategies
### One process split in light subprocesses called threads 👩🏼‍🔧 👨🏼‍🔧👩🏼‍🔧 👨🏼‍🔧
- handled by the OS
- share memory and can use at the same time different CPU cores
How?
- OpenMP (Natively in C / C++ / Fortran. For Python: Pythran, Cython, ...)
- In Python: `threading` and `concurrent.futures`
%% Cell type:markdown id: tags:
**⚠️ in Python, one interpreter per process (~) and the Global Interpreter Lock (GIL)...**
- In a Python program, different threads can run at the same time (and take advantage of multicore)
- But... the Python interpreter runs the Python bytecodes sequentially !
- *Terrible* 🐌 for CPU bounded *if the Python interpreter is used a lot* !
- *No problem* for IO bounded !
- *No problem* for extensions when the interpreter is not used !
%% Cell type:markdown id: tags:
## Different parallel strategies
### One program, $n$ processes 👩🏼‍🔧 👨🏼‍🔧👩🏼‍🔧 👨🏼‍🔧
### Exchange data:
Very efficient and no problem with Python!
- With MPI: mpi4py and h5py parallel
- multiprocessing
- [ZeroMQ](http://zeromq.org/)
%% Cell type:markdown id: tags:
## Other packages for parallel computing with Python
- IPython
- dask
- joblib
%% Cell type:markdown id: tags:
# Code examples
## Target algorithms :
- computes the first prime number (with a **very** brute force algorithm)
- given a set of number, computes the distance map between each number
- computes for a set of number, its factors
## Solutions
- multiprocessing (map)
- multiprocessing (starmap)
- concurent.futures
- ...
......
Supports Markdown
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment