Skip to content
GitLab
Menu
Projects
Groups
Snippets
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in
Toggle navigation
Menu
Open sidebar
python-uga
training-hpc
Commits
544d558e
Commit
544d558e
authored
Jun 18, 2019
by
paugier
Browse files
Add IO bounded in parallel
parent
e160c46a
Changes
1
Hide whitespace changes
Inline
Side-by-side
ipynb/40_parallel.ipynb
View file @
544d558e
...
...
@@ -40,6 +40,53 @@
"- IO bounded (wait for IO)"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"## Different parallel strategies"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "-"
}
},
"source": [
"### IO bounded: one process + `async`/`await`\n",
"\n",
"<p class=\"small\"><br></p>\n",
"\n",
"Cooperative concurrency\n",
"\n",
"<p class=\"small\"><br></p>\n",
"\n",
"Functions able to pause\n",
"\n",
"<p class=\"small\"><br></p>\n",
"\n",
"`asyncio`, [`trio`](https://trio.readthedocs.io)"
]
},
{
"cell_type": "raw",
"metadata": {
"slideshow": {
"slide_type": "-"
}
},
"source": [
"<div align=\"middle\">\n",
"<img src=\"./fig/logo_trio.png\" style=\"width: 15%\">\n",
"</div>"
]
},
{
"cell_type": "markdown",
"metadata": {
...
...
@@ -164,7 +211,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.7.
2
"
"version": "3.7.
3
"
}
},
"nbformat": 4,
...
...
%% Cell type:markdown id: tags:
# Parallel computing (CPU bounded)
Pierre Augier (LEGI), Cyrille Bonamy (LEGI), Eric Maldonado (Irstea), Franck Thollard (ISTerre), Christophe Picard (LJK), Loïc Huder (ISTerre)
%% Cell type:markdown id: tags:
# Parallelism
## Hardware:
-
Multicore CPU
-
Multi nodes super computers (MPI)
-
GPU (Nvidia: Cuda, Cupy) / Intel Xeon Phi
<p
class=
"small"
><br></p>
## Different problems
-
CPU bounded (need to use cores at the same time)
-
IO bounded (wait for IO)
%% Cell type:markdown id: tags:
## Different parallel strategies
%% Cell type:markdown id: tags:
### IO bounded: one process + `async`/`await`
<p
class=
"small"
><br></p>
Cooperative concurrency
<p
class=
"small"
><br></p>
Functions able to pause
<p
class=
"small"
><br></p>
`asyncio`
,
[
`trio`
](
https://trio.readthedocs.io
)
%% Cell type:raw id: tags:
<div
align=
"middle"
>
<img
src=
"./fig/logo_trio.png"
style=
"width: 15%"
>
</div>
%% Cell type:markdown id: tags:
## Different parallel strategies
### One process split in light subprocesses called threads 👩🏼🔧 👨🏼🔧👩🏼🔧 👨🏼🔧
-
handled by the OS
-
share memory and can use at the same time different CPU cores
How?
-
OpenMP (Natively in C / C++ / Fortran. For Python: Pythran, Cython, ...)
-
In Python:
`threading`
and
`concurrent.futures`
%% Cell type:markdown id: tags:
**⚠️ in Python, one interpreter per process (~) and the Global Interpreter Lock (GIL)...**
-
In a Python program, different threads can run at the same time (and take advantage of multicore)
-
But... the Python interpreter runs the Python bytecodes sequentially !
- *Terrible* 🐌 for CPU bounded *if the Python interpreter is used a lot* !
- *No problem* for IO bounded !
- *No problem* for extensions when the interpreter is not used !
%% Cell type:markdown id: tags:
## Different parallel strategies
### One program, $n$ processes 👩🏼🔧 👨🏼🔧👩🏼🔧 👨🏼🔧
### Exchange data:
Very efficient and no problem with Python!
-
With MPI: mpi4py and h5py parallel
-
multiprocessing
-
[
ZeroMQ
](
http://zeromq.org/
)
%% Cell type:markdown id: tags:
## Other packages for parallel computing with Python
-
IPython
-
dask
-
joblib
%% Cell type:markdown id: tags:
# Code examples
## Target algorithms :
-
computes the first prime number (with a
**very**
brute force algorithm)
-
given a set of number, computes the distance map between each number
-
computes for a set of number, its factors
## Solutions
-
multiprocessing (map)
-
multiprocessing (starmap)
-
concurent.futures
-
...
...
...
Write
Preview
Supports
Markdown
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment