Commit 371d6d1a authored by paugier's avatar paugier
Browse files

Add notebooks + better index page

parent 793b1824
Pipeline #49900 failed with stage
in 24 seconds
......@@ -16,4 +16,6 @@ ipynb/index.html
\ No newline at end of file
......@@ -2,6 +2,9 @@ IPYNBDIR := ipynb
IPYNBFILES := $(shell find $(IPYNBDIR) -name '*.ipynb' | grep -v ipynb_checkpoints)
IPYNBPRES = $(addsuffix .slides.html, $(basename $(IPYNBFILES)))
# TPL_FILE = ipynb/slides_reveal_wide.tpl
TPL_FILE = ipynb/slides_reveal.tpl
define STR_HELP
This makefile can be used for
......@@ -17,7 +20,7 @@ endef
export STR_HELP
.PHONY: help lab serve presentations
.PHONY: help lab serve presentations ipynb/index.rst
@echo "$$STR_HELP"
......@@ -25,18 +28,22 @@ help:
rm -f ipynb/*.slides.html
rm -f ipynb/index.html
rm -rf ipynb/tmp
python3 ipynb/
ipynb/index.html: ipynb/index.rst
cd $(IPYNBDIR) && rst2html5 index.rst > index.html
%.slides.html: %.ipynb ipynb/slides_reveal_wide.tpl
jupyter-nbconvert $< --reveal-prefix='./reveal.js' --to slides --template ipynb/slides_reveal_wide.tpl
%.slides.html: %.ipynb $(TPL_FILE)
jupyter-nbconvert $< --reveal-prefix='./reveal.js' --to slides --template $(TPL_FILE)
presentations: $(IPYNBPRES) ipynb/index.html
presentations: ipynb/index.html $(IPYNBPRES)
@echo $(IPYNBPRES)
serve: $(IPYNBPRES) ipynb/index.html
cd $(IPYNBDIR) && python3 -m http.server 9000
cd $(IPYNBDIR) && python3 -m http.server
This diff is collapsed.
%% Cell type:code id: tags:
``` python
# this ipython command loads a special backend for notebooks
%matplotlib notebook
import matplotlib.pyplot as plt
import numpy as np
from IPython.core.display import display as display_base, HTML
import mistune
markdown = mistune.Markdown()
def display(code):
return display_base(HTML(code))
%% Cell type:markdown id: tags:
# Overview of the Python HPC landscape 🚀
%% Cell type:markdown id: tags:
<div align="middle"> Python, only a great glue language? </div>
%% Cell type:markdown id: tags:
<div align="middle"> Pierre Augier </div>
%% Cell type:raw id: tags:
<div align="middle">
<img src="./fig/logo_CNRS.jpg" style="width: 10%"> &nbsp; &nbsp;
<img src="./fig/logo_LEGI.jpg" style="width: 15%"> &nbsp; &nbsp;
<img src="./fig/logo-uga-vo-cmjn.jpg" style="width: 15%">
%% Cell type:markdown id: tags:
# Python for High-Performance Computing?
1. *Fast prototyping* (Numpy!)
2. *Popular*:
- Well-known
- Several great libraries
3. *Share ideas* between developers / scientists
- Popularity counts
- Readability counts
- Expressivity counts
4. Anyway, one needs a good and well-known *scripting language* so yes!
(even considering Julia)
### *Where / when should we stop ?*
%% Cell type:markdown id: tags:
# Python & fast prototyping...
## The software engineering method for scientists 👩‍🔬 👨‍🔬 and HPC
1. Fast prototyping
2. Solidify **as needed**
* ↗ code quality
* [testing](
* refactoring
* [type hints]( and [type checking](
* ↗ performances for bottlenecks (profile ⏱ !)
Again and again: `(1, 2)`, `(1, 2)`, ...
%% Cell type:markdown id: tags:
## Python: a programming language, compromises ⚖️
Designed for fast prototyping & "glue" codes together
- Generalist + easy to learn ⇒ huge and diverse community 👨🏿‍🎓🕵🏼 👩🏼‍🎓 👩🏽‍🏫👨🏽‍💻👩🏾‍🔬 🎅🏼 🌎 🌍 🌏
- Expressivity and readability
- Not oriented towards high performance
(fast and easy dev, easy debug, correctness)
- Highly dynamic 🐒 + introspection (`inspect.stack()`)
- Automatic memory management 💾
- All objects encapsulated 🥥 (PyObject, C struct)
- Objects accessible through "references" ➡️
- Usually interpreted
%% Cell type:markdown id: tags:
## Python interpreters
### [CPython]( <img src="./fig/logo_python.png" align="left" style="width: 7%; margin-bottom: 2px; margin-right: 5px;">
Interpreted (nearly) instruction per instruction, (nearly) no code optimization
The numerical stack (Numpy, Scipy, Scikits, ...) based on the CPython C API (CPython implementation details)!
### [PyPy]( <img src="./fig/logo_pypy.png" align="left" style="width: 7%; margin-bottom: 0px; margin-right: 5px;">
Optimized implementation with tracing Just-In-Time compilation
["Abstractions for free"](
The CPython C API is an issue! PyPy can't accelerate Numpy code!
### [Micropython]( <img src="./fig/logo_micropython.jpeg" align="left" style="width: 7%; margin-bottom: 0px; margin-right: 5px;">
For microcontrollers
%% Cell type:markdown id: tags:
## Python & performance
### References and PyObjects
%% Cell type:code id: tags:
``` python
mylist = [1, 3, 5]
%% Cell type:markdown id: tags:
`list`: array of references towards PyObjects
%% Cell type:markdown id: tags:
### The C / Python border
%% Cell type:code id: tags:
``` python
arr = 2 * np.arange(10)
%% Cell type:markdown id: tags:
## Python & performance
### Python interpreters bad at crunching numbers
%% Cell type:markdown id: tags:
Pure Python terrible 🐢 (except with PyPy)...
%% Cell type:code id: tags:
``` python
from math import sqrt
my_const = 10.
result = [elem * sqrt(my_const * 2 * elem**2) for elem in range(1000)]
%% Cell type:markdown id: tags:
but even this is not *very* efficient (temporary objects)...
%% Cell type:code id: tags:
``` python
import numpy as np
a = np.arange(1000)
result = a * np.sqrt(my_const * 2 * a**2)
%% Cell type:markdown id: tags:
Even slightly worth with PyPy 🙁
%% Cell type:markdown id: tags:
## Is Python efficient enough?
%% Cell type:markdown id: tags:
Python is known to be slow... But what does it mean?
<p class="small"><br></p>
### Efficiency / inefficiency: depends on tasks ⏱
<p class="small"><br></p>
### When is it inefficient? Especially for number crunching 🔢 ...
<p class="small"><br></p>
### Can we write efficient scientific code in 🐍 ?
%% Cell type:markdown id: tags:
## Book
%% Cell type:raw id: tags:
<div align="middle">
<img src="./fig/high-perf-python-book.jpg" style="width: 35%; margin-top: 0px;">
%% Cell type:markdown id: tags:
See also
%% Cell type:markdown id: tags:
# Performance (generalities)
### Measure ⏱, don't guess! Profile to find the bottlenecks.
Cprofile (pstats, SnakeViz), line-profiler, [`perf`](, [perf_events](
<p class="small"><br></p>
### Do not optimize everything!
- *"Premature optimization is the root of all evil"* (Donald Knuth)
- 80 / 20 rule, efficiency important for expensive things and NOT for small things
<p class="small"><br></p>
### CPU or IO bounded problems
<p class="small"><br></p>
### Use the *right algorithms* and the right data structures!
For example, using Numpy arrays instead of Python lists...
<p class="small"><br></p>
### Unittests before optimizing to maintain correctness!
`unittest`, `pytest`
%% Cell type:markdown id: tags:
## "Crunching numbers" and computers architectures
%% Cell type:raw id: tags:
<div align="middle">
<img src="./fig/modern-hardware-1.png" style="width: 30%; margin-top: -30px; margin-bottom: -50px"> &nbsp;
%% Cell type:markdown id: tags:
### CPU optimizations
- pipelining, hyper-threading, vectorization, advanced instructions (simd), ...
- important to get data aligned in memory (arrays)
%% Cell type:markdown id: tags:
# Proper compilation needed for high efficiency !
<p class="small"><br></p>
## Compilation to virtual machine instructions
What does CPython (`compile`, "byte code", nearly no optimization, see `dis` module)
<p class="small"><br></p>
## Compilation to machine instructions
- Just-in-time
Has to be fast (warm up), can be hardware specific
- Ahead-of-time
Can be slow, hardware specific or more general to distribute binaries
*Compilers are usually good for optimizations!* Better than most humans...
<p class="small"><br></p>
## Transpilation
From one language to another language (for example Python to C++)
%% Cell type:markdown id: tags:
# Parallelism
## Hardware:
- Multicore CPU
- Multi nodes super computers (MPI)
- GPU (Nvidia: Cuda, Cupy) / Intel Xeon Phi
<p class="small"><br></p>
## Different problems
- CPU bounded (need to use cores at the same time)
- IO bounded (wait for IO)
%% Cell type:markdown id: tags:
## Different parallel strategies
%% Cell type:markdown id: tags:
### IO bounded: one process + `async`/`await`
<p class="small"><br></p>
Cooperative concurrency
<p class="small"><br></p>
Functions able to pause
<p class="small"><br></p>
`asyncio`, [`trio`](
%% Cell type:raw id: tags:
<div align="middle">
<img src="./fig/logo_trio.png" style="width: 15%">
%% Cell type:markdown id: tags:
## Different parallel strategies
### One process split in light subprocesses called threads 👩🏼‍🔧 👨🏼‍🔧👩🏼‍🔧 👨🏼‍🔧
- handled by the OS
- share memory and can use at the same time different CPU cores
- OpenMP (Natively in C / C++ / Fortran. For Python: Pythran, Cython, ...)
- In Python: `threading` and `concurrent.futures`
%% Cell type:markdown id: tags:
**⚠️ in Python, one interpreter per process (~) and the Global Interpreter Lock (GIL)...**
- In a Python program, different threads can run at the same time (and take advantage of multicore)
- But... the Python interpreter runs the Python bytecodes sequentially !
- *Terrible* 🐌 for CPU bounded *if the Python interpreter is used a lot* !
- *No problem* for IO bounded !
%% Cell type:markdown id: tags:
## Different parallel strategies
### One program, $n$ processes 👩🏼‍🔧 👨🏼‍🔧👩🏼‍🔧 👨🏼‍🔧
### Exchange data:
Very efficient and no problem with Python!
- With MPI: mpi4py and h5py parallel
- multiprocessing
- [ZeroMQ](
%% Cell type:markdown id: tags:
## 2 other packages for parallel computing with Python
- dask
- joblib
%% Cell type:markdown id: tags:
# Python for HPC: first *a glue language*
Many tools to interact with static languages:
&emsp;&emsp;&emsp;&emsp; ctypes, cffi, cython, cppyy, pybind11, f2py, pyo3, ...
%% Cell type:markdown id: tags:
Glue together pieces of native code (C, Fortran, C++, Rust, ...) with a nice syntax
&emsp;&emsp;&emsp;&emsp; ⇒ Numpy, Scipy, ...
%% Cell type:markdown id: tags:
- Numpy: great syntax for expressing algorithms, (nearly) as much information as in Fortran
- Performance of `a @ b` (Numpy) versus `a * b` (Julia)?
%% Cell type:markdown id: tags:
&emsp;&emsp;&emsp;&emsp;&emsp;&emsp;&emsp;&emsp; Same! The same library is called! (often OpenBlas or MKL)
%% Cell type:markdown id: tags:
## General principle for perf with Python (not fully valid for PyPy):
Don't use too often the Python interpreter (and small Python objects) for computationally demanding tasks.
%% Cell type:markdown id: tags:
Pure Python
&emsp;&emsp;&emsp;&emsp; → Numpy
&emsp;&emsp;&emsp;&emsp;&emsp;&emsp;&emsp;&emsp; → Numpy without too many loops (vectorized)
&emsp;&emsp;&emsp;&emsp;&emsp;&emsp;&emsp;&emsp;&emsp;&emsp;&emsp;&emsp;→ C extensions
%% Cell type:markdown id: tags:
But ⚠️ ⚠️ ⚠️ writting a C extension by hand is **not a good idea** ! ⚠️ ⚠️ ⚠️
### *No need to quit the Python language to avoid using too much the Python interpreter* !
%% Cell type:markdown id: tags:
# Tools to
- compile Python
- write C extensions without writing C
Cython, Numba, Pythran, Transonic, PyTorch, ...
%% Cell type:markdown id: tags:
# First conclusions
- Python great language & ecosystem for sciences & data
%% Cell type:markdown id: tags:
- Performance issues, especially for crunching numbers 🔢
*⇒ need to accelerate the "numerical kernels"*
%% Cell type:markdown id: tags:
- Many good accelerators and compilers for Python-Numpy code
- All have pros and cons!
**⇒ We shouldn't have to write specialized code for one accelerator!**
%% Cell type:markdown id: tags:
- Other languages don't replace Python for sciences
- Modern C++ is great and very complementary 💑 with Python
- Julia is interesting but not the heaven on earth
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment