Commit 6f725a90 authored by Franck Thollard's avatar Franck Thollard
Browse files

Merge branch 'master' of gricad-gitlab.univ-grenoble-alpes.fr:python-uga/training-hpc

parents d19944dd b2163dfb
......@@ -6,6 +6,334 @@
"source": [
"# Wrapping codes in static languages"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Fortran with [f2py](https://docs.scipy.org/doc/numpy/f2py/)\n",
"\n",
"`f2py` is a tool that allows to call Fortran code into Python. It is a part of `numpy` meaning that to use it, we only need to install and import numpy (which should already be done if you do scientific Python !) :\n",
"\n",
"```bash\n",
"pip3 install numpy\n",
"```"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### How does it work ?\n",
"The documentation gives several ways to wrap Fortran codes but it all boils down to the same thing:\n",
"\n",
"**f2py allows to wrap the Fortran code in a Python module that can be then imported**\n",
"\n",
"Given this simple Fortran (F90) snippet, that computes the sum of squares of the element of an array:"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"*pyfiles/f2py/file_to_wrap.f90*\n",
"\n",
"```fortran\n",
"subroutine sum_squares(A, res)\n",
" implicit none\n",
" real, dimension(:) :: A\n",
" real :: res\n",
" integer :: i, N\n",
"\n",
" N = size(A)\n",
" res = 0.\n",
"\n",
" do i=1, N\n",
" res = res + A(i)*A(i)\n",
" end do\n",
"\n",
"end subroutine\n",
"```"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The Fortran code can then be wrapped in one command:"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"```bash\n",
"# Syntax: python3 -m numpy.f2py -c <Fortran_files> -m <module_name>\n",
"python3 -m numpy.f2py -c \"../pyfiles/f2py/file_to_wrap.f90\" -m wrap_f90\n",
"```\n",
"\n",
"This command calls the module `f2py` of `numpy` to compile (`-c`) *file_to_wrap.f90* into a Python module (`-m`) named *wrap_f90*. The module can then be imported in Python:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"import numpy as np\n",
"import wrap_f90\n",
"\n",
"A = np.ones(10)\n",
"result = 0.\n",
"\n",
"wrap_f90.sum_squares(A, result)\n",
"print(result)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### With intents"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"In Fortran, it is considered best practice to put intents to subroutine arguments. This also helps `f2py` to wrap efficiently the code but also changes the subroutine a bit.\n",
"\n",
"Let's wrap the code updated with intents:"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"*pyfiles/f2py/file_to_wrap2.f90*\n",
"\n",
"```fortran\n",
"subroutine sum_squares(A, res)\n",
" implicit none\n",
" real, dimension(:), intent(in) :: A\n",
" real, intent(out) :: res\n",
" integer :: i, N\n",
"\n",
" N = size(A)\n",
" res = 0.\n",
"\n",
" do i=1, N\n",
" res = res + A(i)*A(i)\n",
" end do\n",
"\n",
"end subroutine\n",
"```"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Again, we wrap..."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"```bash\n",
"python3 -m numpy.f2py -c \"../pyfiles/f2py/file_to_wrap2.f90\" -m wrap_f90\n",
"```"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"And we import..."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"import numpy as np\n",
"import wrap_f90\n",
"\n",
"A = np.ones(10)\n",
"\n",
"result = wrap_f90.sum_squares(A)\n",
"print(result)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"This time, f2py recognized that `result` was a outgoing arg. As a consequence, the subroutine was wrapped smartly and made to return the arg.\n",
"\n",
"Note that using a `function` (in the Fortran sense of the term) leads to the same result (see the other example in *pyfiles/f2py/file_to_wrap2.f90*)."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### With modules"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"In Fortran, it is also considered best practice to organize the subroutines in modules. These are highly similar to Python modules and are in fact, intepreted as such by f2py !\n",
"\n",
"Consider the following code that implements the dtw and cort computations in Fortran:\n",
"\n",
"*pyfiles/dtw_cort_dist/V9_fortran/dtw_cort.f90*\n",
"```fortran\n",
"module dtw_cort\n",
" implicit none\n",
"\n",
" contains\n",
" subroutine dtwdistance(s1, s2, dtw_result)\n",
" ! Computes the dtw between s1 and s2 with distance the absolute distance\n",
" doubleprecision, intent(in) :: s1(:), s2(:)\n",
" doubleprecision, intent(out) :: dtw_result\n",
"\n",
" integer :: i, j\n",
" integer :: len_s1, len_s2\n",
" doubleprecision :: dist\n",
" doubleprecision, allocatable :: dtw_mat(:, :)\n",
"\n",
" len_s1 = size(s1)\n",
" len_s2 = size(s1)\n",
"\n",
" allocate(dtw_mat(len_s1, len_s2))\n",
"\n",
" dtw_mat(1, 1) = dabs(s1(1) - s2(1))\n",
"\n",
" do j = 2, len_s2\n",
" dist = dabs(s1(1) - s2(j))\n",
" dtw_mat(1, j) = dist + dtw_mat(1, j-1)\n",
" end do\n",
"\n",
" do i = 2, len_s1\n",
" dist = dabs(s1(i) - s2(1))\n",
" dtw_mat(i, 1) = dist + dtw_mat(i-1, 1)\n",
" end do\n",
"\n",
" ! Fill the dtw_matrix\n",
" do i = 2, len_s1\n",
" do j = 2, len_s2\n",
" dist = dabs(s1(i) - s2(j))\n",
" dtw_mat(i, j) = dist + dmin1(dtw_mat(i - 1, j), &\n",
" dtw_mat(i, j - 1), &\n",
" dtw_mat(i - 1, j - 1))\n",
" end do\n",
" end do\n",
"\n",
" dtw_result = dtw_mat(len_s1, len_s2)\n",
"\n",
" end subroutine dtwdistance\n",
"\n",
" doubleprecision function cort(s1, s2)\n",
" ! Computes the cort between s1 and s2 (assuming they have the same length)\n",
" doubleprecision, intent(in) :: s1(:), s2(:)\n",
"\n",
" integer :: len_s1, t\n",
" doubleprecision :: slope_1, slope_2\n",
" doubleprecision :: num, sum_square_x, sum_square_y\n",
"\n",
" len_s1 = size(s1)\n",
" num = 0\n",
" sum_square_x = 0\n",
" sum_square_y = 0\n",
"\n",
" do t=1, len_s1 - 1\n",
" slope_1 = s1(t + 1) - s1(t)\n",
" slope_2 = s2(t + 1) - s2(t)\n",
" num = num + slope_1 * slope_2\n",
" sum_square_x = sum_square_x + slope_1 * slope_1\n",
" sum_square_y = sum_square_y + slope_2 * slope_2\n",
" end do\n",
"\n",
" cort = num / (dsqrt(sum_square_x*sum_square_y))\n",
"\n",
" end function cort\n",
"end module dtw_cort\n",
"```"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The subroutines `dtwdistance` and `cort` are part of the `dtw_cort` module. The file can be wrapped as before\n",
"```bash\n",
" python3 -m numpy.f2py -c \"../pyfiles/dtw_cort_dist/V9_fortran/dtw_cort.f90\" -m distances_fort\n",
"```"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"But the import slighlty changes as `dtw_cort` is now a module of `distances_fort`:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"import numpy as np\n",
"from distances_fort import dtw_cort\n",
"\n",
"cort_result = dtw_cort.cort(s1, s2)\n",
"print(cort_result)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Note that the wrapping integrates the documentation of the function (if written...) !"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from distances_fort import dtw_cort\n",
"\n",
"print(dtw_cort.cort.__doc__)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### To go further...\n",
"\n",
"Running the command `python3 -m numpy.f2py` (without arguments) gives a lot of information on the supported arguments for further uses of `f2py`. Know that you can this way:\n",
"* Specify the compiler to use\n",
"* Give the compiler flags (warnings, optimisations...)\n",
"* Specify the functions to wrap\n",
"* ...\n",
"\n",
"The documentation of f2py (https://docs.scipy.org/doc/numpy/f2py/) can also help, covering notably:\n",
"* `Cf2py` directives to overcome F77 limitations (e.g. intents)\n",
"* How to integrate Fortran sources to your Python packages and compile them on install\n",
"* How to use `f2py` inside Python scripts\n",
"* ..."
]
}
],
"metadata": {
......@@ -24,7 +352,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.7.2"
"version": "3.6.7"
}
},
"nbformat": 4,
......
%% Cell type:markdown id: tags:
# Wrapping codes in static languages
%% Cell type:markdown id: tags:
## Fortran with [f2py](https://docs.scipy.org/doc/numpy/f2py/)
`f2py` is a tool that allows to call Fortran code into Python. It is a part of `numpy` meaning that to use it, we only need to install and import numpy (which should already be done if you do scientific Python !) :
```bash
pip3 install numpy
```
%% Cell type:markdown id: tags:
### How does it work ?
The documentation gives several ways to wrap Fortran codes but it all boils down to the same thing:
**f2py allows to wrap the Fortran code in a Python module that can be then imported**
Given this simple Fortran (F90) snippet, that computes the sum of squares of the element of an array:
%% Cell type:markdown id: tags:
*pyfiles/f2py/file_to_wrap.f90*
```fortran
subroutine sum_squares(A, res)
implicit none
real, dimension(:) :: A
real :: res
integer :: i, N
N = size(A)
res = 0.
do i=1, N
res = res + A(i)*A(i)
end do
end subroutine
```
%% Cell type:markdown id: tags:
The Fortran code can then be wrapped in one command:
%% Cell type:markdown id: tags:
```bash
# Syntax: python3 -m numpy.f2py -c <Fortran_files> -m <module_name>
python3 -m numpy.f2py -c "../pyfiles/f2py/file_to_wrap.f90" -m wrap_f90
```
This command calls the module `f2py` of `numpy` to compile (`-c`) *file_to_wrap.f90* into a Python module (`-m`) named *wrap_f90*. The module can then be imported in Python:
%% Cell type:code id: tags:
``` python
import numpy as np
import wrap_f90
A = np.ones(10)
result = 0.
wrap_f90.sum_squares(A, result)
print(result)
```
%% Cell type:markdown id: tags:
### With intents
%% Cell type:markdown id: tags:
In Fortran, it is considered best practice to put intents to subroutine arguments. This also helps `f2py` to wrap efficiently the code but also changes the subroutine a bit.
Let's wrap the code updated with intents:
%% Cell type:markdown id: tags:
*pyfiles/f2py/file_to_wrap2.f90*
```fortran
subroutine sum_squares(A, res)
implicit none
real, dimension(:), intent(in) :: A
real, intent(out) :: res
integer :: i, N
N = size(A)
res = 0.
do i=1, N
res = res + A(i)*A(i)
end do
end subroutine
```
%% Cell type:markdown id: tags:
Again, we wrap...
%% Cell type:markdown id: tags:
```bash
python3 -m numpy.f2py -c "../pyfiles/f2py/file_to_wrap2.f90" -m wrap_f90
```
%% Cell type:markdown id: tags:
And we import...
%% Cell type:code id: tags:
``` python
import numpy as np
import wrap_f90
A = np.ones(10)
result = wrap_f90.sum_squares(A)
print(result)
```
%% Cell type:markdown id: tags:
This time, f2py recognized that `result` was a outgoing arg. As a consequence, the subroutine was wrapped smartly and made to return the arg.
Note that using a `function` (in the Fortran sense of the term) leads to the same result (see the other example in *pyfiles/f2py/file_to_wrap2.f90*).
%% Cell type:markdown id: tags:
### With modules
%% Cell type:markdown id: tags:
In Fortran, it is also considered best practice to organize the subroutines in modules. These are highly similar to Python modules and are in fact, intepreted as such by f2py !
Consider the following code that implements the dtw and cort computations in Fortran:
*pyfiles/dtw_cort_dist/V9_fortran/dtw_cort.f90*
```fortran
module dtw_cort
implicit none
contains
subroutine dtwdistance(s1, s2, dtw_result)
! Computes the dtw between s1 and s2 with distance the absolute distance
doubleprecision, intent(in) :: s1(:), s2(:)
doubleprecision, intent(out) :: dtw_result
integer :: i, j
integer :: len_s1, len_s2
doubleprecision :: dist
doubleprecision, allocatable :: dtw_mat(:, :)
len_s1 = size(s1)
len_s2 = size(s1)
allocate(dtw_mat(len_s1, len_s2))
dtw_mat(1, 1) = dabs(s1(1) - s2(1))
do j = 2, len_s2
dist = dabs(s1(1) - s2(j))
dtw_mat(1, j) = dist + dtw_mat(1, j-1)
end do
do i = 2, len_s1
dist = dabs(s1(i) - s2(1))
dtw_mat(i, 1) = dist + dtw_mat(i-1, 1)
end do
! Fill the dtw_matrix
do i = 2, len_s1
do j = 2, len_s2
dist = dabs(s1(i) - s2(j))
dtw_mat(i, j) = dist + dmin1(dtw_mat(i - 1, j), &
dtw_mat(i, j - 1), &
dtw_mat(i - 1, j - 1))
end do
end do
dtw_result = dtw_mat(len_s1, len_s2)
end subroutine dtwdistance
doubleprecision function cort(s1, s2)
! Computes the cort between s1 and s2 (assuming they have the same length)
doubleprecision, intent(in) :: s1(:), s2(:)
integer :: len_s1, t
doubleprecision :: slope_1, slope_2
doubleprecision :: num, sum_square_x, sum_square_y
len_s1 = size(s1)
num = 0
sum_square_x = 0
sum_square_y = 0
do t=1, len_s1 - 1
slope_1 = s1(t + 1) - s1(t)
slope_2 = s2(t + 1) - s2(t)
num = num + slope_1 * slope_2
sum_square_x = sum_square_x + slope_1 * slope_1
sum_square_y = sum_square_y + slope_2 * slope_2
end do
cort = num / (dsqrt(sum_square_x*sum_square_y))
end function cort
end module dtw_cort
```
%% Cell type:markdown id: tags:
The subroutines `dtwdistance` and `cort` are part of the `dtw_cort` module. The file can be wrapped as before
```bash
python3 -m numpy.f2py -c "../pyfiles/dtw_cort_dist/V9_fortran/dtw_cort.f90" -m distances_fort
```
%% Cell type:markdown id: tags:
But the import slighlty changes as `dtw_cort` is now a module of `distances_fort`:
%% Cell type:code id: tags:
``` python
import numpy as np
from distances_fort import dtw_cort
cort_result = dtw_cort.cort(s1, s2)
print(cort_result)
```
%% Cell type:markdown id: tags:
Note that the wrapping integrates the documentation of the function (if written...) !
%% Cell type:code id: tags:
``` python
from distances_fort import dtw_cort
print(dtw_cort.cort.__doc__)
```
%% Cell type:markdown id: tags:
### To go further...
Running the command `python3 -m numpy.f2py` (without arguments) gives a lot of information on the supported arguments for further uses of `f2py`. Know that you can this way:
* Specify the compiler to use
* Give the compiler flags (warnings, optimisations...)
* Specify the functions to wrap
* ...
The documentation of f2py (https://docs.scipy.org/doc/numpy/f2py/) can also help, covering notably:
* `Cf2py` directives to overcome F77 limitations (e.g. intents)
* How to integrate Fortran sources to your Python packages and compile them on install
* How to use `f2py` inside Python scripts
* ...
......
......@@ -62,8 +62,8 @@ def DTWDistance(s1, s2):
def cort(s1, s2):
""" Computes the cort between series one and two (assuming they have the same length)
:param s1: the first serie (or any iterable over floats64)
:param s2: the second serie (or any iterable over floats64)
:param s1: the first series (or any iterable over floats64)
:param s2: the second series (or any iterable over floats64)
:returns: the cort distance
:rtype: float64
......
......@@ -49,9 +49,12 @@ module dtw_cort
integer :: len_s1, t
doubleprecision :: slope_1, slope_2
doubleprecision :: num=0, sum_square_x=0, sum_square_y=0
doubleprecision :: num, sum_square_x, sum_square_y
len_s1 = size(s1)
num = 0
sum_square_x = 0
sum_square_y = 0
do t=1, len_s1 - 1
slope_1 = s1(t + 1) - s1(t)
......@@ -61,7 +64,7 @@ module dtw_cort
sum_square_y = sum_square_y + slope_2 * slope_2
end do
cort = num / dsqrt(sum_square_x*sum_square_x)
cort = num / (dsqrt(sum_square_x*sum_square_y))
end function cort
......
subroutine sum_squares(A, res)
implicit none
real, dimension(:) :: A
real :: res
integer :: i, N
N = size(A)
res = 0.
do i=1, N
res = res + A(i)*A(i)
end do
end subroutine
\ No newline at end of file
subroutine sum_squares(A, res)
implicit none
real, dimension(:), intent(in) :: A
real, intent(out) :: res
integer :: i, N
N = size(A)
res = 0.
do i=1, N
res = res + A(i)*A(i)
end do
end subroutine