Commit bbb56c4c authored by Franck Thollard's avatar Franck Thollard
Browse files

adding a small intro + alternatives

parent 76162001
...@@ -14,7 +14,8 @@ ...@@ -14,7 +14,8 @@
"We consider here wrapping two static languages: C and fortran.\n", "We consider here wrapping two static languages: C and fortran.\n",
"\n", "\n",
"We classically wrapp already existing code to access them via python. \n", "We classically wrapp already existing code to access them via python. \n",
"\n" "\n",
"Depending on the language to wrap the tool to use are a bit different. \n"
] ]
}, },
{ {
...@@ -479,6 +480,21 @@ ...@@ -479,6 +480,21 @@
"source": [ "source": [
"sq = dllib.square(2.0, 3.0)" "sq = dllib.square(2.0, 3.0)"
] ]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Alternatives techniques: \n",
"------------------------------------\n",
"\n",
"The historical tool is swig (http://swig.org/). It allows to access C/C++ code from a variety of languages. \n",
"It requires the writing of an intermediate file that describes the C API. \n",
"\n",
"From now wrapping C code can be done quite easily using CFFI as presented before. \n",
"\n",
"For wrapping C++ code, one will consider pybind11 (https://github.com/pybind/pybind11) that relies on features available from the 11 versions of C++. "
]
} }
], ],
"metadata": { "metadata": {
......
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
# Wrapping codes in static languages # Wrapping codes in static languages
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
We consider here wrapping two static languages: C and fortran. We consider here wrapping two static languages: C and fortran.
We classically wrapp already existing code to access them via python. We classically wrapp already existing code to access them via python.
Depending on the language to wrap the tool to use are a bit different.
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
## Fortran with [f2py](https://docs.scipy.org/doc/numpy/f2py/) ## Fortran with [f2py](https://docs.scipy.org/doc/numpy/f2py/)
`f2py` is a tool that allows to call Fortran code into Python. It is a part of `numpy` meaning that to use it, we only need to install and import numpy (which should already be done if you do scientific Python !) : `f2py` is a tool that allows to call Fortran code into Python. It is a part of `numpy` meaning that to use it, we only need to install and import numpy (which should already be done if you do scientific Python !) :
```bash ```bash
pip3 install numpy pip3 install numpy
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
### How does it work ? ### How does it work ?
The documentation gives several ways to wrap Fortran codes but it all boils down to the same thing: The documentation gives several ways to wrap Fortran codes but it all boils down to the same thing:
**f2py allows to wrap the Fortran code in a Python module that can be then imported** **f2py allows to wrap the Fortran code in a Python module that can be then imported**
Given this simple Fortran (F90) snippet, that computes the sum of squares of the element of an array: Given this simple Fortran (F90) snippet, that computes the sum of squares of the element of an array:
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
*pyfiles/f2py/file_to_wrap.f90* *pyfiles/f2py/file_to_wrap.f90*
```fortran ```fortran
subroutine sum_squares(A, res) subroutine sum_squares(A, res)
implicit none implicit none
real, dimension(:) :: A real, dimension(:) :: A
real :: res real :: res
integer :: i, N integer :: i, N
N = size(A) N = size(A)
res = 0. res = 0.
do i=1, N do i=1, N
res = res + A(i)*A(i) res = res + A(i)*A(i)
end do end do
end subroutine end subroutine
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
The Fortran code can then be wrapped in one command: The Fortran code can then be wrapped in one command:
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
```bash ```bash
# Syntax: python3 -m numpy.f2py -c <Fortran_files> -m <module_name> # Syntax: python3 -m numpy.f2py -c <Fortran_files> -m <module_name>
python3 -m numpy.f2py -c "../pyfiles/f2py/file_to_wrap.f90" -m wrap_f90 python3 -m numpy.f2py -c "../pyfiles/f2py/file_to_wrap.f90" -m wrap_f90
``` ```
This command calls the module `f2py` of `numpy` to compile (`-c`) *file_to_wrap.f90* into a Python module (`-m`) named *wrap_f90*. The module can then be imported in Python: This command calls the module `f2py` of `numpy` to compile (`-c`) *file_to_wrap.f90* into a Python module (`-m`) named *wrap_f90*. The module can then be imported in Python:
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
import numpy as np import numpy as np
import wrap_f90 import wrap_f90
A = np.ones(10) A = np.ones(10)
result = 0. result = 0.
wrap_f90.sum_squares(A, result) wrap_f90.sum_squares(A, result)
print(result) print(result)
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
### With intents ### With intents
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
In Fortran, it is considered best practice to put intents to subroutine arguments. This also helps `f2py` to wrap efficiently the code but also changes the subroutine a bit. In Fortran, it is considered best practice to put intents to subroutine arguments. This also helps `f2py` to wrap efficiently the code but also changes the subroutine a bit.
Let's wrap the code updated with intents: Let's wrap the code updated with intents:
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
*pyfiles/f2py/file_to_wrap2.f90* *pyfiles/f2py/file_to_wrap2.f90*
```fortran ```fortran
subroutine sum_squares(A, res) subroutine sum_squares(A, res)
implicit none implicit none
real, dimension(:), intent(in) :: A real, dimension(:), intent(in) :: A
real, intent(out) :: res real, intent(out) :: res
integer :: i, N integer :: i, N
N = size(A) N = size(A)
res = 0. res = 0.
do i=1, N do i=1, N
res = res + A(i)*A(i) res = res + A(i)*A(i)
end do end do
end subroutine end subroutine
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
Again, we wrap... Again, we wrap...
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
```bash ```bash
python3 -m numpy.f2py -c "../pyfiles/f2py/file_to_wrap2.f90" -m wrap_f90 python3 -m numpy.f2py -c "../pyfiles/f2py/file_to_wrap2.f90" -m wrap_f90
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
And we import... And we import...
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
import numpy as np import numpy as np
import wrap_f90 import wrap_f90
A = np.ones(10) A = np.ones(10)
result = wrap_f90.sum_squares(A) result = wrap_f90.sum_squares(A)
print(result) print(result)
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
This time, f2py recognized that `result` was a outgoing arg. As a consequence, the subroutine was wrapped smartly and made to return the arg. This time, f2py recognized that `result` was a outgoing arg. As a consequence, the subroutine was wrapped smartly and made to return the arg.
Note that using a `function` (in the Fortran sense of the term) leads to the same result (see the other example in *pyfiles/f2py/file_to_wrap2.f90*). Note that using a `function` (in the Fortran sense of the term) leads to the same result (see the other example in *pyfiles/f2py/file_to_wrap2.f90*).
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
### With modules ### With modules
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
In Fortran, it is also considered best practice to organize the subroutines in modules. These are highly similar to Python modules and are in fact, intepreted as such by f2py ! In Fortran, it is also considered best practice to organize the subroutines in modules. These are highly similar to Python modules and are in fact, intepreted as such by f2py !
Consider the following code that implements the dtw and cort computations in Fortran: Consider the following code that implements the dtw and cort computations in Fortran:
*pyfiles/dtw_cort_dist/V9_fortran/dtw_cort.f90* *pyfiles/dtw_cort_dist/V9_fortran/dtw_cort.f90*
```fortran ```fortran
module dtw_cort module dtw_cort
implicit none implicit none
contains contains
subroutine dtwdistance(s1, s2, dtw_result) subroutine dtwdistance(s1, s2, dtw_result)
! Computes the dtw between s1 and s2 with distance the absolute distance ! Computes the dtw between s1 and s2 with distance the absolute distance
doubleprecision, intent(in) :: s1(:), s2(:) doubleprecision, intent(in) :: s1(:), s2(:)
doubleprecision, intent(out) :: dtw_result doubleprecision, intent(out) :: dtw_result
integer :: i, j integer :: i, j
integer :: len_s1, len_s2 integer :: len_s1, len_s2
doubleprecision :: dist doubleprecision :: dist
doubleprecision, allocatable :: dtw_mat(:, :) doubleprecision, allocatable :: dtw_mat(:, :)
len_s1 = size(s1) len_s1 = size(s1)
len_s2 = size(s1) len_s2 = size(s1)
allocate(dtw_mat(len_s1, len_s2)) allocate(dtw_mat(len_s1, len_s2))
dtw_mat(1, 1) = dabs(s1(1) - s2(1)) dtw_mat(1, 1) = dabs(s1(1) - s2(1))
do j = 2, len_s2 do j = 2, len_s2
dist = dabs(s1(1) - s2(j)) dist = dabs(s1(1) - s2(j))
dtw_mat(1, j) = dist + dtw_mat(1, j-1) dtw_mat(1, j) = dist + dtw_mat(1, j-1)
end do end do
do i = 2, len_s1 do i = 2, len_s1
dist = dabs(s1(i) - s2(1)) dist = dabs(s1(i) - s2(1))
dtw_mat(i, 1) = dist + dtw_mat(i-1, 1) dtw_mat(i, 1) = dist + dtw_mat(i-1, 1)
end do end do
! Fill the dtw_matrix ! Fill the dtw_matrix
do i = 2, len_s1 do i = 2, len_s1
do j = 2, len_s2 do j = 2, len_s2
dist = dabs(s1(i) - s2(j)) dist = dabs(s1(i) - s2(j))
dtw_mat(i, j) = dist + dmin1(dtw_mat(i - 1, j), & dtw_mat(i, j) = dist + dmin1(dtw_mat(i - 1, j), &
dtw_mat(i, j - 1), & dtw_mat(i, j - 1), &
dtw_mat(i - 1, j - 1)) dtw_mat(i - 1, j - 1))
end do end do
end do end do
dtw_result = dtw_mat(len_s1, len_s2) dtw_result = dtw_mat(len_s1, len_s2)
end subroutine dtwdistance end subroutine dtwdistance
doubleprecision function cort(s1, s2) doubleprecision function cort(s1, s2)
! Computes the cort between s1 and s2 (assuming they have the same length) ! Computes the cort between s1 and s2 (assuming they have the same length)
doubleprecision, intent(in) :: s1(:), s2(:) doubleprecision, intent(in) :: s1(:), s2(:)
integer :: len_s1, t integer :: len_s1, t
doubleprecision :: slope_1, slope_2 doubleprecision :: slope_1, slope_2
doubleprecision :: num, sum_square_x, sum_square_y doubleprecision :: num, sum_square_x, sum_square_y
len_s1 = size(s1) len_s1 = size(s1)
num = 0 num = 0
sum_square_x = 0 sum_square_x = 0
sum_square_y = 0 sum_square_y = 0
do t=1, len_s1 - 1 do t=1, len_s1 - 1
slope_1 = s1(t + 1) - s1(t) slope_1 = s1(t + 1) - s1(t)
slope_2 = s2(t + 1) - s2(t) slope_2 = s2(t + 1) - s2(t)
num = num + slope_1 * slope_2 num = num + slope_1 * slope_2
sum_square_x = sum_square_x + slope_1 * slope_1 sum_square_x = sum_square_x + slope_1 * slope_1
sum_square_y = sum_square_y + slope_2 * slope_2 sum_square_y = sum_square_y + slope_2 * slope_2
end do end do
cort = num / (dsqrt(sum_square_x*sum_square_y)) cort = num / (dsqrt(sum_square_x*sum_square_y))
end function cort end function cort
end module dtw_cort end module dtw_cort
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
The subroutines `dtwdistance` and `cort` are part of the `dtw_cort` module. The file can be wrapped as before The subroutines `dtwdistance` and `cort` are part of the `dtw_cort` module. The file can be wrapped as before
```bash ```bash
python3 -m numpy.f2py -c "../pyfiles/dtw_cort_dist/V9_fortran/dtw_cort.f90" -m distances_fort python3 -m numpy.f2py -c "../pyfiles/dtw_cort_dist/V9_fortran/dtw_cort.f90" -m distances_fort
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
But the import slighlty changes as `dtw_cort` is now a module of `distances_fort`: But the import slighlty changes as `dtw_cort` is now a module of `distances_fort`:
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
import numpy as np import numpy as np
from distances_fort import dtw_cort from distances_fort import dtw_cort
cort_result = dtw_cort.cort(s1, s2) cort_result = dtw_cort.cort(s1, s2)
print(cort_result) print(cort_result)
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
Note that the wrapping integrates the documentation of the function (if written...) ! Note that the wrapping integrates the documentation of the function (if written...) !
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
from distances_fort import dtw_cort from distances_fort import dtw_cort
print(dtw_cort.cort.__doc__) print(dtw_cort.cort.__doc__)
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
### To go further... ### To go further...
Running the command `python3 -m numpy.f2py` (without arguments) gives a lot of information on the supported arguments for further uses of `f2py`. Know that you can this way: Running the command `python3 -m numpy.f2py` (without arguments) gives a lot of information on the supported arguments for further uses of `f2py`. Know that you can this way:
* Specify the compiler to use * Specify the compiler to use
* Give the compiler flags (warnings, optimisations...) * Give the compiler flags (warnings, optimisations...)
* Specify the functions to wrap * Specify the functions to wrap
* ... * ...
The documentation of f2py (https://docs.scipy.org/doc/numpy/f2py/) can also help, covering notably: The documentation of f2py (https://docs.scipy.org/doc/numpy/f2py/) can also help, covering notably:
* `Cf2py` directives to overcome F77 limitations (e.g. intents) * `Cf2py` directives to overcome F77 limitations (e.g. intents)
* How to integrate Fortran sources to your Python packages and compile them on install * How to integrate Fortran sources to your Python packages and compile them on install
* How to use `f2py` inside Python scripts * How to use `f2py` inside Python scripts
* ... * ...
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
Wrapping C code Wrapping C code
-------------------------- --------------------------
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
They are different ways of wrapping C code. We present CFFI. They are different ways of wrapping C code. We present CFFI.
The workflow is the following: The workflow is the following:
1. Get your C code working (with a .c and a .h) 1. Get your C code working (with a .c and a .h)
2. Set up your packaging to compile your code as a module 2. Set up your packaging to compile your code as a module
3. Compile your code 3. Compile your code
4. In the python code, declare the function you will be using 4. In the python code, declare the function you will be using
5. In the python code, open/load the compiled module 5. In the python code, open/load the compiled module
6. Use your functions 6. Use your functions
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
**1. Get your C code working (with a .c and a .h)** **1. Get your C code working (with a .c and a .h)**
Ok, supposed to be done Ok, supposed to be done
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
**2. Set up your packaging to compile your code as a module** **2. Set up your packaging to compile your code as a module**
We give the compilation instructions in the file setup.py: We give the compilation instructions in the file setup.py:
%% Cell type:raw id: tags: %% Cell type:raw id: tags:
from setuptools import setup, Extension from setuptools import setup, Extension
version = "0.1" version = "0.1"
module_distance = Extension( module_distance = Extension(
name="cdtw", name="cdtw",
sources=["cdtw.c"], sources=["cdtw.c"],
) )
setup( setup(
name="dtw_cort_dist_mat", name="dtw_cort_dist_mat",
version=version, version=version,
description="data scientist tool for time series", description="data scientist tool for time series",
long_description="data scientist tool for time series", long_description="data scientist tool for time series",
classifiers=[], classifiers=[],
author="Robert Bidochon", author="Robert Bidochon",
author_email="robert@bidochon.fr", author_email="robert@bidochon.fr",
license="GPL", license="GPL",
include_package_data=True, include_package_data=True,
install_requires=["cffi", "numpy", "setuptools"], install_requires=["cffi", "numpy", "setuptools"],
entry_points="", entry_points="",
ext_modules=[module_distance], ext_modules=[module_distance],
) )
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
**3. Compile your code** **3. Compile your code**
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
in a terminal, type: in a terminal, type:
python3 setup.py build_ext python3 setup.py build_ext
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
**4. In the python code, declare the function you will be using** **4. In the python code, declare the function you will be using**
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
from cffi import FFI from cffi import FFI
ffi = FFI() ffi = FFI()
ffi.cdef("double square(double x, double y);") ffi.cdef("double square(double x, double y);")
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
**5. In the python code, open/load the compiled module** **5. In the python code, open/load the compiled module**
%% Cell type:raw id: tags: %% Cell type:raw id: tags:
dllib = ffi.dlopen( dllib = ffi.dlopen(
str(my_dir / ("cdtw" + sysconfig.get_config_var("EXT_SUFFIX"))) str(my_dir / ("cdtw" + sysconfig.get_config_var("EXT_SUFFIX")))
) )
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
**6. Use your functions** **6. Use your functions**
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
sq = dllib.square(2.0, 3.0) sq = dllib.square(2.0, 3.0)
%% Cell type:markdown id: tags:
Alternatives techniques:
------------------------------------
The historical tool is swig (http://swig.org/). It allows to access C/C++ code from a variety of languages.
It requires the writing of an intermediate file that describes the C API.
From now wrapping C code can be done quite easily using CFFI as presented before.
For wrapping C++ code, one will consider pybind11 (https://github.com/pybind/pybind11) that relies on features available from the 11 versions of C++.
......
Supports Markdown
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment