Commit a3a12860 authored by Franck Thollard's avatar Franck Thollard
Browse files

adding cffi to wrappers

parent 8a9d9eb6
...@@ -7,6 +7,16 @@ ...@@ -7,6 +7,16 @@
"# Wrapping codes in static languages" "# Wrapping codes in static languages"
] ]
}, },
{
"cell_type": "markdown",
"metadata": {},
"source": [
"We consider here wrapping two static languages: C and fortran.\n",
"\n",
"We classically wrapp already existing code to access them via python. \n",
"\n"
]
},
{ {
"cell_type": "markdown", "cell_type": "markdown",
"metadata": {}, "metadata": {},
...@@ -334,6 +344,141 @@ ...@@ -334,6 +344,141 @@
"* How to use `f2py` inside Python scripts\n", "* How to use `f2py` inside Python scripts\n",
"* ..." "* ..."
] ]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Wrapping C code\n",
"--------------------------"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"They are different ways of wrapping C code. We present CFFI. \n",
"\n",
"The workflow is the following:\n",
" 1. Get your C code working (with a .c and a .h)\n",
" 2. Set up your packaging to compile your code as a module\n",
" 3. Compile your code\n",
" 4. In the python code, declare the function you will be using\n",
" 5. In the python code, open/load the compiled module\n",
" 6. Use your functions\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**1. Get your C code working (with a .c and a .h)**\n",
"\n",
"Ok, supposed to be done"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**2. Set up your packaging to compile your code as a module**\n",
"\n",
"We give the compilation instructions in the file setup.py:\n"
]
},
{
"cell_type": "raw",
"metadata": {},
"source": [
"from setuptools import setup, Extension\n",
"\n",
"version = \"0.1\"\n",
"\n",
"module_distance = Extension(\n",
" name=\"cdtw\",\n",
" sources=[\"cdtw.c\"],\n",
")\n",
"\n",
"setup(\n",
" name=\"dtw_cort_dist_mat\",\n",
" version=version,\n",
" description=\"data scientist tool for time series\",\n",
" long_description=\"data scientist tool for time series\",\n",
" classifiers=[], \n",
" author=\"Robert Bidochon\",\n",
" author_email=\"robert@bidochon.fr\",\n",
" license=\"GPL\",\n",
" include_package_data=True,\n",
" install_requires=[\"cffi\", \"numpy\", \"setuptools\"],\n",
" entry_points=\"\",\n",
" ext_modules=[module_distance],\n",
")\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**3. Compile your code**"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"in a terminal, type:\n",
"\n",
"python3 setup.py build_ext"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**4. In the python code, declare the function you will be using**"
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {},
"outputs": [],
"source": [
"from cffi import FFI\n",
"\n",
"ffi = FFI()\n",
"ffi.cdef(\"double square(double x, double y);\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**5. In the python code, open/load the compiled module**"
]
},
{
"cell_type": "raw",
"metadata": {},
"source": [
"dllib = ffi.dlopen(\n",
" str(my_dir / (\"cdtw\" + sysconfig.get_config_var(\"EXT_SUFFIX\")))\n",
")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**6. Use your functions**"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"sq = dllib.square(2.0, 3.0)"
]
} }
], ],
"metadata": { "metadata": {
...@@ -352,7 +497,7 @@ ...@@ -352,7 +497,7 @@
"name": "python", "name": "python",
"nbconvert_exporter": "python", "nbconvert_exporter": "python",
"pygments_lexer": "ipython3", "pygments_lexer": "ipython3",
"version": "3.6.7" "version": "3.7.3"
} }
}, },
"nbformat": 4, "nbformat": 4,
......
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
# Wrapping codes in static languages # Wrapping codes in static languages
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
We consider here wrapping two static languages: C and fortran.
We classically wrapp already existing code to access them via python.
%% Cell type:markdown id: tags:
## Fortran with [f2py](https://docs.scipy.org/doc/numpy/f2py/) ## Fortran with [f2py](https://docs.scipy.org/doc/numpy/f2py/)
`f2py` is a tool that allows to call Fortran code into Python. It is a part of `numpy` meaning that to use it, we only need to install and import numpy (which should already be done if you do scientific Python !) : `f2py` is a tool that allows to call Fortran code into Python. It is a part of `numpy` meaning that to use it, we only need to install and import numpy (which should already be done if you do scientific Python !) :
```bash ```bash
pip3 install numpy pip3 install numpy
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
### How does it work ? ### How does it work ?
The documentation gives several ways to wrap Fortran codes but it all boils down to the same thing: The documentation gives several ways to wrap Fortran codes but it all boils down to the same thing:
**f2py allows to wrap the Fortran code in a Python module that can be then imported** **f2py allows to wrap the Fortran code in a Python module that can be then imported**
Given this simple Fortran (F90) snippet, that computes the sum of squares of the element of an array: Given this simple Fortran (F90) snippet, that computes the sum of squares of the element of an array:
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
*pyfiles/f2py/file_to_wrap.f90* *pyfiles/f2py/file_to_wrap.f90*
```fortran ```fortran
subroutine sum_squares(A, res) subroutine sum_squares(A, res)
implicit none implicit none
real, dimension(:) :: A real, dimension(:) :: A
real :: res real :: res
integer :: i, N integer :: i, N
N = size(A) N = size(A)
res = 0. res = 0.
do i=1, N do i=1, N
res = res + A(i)*A(i) res = res + A(i)*A(i)
end do end do
end subroutine end subroutine
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
The Fortran code can then be wrapped in one command: The Fortran code can then be wrapped in one command:
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
```bash ```bash
# Syntax: python3 -m numpy.f2py -c <Fortran_files> -m <module_name> # Syntax: python3 -m numpy.f2py -c <Fortran_files> -m <module_name>
python3 -m numpy.f2py -c "../pyfiles/f2py/file_to_wrap.f90" -m wrap_f90 python3 -m numpy.f2py -c "../pyfiles/f2py/file_to_wrap.f90" -m wrap_f90
``` ```
This command calls the module `f2py` of `numpy` to compile (`-c`) *file_to_wrap.f90* into a Python module (`-m`) named *wrap_f90*. The module can then be imported in Python: This command calls the module `f2py` of `numpy` to compile (`-c`) *file_to_wrap.f90* into a Python module (`-m`) named *wrap_f90*. The module can then be imported in Python:
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
import numpy as np import numpy as np
import wrap_f90 import wrap_f90
A = np.ones(10) A = np.ones(10)
result = 0. result = 0.
wrap_f90.sum_squares(A, result) wrap_f90.sum_squares(A, result)
print(result) print(result)
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
### With intents ### With intents
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
In Fortran, it is considered best practice to put intents to subroutine arguments. This also helps `f2py` to wrap efficiently the code but also changes the subroutine a bit. In Fortran, it is considered best practice to put intents to subroutine arguments. This also helps `f2py` to wrap efficiently the code but also changes the subroutine a bit.
Let's wrap the code updated with intents: Let's wrap the code updated with intents:
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
*pyfiles/f2py/file_to_wrap2.f90* *pyfiles/f2py/file_to_wrap2.f90*
```fortran ```fortran
subroutine sum_squares(A, res) subroutine sum_squares(A, res)
implicit none implicit none
real, dimension(:), intent(in) :: A real, dimension(:), intent(in) :: A
real, intent(out) :: res real, intent(out) :: res
integer :: i, N integer :: i, N
N = size(A) N = size(A)
res = 0. res = 0.
do i=1, N do i=1, N
res = res + A(i)*A(i) res = res + A(i)*A(i)
end do end do
end subroutine end subroutine
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
Again, we wrap... Again, we wrap...
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
```bash ```bash
python3 -m numpy.f2py -c "../pyfiles/f2py/file_to_wrap2.f90" -m wrap_f90 python3 -m numpy.f2py -c "../pyfiles/f2py/file_to_wrap2.f90" -m wrap_f90
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
And we import... And we import...
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
import numpy as np import numpy as np
import wrap_f90 import wrap_f90
A = np.ones(10) A = np.ones(10)
result = wrap_f90.sum_squares(A) result = wrap_f90.sum_squares(A)
print(result) print(result)
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
This time, f2py recognized that `result` was a outgoing arg. As a consequence, the subroutine was wrapped smartly and made to return the arg. This time, f2py recognized that `result` was a outgoing arg. As a consequence, the subroutine was wrapped smartly and made to return the arg.
Note that using a `function` (in the Fortran sense of the term) leads to the same result (see the other example in *pyfiles/f2py/file_to_wrap2.f90*). Note that using a `function` (in the Fortran sense of the term) leads to the same result (see the other example in *pyfiles/f2py/file_to_wrap2.f90*).
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
### With modules ### With modules
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
In Fortran, it is also considered best practice to organize the subroutines in modules. These are highly similar to Python modules and are in fact, intepreted as such by f2py ! In Fortran, it is also considered best practice to organize the subroutines in modules. These are highly similar to Python modules and are in fact, intepreted as such by f2py !
Consider the following code that implements the dtw and cort computations in Fortran: Consider the following code that implements the dtw and cort computations in Fortran:
*pyfiles/dtw_cort_dist/V9_fortran/dtw_cort.f90* *pyfiles/dtw_cort_dist/V9_fortran/dtw_cort.f90*
```fortran ```fortran
module dtw_cort module dtw_cort
implicit none implicit none
contains contains
subroutine dtwdistance(s1, s2, dtw_result) subroutine dtwdistance(s1, s2, dtw_result)
! Computes the dtw between s1 and s2 with distance the absolute distance ! Computes the dtw between s1 and s2 with distance the absolute distance
doubleprecision, intent(in) :: s1(:), s2(:) doubleprecision, intent(in) :: s1(:), s2(:)
doubleprecision, intent(out) :: dtw_result doubleprecision, intent(out) :: dtw_result
integer :: i, j integer :: i, j
integer :: len_s1, len_s2 integer :: len_s1, len_s2
doubleprecision :: dist doubleprecision :: dist
doubleprecision, allocatable :: dtw_mat(:, :) doubleprecision, allocatable :: dtw_mat(:, :)
len_s1 = size(s1) len_s1 = size(s1)
len_s2 = size(s1) len_s2 = size(s1)
allocate(dtw_mat(len_s1, len_s2)) allocate(dtw_mat(len_s1, len_s2))
dtw_mat(1, 1) = dabs(s1(1) - s2(1)) dtw_mat(1, 1) = dabs(s1(1) - s2(1))
do j = 2, len_s2 do j = 2, len_s2
dist = dabs(s1(1) - s2(j)) dist = dabs(s1(1) - s2(j))
dtw_mat(1, j) = dist + dtw_mat(1, j-1) dtw_mat(1, j) = dist + dtw_mat(1, j-1)
end do end do
do i = 2, len_s1 do i = 2, len_s1
dist = dabs(s1(i) - s2(1)) dist = dabs(s1(i) - s2(1))
dtw_mat(i, 1) = dist + dtw_mat(i-1, 1) dtw_mat(i, 1) = dist + dtw_mat(i-1, 1)
end do end do
! Fill the dtw_matrix ! Fill the dtw_matrix
do i = 2, len_s1 do i = 2, len_s1
do j = 2, len_s2 do j = 2, len_s2
dist = dabs(s1(i) - s2(j)) dist = dabs(s1(i) - s2(j))
dtw_mat(i, j) = dist + dmin1(dtw_mat(i - 1, j), & dtw_mat(i, j) = dist + dmin1(dtw_mat(i - 1, j), &
dtw_mat(i, j - 1), & dtw_mat(i, j - 1), &
dtw_mat(i - 1, j - 1)) dtw_mat(i - 1, j - 1))
end do end do
end do end do
dtw_result = dtw_mat(len_s1, len_s2) dtw_result = dtw_mat(len_s1, len_s2)
end subroutine dtwdistance end subroutine dtwdistance
doubleprecision function cort(s1, s2) doubleprecision function cort(s1, s2)
! Computes the cort between s1 and s2 (assuming they have the same length) ! Computes the cort between s1 and s2 (assuming they have the same length)
doubleprecision, intent(in) :: s1(:), s2(:) doubleprecision, intent(in) :: s1(:), s2(:)
integer :: len_s1, t integer :: len_s1, t
doubleprecision :: slope_1, slope_2 doubleprecision :: slope_1, slope_2
doubleprecision :: num, sum_square_x, sum_square_y doubleprecision :: num, sum_square_x, sum_square_y
len_s1 = size(s1) len_s1 = size(s1)
num = 0 num = 0
sum_square_x = 0 sum_square_x = 0
sum_square_y = 0 sum_square_y = 0
do t=1, len_s1 - 1 do t=1, len_s1 - 1
slope_1 = s1(t + 1) - s1(t) slope_1 = s1(t + 1) - s1(t)
slope_2 = s2(t + 1) - s2(t) slope_2 = s2(t + 1) - s2(t)
num = num + slope_1 * slope_2 num = num + slope_1 * slope_2
sum_square_x = sum_square_x + slope_1 * slope_1 sum_square_x = sum_square_x + slope_1 * slope_1
sum_square_y = sum_square_y + slope_2 * slope_2 sum_square_y = sum_square_y + slope_2 * slope_2
end do end do
cort = num / (dsqrt(sum_square_x*sum_square_y)) cort = num / (dsqrt(sum_square_x*sum_square_y))
end function cort end function cort
end module dtw_cort end module dtw_cort
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
The subroutines `dtwdistance` and `cort` are part of the `dtw_cort` module. The file can be wrapped as before The subroutines `dtwdistance` and `cort` are part of the `dtw_cort` module. The file can be wrapped as before
```bash ```bash
python3 -m numpy.f2py -c "../pyfiles/dtw_cort_dist/V9_fortran/dtw_cort.f90" -m distances_fort python3 -m numpy.f2py -c "../pyfiles/dtw_cort_dist/V9_fortran/dtw_cort.f90" -m distances_fort
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
But the import slighlty changes as `dtw_cort` is now a module of `distances_fort`: But the import slighlty changes as `dtw_cort` is now a module of `distances_fort`:
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
import numpy as np import numpy as np
from distances_fort import dtw_cort from distances_fort import dtw_cort
cort_result = dtw_cort.cort(s1, s2) cort_result = dtw_cort.cort(s1, s2)
print(cort_result) print(cort_result)
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
Note that the wrapping integrates the documentation of the function (if written...) ! Note that the wrapping integrates the documentation of the function (if written...) !
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
from distances_fort import dtw_cort from distances_fort import dtw_cort
print(dtw_cort.cort.__doc__) print(dtw_cort.cort.__doc__)
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
### To go further... ### To go further...
Running the command `python3 -m numpy.f2py` (without arguments) gives a lot of information on the supported arguments for further uses of `f2py`. Know that you can this way: Running the command `python3 -m numpy.f2py` (without arguments) gives a lot of information on the supported arguments for further uses of `f2py`. Know that you can this way:
* Specify the compiler to use * Specify the compiler to use
* Give the compiler flags (warnings, optimisations...) * Give the compiler flags (warnings, optimisations...)
* Specify the functions to wrap * Specify the functions to wrap
* ... * ...
The documentation of f2py (https://docs.scipy.org/doc/numpy/f2py/) can also help, covering notably: The documentation of f2py (https://docs.scipy.org/doc/numpy/f2py/) can also help, covering notably:
* `Cf2py` directives to overcome F77 limitations (e.g. intents) * `Cf2py` directives to overcome F77 limitations (e.g. intents)
* How to integrate Fortran sources to your Python packages and compile them on install * How to integrate Fortran sources to your Python packages and compile them on install
* How to use `f2py` inside Python scripts * How to use `f2py` inside Python scripts
* ... * ...
%% Cell type:markdown id: tags:
Wrapping C code
--------------------------
%% Cell type:markdown id: tags:
They are different ways of wrapping C code. We present CFFI.
The workflow is the following:
1. Get your C code working (with a .c and a .h)
2. Set up your packaging to compile your code as a module
3. Compile your code
4. In the python code, declare the function you will be using
5. In the python code, open/load the compiled module
6. Use your functions
%% Cell type:markdown id: tags:
**1. Get your C code working (with a .c and a .h)**
Ok, supposed to be done
%% Cell type:markdown id: tags:
**2. Set up your packaging to compile your code as a module**
We give the compilation instructions in the file setup.py:
%% Cell type:raw id: tags:
from setuptools import setup, Extension
version = "0.1"
module_distance = Extension(
name="cdtw",
sources=["cdtw.c"],
)
setup(
name="dtw_cort_dist_mat",
version=version,
description="data scientist tool for time series",
long_description="data scientist tool for time series",
classifiers=[],
author="Robert Bidochon",
author_email="robert@bidochon.fr",
license="GPL",
include_package_data=True,
install_requires=["cffi", "numpy", "setuptools"],
entry_points="",
ext_modules=[module_distance],
)
%% Cell type:markdown id: tags:
**3. Compile your code**
%% Cell type:markdown id: tags:
in a terminal, type:
python3 setup.py build_ext
%% Cell type:markdown id: tags:
**4. In the python code, declare the function you will be using**
%% Cell type:code id: tags:
``` python
from cffi import FFI
ffi = FFI()
ffi.cdef("double square(double x, double y);")
```
%% Cell type:markdown id: tags:
**5. In the python code, open/load the compiled module**
%% Cell type:raw id: tags:
dllib = ffi.dlopen(
str(my_dir / ("cdtw" + sysconfig.get_config_var("EXT_SUFFIX")))
)
%% Cell type:markdown id: tags:
**6. Use your functions**
%% Cell type:markdown id: tags:
sq = dllib.square(2.0, 3.0)
......
Supports Markdown
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment