Skip to content
GitLab
Menu
Projects
Groups
Snippets
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in
Toggle navigation
Menu
Open sidebar
python-uga
training-hpc
Commits
58fdca8e
Commit
58fdca8e
authored
Jun 13, 2019
by
Raphael Bacher
Browse files
Merge branch 'master' of
https://gricad-gitlab.univ-grenoble-alpes.fr/python-uga/training-hpc
parents
4cd6775c
b680e8e9
Changes
4
Expand all
Hide whitespace changes
Inline
Side-by-side
ipynb/12_pandas.ipynb
View file @
58fdca8e
This diff is collapsed.
Click to expand it.
This diff is collapsed.
Click to expand it.
ipynb/21_pres_algos_dtw_cort.ipynb
View file @
58fdca8e
...
...
@@ -2,7 +2,11 @@
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"A running example\n",
"================"
...
...
@@ -30,7 +34,11 @@
},
{
"cell_type": "markdown",
"metadata": {},
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"Dynamic Time Wrapping \n",
"------------------------------------\n",
...
...
@@ -43,17 +51,16 @@
"\n",
"What do we compute: \n",
"-------------------------------\n",
"The transformation (with minimal cost) to transform one serie in the other one. \n",
"\n",
"\n",
"Example of what is computed\n",
"-------------------------------------------\n",
"\n"
"The transformation (with minimal cost) to transform one serie in the other one. "
]
},
{
"cell_type": "markdown",
"metadata": {},
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"Example of result of dtw :\n",
"-----------------------------------\n",
...
...
@@ -65,7 +72,11 @@
},
{
"cell_type": "markdown",
"metadata": {},
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"\n",
"Idea of the algorithm\n",
...
...
@@ -92,7 +103,11 @@
},
{
"cell_type": "markdown",
"metadata": {},
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"(image from https://riptutorial.com/dynamic-programming/example/25780/introduction-to-dynamic-time-warping\n",
"\n",
...
...
@@ -110,7 +125,11 @@
},
{
"cell_type": "markdown",
"metadata": {},
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"Easy enough to implement:\n",
"----------------------------------------"
...
...
@@ -159,7 +178,11 @@
{
"cell_type": "code",
"execution_count": 5,
"metadata": {},
"metadata": {
"slideshow": {
"slide_type": "-"
}
},
"outputs": [
{
"name": "stdout",
...
...
@@ -198,7 +221,11 @@
},
{
"cell_type": "markdown",
"metadata": {},
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"\n",
"Cort \n",
...
...
@@ -223,9 +250,13 @@
},
{
"cell_type": "markdown",
"metadata": {},
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"
What
do we compute:\n",
"
How
do we compute:\n",
"------------------------------\n",
"\n",
"$$ \\begin{eqnarray}\n",
...
...
@@ -238,7 +269,11 @@
},
{
"cell_type": "markdown",
"metadata": {},
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"Easy enough to implement:\n",
"---------------------------------------"
...
...
@@ -296,6 +331,7 @@
}
],
"metadata": {
"celltoolbar": "Slideshow",
"kernelspec": {
"display_name": "Python 3",
"language": "python",
...
...
%% Cell type:markdown id: tags:
A running example
================
%% Cell type:markdown id: tags:
Algorithm on series (
*e.g.*
time series, character strings, ...)
A series is a one dimensional ordered series of items (e.g. an numerical array, a string).
We want to compute a dissimilarity measure between series. The measure can either apply to series of
same length or not, and can be a metric (i.e. symmetric, $d(x, y) = 0
\i
ff x = y$, triangular inequality).
We consider two (dis)similarity measures with different features.
S1 and S2 two series of length |S1| and |S2|
-
**dtw**
: similarity measure: dynamic time wrapping, complexity O(|S1|
*
|S2|)
-
**cort**
: normalized cosine similarity measure between derivatives, complexity O(|S1| + |S2|)
%% Cell type:markdown id: tags:
Dynamic Time Wrapping
------------------------------------
-
**Input:**
two series, S1 and S2, not necessarily of same length
-
**Output:**
a dissimilarity measure
-
**Complexity:**
O([S1|
*
|S2|)
-
**Metric:**
no: does not respect the triangular inequality
-
**Side product:**
an alignment between the series can be stored
What do we compute:
-------------------------------
The transformation (with minimal cost) to transform one serie in the other one.
Example of what is computed
-------------------------------------------
%% Cell type:markdown id: tags:
Example of result of dtw :
-----------------------------------
<div
align=
"middle"
>
<img
src=
"./fig/dtw_example.png"
style=
"width: 100%"
>
</div>
%% Cell type:markdown id: tags:
Idea of the algorithm
------------------------------
Inspired from
https://riptutorial.com/dynamic-programming/example/25780/introduction-to-dynamic-time-warping
Let $a$ and $b$ be two series. We have:
-
dtw is a dynamic programming algorithm: the solution is built incrementally
-
a table $t$ is incrementally filled.
-
the value of the cell $t[i, j]$ holds the
*distance*
between the sub series $a[:i]$ and $b[:j]$
-
the value of the cell $t[i, j]$ is computed using the values of cells $t[i-1, j]$, $t[i, j-1]$ and $t[i-1, j-1]$:
$$t[i, j] = d(i, j) + min(t[i-1, j], t[i-1, j-1], t[i, j-1])$$
where $d(i, j)$ is the distance between $s[i]$ and $s[j]$ (we will use the absolute difference)
An example with two series [0, 1, 1, 2, 2, 3, 5] and [0, 1, 2, 3, 5, 5, 5, 6]
%% Cell type:markdown id: tags:
(image from https://riptutorial.com/dynamic-programming/example/25780/introduction-to-dynamic-time-warping
<div
align=
"middle"
>
<img
src=
"./fig/dtw_ex_table.jpg"
style=
"width: 100%"
>
</div>
%% Cell type:markdown id: tags:
why 6 in the t[-1, -1] ?
%% Cell type:markdown id: tags:
Easy enough to implement:
----------------------------------------
%% Cell type:code id: tags:
```
python
import
numpy
as
np
def
DTWDistance_pure_python
(
s1
,
s2
):
""" Computes the dtw between s1 and s2 with distance the absolute distance
:param s1: the first series (ie an iterable over floats64)
:param s2: the second series (ie an iterable over floats64)
:returns: the dtw distance
:rtype: float64
"""
_dtw_mat
=
np
.
empty
([
len
(
s1
),
len
(
s2
)])
_dtw_mat
[
0
,
0
]
=
abs
(
s1
[
0
]
-
s2
[
0
])
# two special cases : filling first row and columns
for
j
in
range
(
1
,
len
(
s2
)):
dist
=
abs
(
s1
[
0
]
-
s2
[
j
])
_dtw_mat
[
0
,
j
]
=
dist
+
_dtw_mat
[
0
,
j
-
1
]
for
i
in
range
(
1
,
len
(
s1
)):
dist
=
abs
(
s1
[
i
]
-
s2
[
0
])
_dtw_mat
[
i
,
0
]
=
dist
+
_dtw_mat
[(
i
-
1
,
0
)]
# filling the matrix
for
i
in
range
(
1
,
len
(
s1
)):
for
j
in
range
(
1
,
len
(
s2
)):
dist
=
abs
(
s1
[
i
]
-
s2
[
j
])
_dtw_mat
[(
i
,
j
)]
=
dist
+
min
(
_dtw_mat
[
i
-
1
,
j
],
_dtw_mat
[
i
,
j
-
1
],
_dtw_mat
[
i
-
1
,
j
-
1
])
return
_dtw_mat
[
len
(
s1
)
-
1
,
len
(
s2
)
-
1
],
_dtw_mat
```
%% Cell type:code id: tags:
```
python
x
=
[
1
,
2
,
3
,
5
,
5
,
5
,
6
]
y
=
[
1
,
1
,
2
,
2
,
3
,
5
]
nx
=
len
(
x
)
ny
=
len
(
y
)
d
,
mat
=
DTWDistance_pure_python
(
x
,
y
)
print
(
d
)
mat
```
%%%% Output: stream
1.0
%%%% Output: execute_result
array([[ 0., 0., 1., 2., 4., 8.],
[ 1., 1., 0., 0., 1., 4.],
[ 3., 3., 1., 1., 0., 2.],
[ 7., 7., 4., 4., 2., 0.],
[11., 11., 7., 7., 4., 0.],
[15., 15., 10., 10., 6., 0.],
[20., 20., 14., 14., 9., 1.]])
%% Cell type:markdown id: tags:
Cort
-------
**Input**
: two series S1 and S2
*of same length*
**Output:**
a similarity measure
**Complexity:**
O(|S1|+|S2|)
**Metric:**
yes
What do we compute:
-------------------------------
The cosine similarity measure between derivatives of the series.
%% Cell type:markdown id: tags:
What
do we compute:
How
do we compute:
------------------------------
$$
\b
egin{eqnarray}
cort(A, B) &=&
\c
os(dA, dB)
\\
&=&
\f
rac{dA
\c
dot dB}{
\V
ert dA
\V
ert
\V
ert dB
\V
ert}
\\
&=&
\f
rac{
\s
um_{i=0}^{T} dA_i dB_i}{
\V
ert dA
\V
ert
\V
ert dB
\V
ert}
\\
&=&
\f
rac{
\s
um_{i=0}^{T-1} (A_{i+1}-A_i) (B_{i+1}-B_i)}{
\s
qrt{
\s
um_{i=0}^{T-1} (A_{i+1}-A_i)^2}
\s
qrt{
\s
um_{i=0}^{T-1} (B_{i+1}-B_i)^2}}
\e
nd{eqnarray} $$
%% Cell type:markdown id: tags:
Easy enough to implement:
---------------------------------------
%% Cell type:code id: tags:
```
python
from
math
import
sqrt
def
cort
(
s1
,
s2
):
""" Computes the cort between series one and two (assuming they have the same length)
:param s1: the first series (or any iterable over floats64)
:param s2: the second series (or any iterable over floats64)
:returns: the cort distance
:rtype: float
:precondition: series are assumed to be of same size
"""
num
=
0.0
sum_square_x
=
0.0
sum_square_y
=
0.0
for
t
in
range
(
len
(
s1
)
-
1
):
slope_1
=
s1
[
t
+
1
]
-
s1
[
t
]
slope_2
=
s2
[
t
+
1
]
-
s2
[
t
]
num
=
num
+
slope_1
*
slope_2
sum_square_x
=
sum_square_x
+
(
slope_1
*
slope_1
)
sum_square_y
=
sum_square_y
+
(
slope_2
*
slope_2
)
return
num
/
(
sqrt
(
sum_square_x
*
sum_square_y
))
```
%% Cell type:code id: tags:
```
python
x
=
[
1
,
2
,
3
,
5
,
5
,
6
]
y
=
[
1
,
1
,
2
,
2
,
3
,
5
]
print
(
f
"cort(x,2*x)=
{
cort
(
x
,
2
*
x
)
}
cort([1,2], [2,1])=
{
cort
([
1
,
2
],
[
2
,
1
])
}
"
)
```
%%%% Output: stream
cort(x,2*x)=1.0 cort([1,2], [2,1])=-1.0
...
...
requirements.txt
View file @
58fdca8e
# to be used with:
# conda install --file requirements.txt
# utilities
python
==3.7
python
==3.7
.3
ipython
spyder
jupyter
nbconvert
>=5.5
invoke
# base numerics
numpy
...
...
@@ -13,7 +14,7 @@ matplotlib
pandas
h5py
#vizu
pyviz
#
pyviz
# (not available in conda!)
# wrapper
cffi
# accelerators
...
...
todo.md
View file @
58fdca8e
...
...
@@ -2,18 +2,11 @@
-
[?] Make a page in our website with the presentations
-
[Franck] Split 21_pres_algos_dtw_cort.ipynb in more than one page!
-
[Franck, Loic] 30_wrapping.ipynb (que retenir des exemples?)
-
[Pierre, Franck] 31_accelerators.ipynb (que retenir des exemples?)
-
[Pierre] Pythran without Transonic, (loop and vec) for Transonic and Numba
-
[Pierre, Franck] parallel (presentation + examples)
-
[Franck, Raphaël, Cyrille] présentation 41_cluster.ipynb (Charliecloud pour
Cyrille)
-
[Franck] Cleanup
-
old_snippet and delete this directory
Write
Preview
Supports
Markdown
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment