Vous avez reçu un message "Your GitLab account has been locked ..." ? Pas d'inquiétude : lisez cet article https://docs.gricad-pages.univ-grenoble-alpes.fr/help/unlock/

scripts.rst 65.8 KB
Newer Older
Olga Stamati's avatar
Olga Stamati committed
1
.. _scriptsTutorial:
2

Olga Stamati's avatar
Olga Stamati committed
3
*******************
4
Scripts in spam
Olga Stamati's avatar
Olga Stamati committed
5
*******************
6

Olga Stamati's avatar
Olga Stamati committed
7
8
9
10
11
12
13
14
.. graphviz:: graphs/DIC.dot

|
|
|
|
|

Olga Stamati's avatar
Olga Stamati committed
15
**Spam** provides scripts that tie together a number of the functions available in python.
16
17
18
These are expected to be called from the command line (after activating your virtual environment):


19
.. code-block:: console
20

21
22
    $ source /path/to/spam/venv/bin/activate
    (spam) $ spam-ldic -activateSomething -setSomething 5 aFileName.tif
23

24

Olga Stamati's avatar
Olga Stamati committed
25
All scripts will report all their options with a short explanation if they are simply run with **- -help**:
26

27
.. code-block:: console
28

29
30
    $ source /path/to/spam/venv/bin/activate
    (spam) $ spam-ldic --help
31

32
33
.. At the moment the most developed scripts are for local image correlation -- they take advantage of the `spam.DIC.lucasKanade` and handle input and outputs of 2D or 3D volumes (see below).

34
35
36
37
38
39
.. note::
  Almost all scripts take in images in the form of **one TIFF file** per state.
  If your 3D data is in the form of a series of slices, please save them as a single 3D TIFF file.
  This can be done by opening it as a stack in ImageJ/Fiji and saving directly,
  or loading each slice into a 3D numpy array and saving it with `tifffile.imsave`

Olga Stamati's avatar
Olga Stamati committed
40
Here's a list of existing **spam** scripts organised into categories:
41

42
    - Registration:
43

44
        - **spam-ereg** — Eye-alignment tool for pre-registration alignment
Olga Stamati's avatar
Olga Stamati committed
45
46
        - **spam-reg**  — Registration between two images, measuring a **single** Φ (homogeneous deformation function) that maps im1 into im2, see: :ref:`reg`

47
48
        - **spam-mmr** — Multi-modal registration for images acquired in different modalities
        - **spam-mmr-graphical** — Graphical version of the above
49

50
    - Regular grid correlation:
51

Olga Stamati's avatar
Olga Stamati committed
52
        - **spam-ldic** — "Standard" local image correlation with independent subvolumes distributed on a regular grid, see: :ref:`ldic`
53
        - **spam-regularStrain** — Takes in displacements from a TSV file from above and computes strains :ref:`regularStrainScript`
54

55
    - Global correlation:
56

57
58
        - **spam-gdic** — Global image correlation in a tetrahedral mesh, see: :ref:`globalDICscript`
        - **spam-deformImageFromField** — Deforms an image from a registration TSV file or a displacement field interpolated with a tetrahedral mesh
59

60
    - Labelled/discrete operations:
61

62
63
64
        - **spam-ITKwatershed** — Morphological watershed from ITK, requires binary image as input
        - **spam-ddic** — Discrete image correlation with discrete objects defined with labelled image see: :ref:`discreteDICscript`
        - **spam-moveGrains** — Deform a labelled image with displacements defined for each label
65
        - **spam-discreteStrain** — Computes strains on a tetrahedral mesh linking grain centres
Edward Andò's avatar
Edward Andò committed
66
        - **spam-ereg-discrete** — A graphical tool to pre-align grains for correlation
67

Olga Stamati's avatar
Olga Stamati committed
68
69
70
71
72
73
|
|
|
|
|

74
75
Description of image correlation scripts
#########################################
Olga Stamati's avatar
Olga Stamati committed
76
77
78
|
|
.. _ereg:
79

Olga Stamati's avatar
Olga Stamati committed
80
81
Eye-registration script (spam-ereg)
------------------------------------
82

Olga Stamati's avatar
Olga Stamati committed
83
This script performs an **alignment by eye** between two images.
84
Since a direct involvement of the user is needed, the alignment is done using a **graphical user interface** (GUI).
Olga Stamati's avatar
Olga Stamati committed
85

Olga Stamati's avatar
Olga Stamati committed
86
87
The result of this eye-alignment is a **single** Φ (linear and homogeneous deformation function, see :ref:`imageCorrelationTheory`) that maps im1 into im2.

Olga Stamati's avatar
Olga Stamati committed
88
89
.. tip:: When to use this script?

Olga Stamati's avatar
Olga Stamati committed
90
91
92
93
94
95
            This pre-registration step can be very useful when the images are **completely misaligned**.
            Such a state could look like this:

        .. figure:: images/tutorial/scripts/eregInput.png
                    :align: center

Olga Stamati's avatar
Olga Stamati committed
96
97

``Input:``
Olga Stamati's avatar
Olga Stamati committed
98
.............
Olga Stamati's avatar
Olga Stamati committed
99
100
101
102
103
104
105
106
107
108
109
110

    **Required**

        - **Two TIFF greyscale images**, representing the initial (im1) and the deformed (im2) configuration

    **Optional**

        -  A *".tsv"* file containing an initial guess of the transformation between im1 and im2

            .. HINT:: Make sure that the format of this file is compatible with a spam-style correlation file. (Run, for example, this script and check the result)

``Output:``
Olga Stamati's avatar
Olga Stamati committed
111
.............
Olga Stamati's avatar
Olga Stamati committed
112

113
114
    - A *".tsv"* file containing:

115
        - The point where Φ is applied. This is the centre of im1
116
117
        - The components of Φ (*Fzz*, *Fzy*, *Fzx*, *Zdisp*, *etc*). See :ref:`imageCorrelationTheory` for explanation of these components
        - For compatibility reasons, information related to the iterative algorithm in ``spam.DIC.register()`` (*iterations*, *error*, *deltaPhiNorm*, *returnStatus*)
Olga Stamati's avatar
Olga Stamati committed
118
119
120
121
122
123
124

        .. HINT:: This file can be used as an **input** to other correlation scripts, like:

                - :ref:`register`
                - :ref:`pixelSearch`
                - :ref:`ldic`

125
126
    .. ATTENTION:: Should we have an option to save the deformed? 
                    A *".tif"* file corresponding to image 1 deformed by Φ
Olga Stamati's avatar
Olga Stamati committed
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156

The script can be run like this:

(1) Input files are selected through the pop-up window:

.. code-block:: console

    $ source /path/to/spam/venv/bin/activate
    (spam) $ spam-ereg \                                            # the script


(2) Input images are given in the terminal:

.. code-block:: console

    $ source /path/to/spam/venv/bin/activate
    (spam) $ spam-ereg \                                            # the script
            /path/to/my/data/im1.tif /path/to/my/data/im2.tif       # the two 3D images (tiff files) to allign

(3) Input images and the initial guess are given in the terminal:

.. code-block:: console

    $ source /path/to/spam/venv/bin/activate
    (spam) $ spam-ereg \                                            # the script
            /path/to/my/data/im1.tif /path/to/my/data/im2.tif\      # the two 3D images (tiff files) to allign
            /path/to/my/data/initialPhi.tsv                         # and the initial guess

.. tip:: How to evaluate the result of this script?:

157
            Check the **greylevel residual fields**.
158
            If not saved, create the deformed image 1 by running ``spam-deformImageFromField``.
Olga Stamati's avatar
Olga Stamati committed
159
160
161
162
163
164
165
166
            Compare the initial difference (im1.tif - im2.tif) and the new difference (im1Def.tif - im2.tif), which should look like this:

            .. figure:: images/tutorial/scripts/eregResult.png
                :align: center

        Are you satisfied with this alignment?

            If yes, and you wish to **measure** the deformation between your two images, **keep this Φ** and move towards :ref:`register`, :ref:`pixelSearch` or :ref:`ldic`.
Olga Stamati's avatar
Olga Stamati committed
167
168
169
170
171
172
173
174
175
176

|
|
|
|
|

.. _register:

Registration script (spam-reg)
177
178
---------------------------------------------

Olga Stamati's avatar
Olga Stamati committed
179
This script performs a *non-rigid registration* between two images of the same modality.
180

Olga Stamati's avatar
Olga Stamati committed
181
182
183
184
It measures a **single** Φ (linear and homogeneous deformation function, see :ref:`imageCorrelationTheory`) that maps im1 into im2.

.. tip:: When to use this script?

Olga Stamati's avatar
Olga Stamati committed
185
186
187
188
189
190
            The registration step can be very useful when there is a **smooth** and **homogeneous** deformation between the two configurations. 
            Such a deformation could look like this:

        .. figure:: images/tutorial/scripts/regInput.png
                    :align: center

Olga Stamati's avatar
Olga Stamati committed
191
192

.. IMPORTANT:: This script is actually a wraper for ``spam.DIC.registerMultiscale()`` (see here_).
193
               It automatically performs a **multiscale registration**, starting from **4 times downscaled** images propagating the result to the top scale. To change this behaviour set ``-bb`` and ``-be`` in the input parameters
Olga Stamati's avatar
Olga Stamati committed
194
.. _here: https://ttk.gricad-pages.univ-grenoble-alpes.fr/spam/DIC.html#DIC.correlate.registerMultiscale
195

Olga Stamati's avatar
Olga Stamati committed
196
``Input:``
Olga Stamati's avatar
Olga Stamati committed
197
.............
198

Olga Stamati's avatar
Olga Stamati committed
199
    **Required**
200

Olga Stamati's avatar
Olga Stamati committed
201
202
203
204
205
206
        - **Two 2D or 3D TIFF greyscale images**, representing the initial (im1) and the deformed (im2) configuration

    **Optional**

        -  ``-pf``:  *".tsv"* file containing an initial guess of the transformation between im1 and im2.

207
            .. HINT:: This can come from :ref:`ereg` or a previous :ref:`register`.  If this guess corresponds to downscaled images, use ``-pfb`` to set the binning ratio
Olga Stamati's avatar
Olga Stamati committed
208
209
210
211
212
213
214

        - ``-bb`` and ``-be``: Binning levels (downscaling) to start (``-bb``) and end (``-be``) the multiscale procedure

            .. HINT:: If you **do not** want to downscale your images set ``-bb`` to 1. By default ``-be`` is the original scale of the input images

        - ``-mf1``: **Mask** of **im1** defining the zones to correlate in im1. Should be a **binary** image (same size as im1), where *False* pixels are not correlated
        - ``-m``: **Margin** (in pixels) to allow **space for interpolation** and movement. Can be set different in each direction (Z, Y, X) with ``-m3``
215
216
217
218
        - ``-rig``: Performs a **rigid** registration. Only translations and rotations will be measured
        - ``-ug``: **Updates gradient** during newton iterations inside ``spam.DIC.register()``. More computation time but sometimes more robust and possibly fewer iterations
        - ``-it``: Regulates **convergence options** for ``spam.DIC.register()``: The number of maximum iterations
        - ``-dp``: Regulates **convergence options** for ``spam.DIC.register()``: The smallest change in the norm of Φ (:math:`\delta\Phi_{min}`)
Olga Stamati's avatar
Olga Stamati committed
219
220
221
222
223
224
225
        - ``-o``:  Defines the order of greylevel **interpolation**; trilinear (``-o 1``) or cubic (``-o 3``)
        - ``-g``: Will plot **graphs** showing the **progress** of the registration
        - ``-od``: Defines the **output** directory
        - ``-pre``: Defines the prefix for the **output** files
        - ``-def``: Ask to **save** the **deformed** im1 at the end of the iterative algorithm. This is the measured Φ applied to im1

``Output:``
Olga Stamati's avatar
Olga Stamati committed
226
.............
Olga Stamati's avatar
Olga Stamati committed
227
228
229
230
231
232
233

    - A *".tsv"* file containing:

        - The point where Φ was measured. This is the centre of im1 or the centre of mass of the mask (if given)
        - The components of the measured deformation function Φ (*Fzz*, *Fzy*, *Fzx*, *Zdisp*, *etc*). See :ref:`imageCorrelationTheory` for explanation of these components
        - Information related to the iterative algorithm status (*iterations*, *error*, *returnStatus*, *deltaPhiNorm*)

Olga Stamati's avatar
Olga Stamati committed
234
235
236
237
238
239
240
        .. HINT:: This file can be used as an **input** to other correlation scripts, like:

                - :ref:`register`
                - :ref:`pixelSearch`
                - :ref:`ldic`


Olga Stamati's avatar
Olga Stamati committed
241
242
243
    - If asked with ``-def``: A *".tif"* file corresponding to im1 deformed by the measured Φ

The script can be run like this:
244

245
.. code-block:: console
246

247
    $ source /path/to/spam/venv/bin/activate
Olga Stamati's avatar
Olga Stamati committed
248
249
250
251
    (spam) $ spam-reg \                                              # the script
                /path/to/my/data/im1.tif /path/to/my/data/im2.tif\   # the two 3D images (tiff files) to correlate
                -pf /path/to/my/data/initialPhi.tsv\                 # initial guess
                -bb 2 -m 10 -def                                     # ask to start from half-sized images, set margin, ask for deformed image
252

Olga Stamati's avatar
Olga Stamati committed
253
.. tip:: How to evaluate the result of this script?:
254

Olga Stamati's avatar
Olga Stamati committed
255
            - The registration is **successful** when the iterative algorithm has **converged**. Check **returnStatus** inside the *".tsv"* file:
256

257
258
259
260
261
                - If **returnStatus =  2**: Registration **converged** reaching desired precision in :math:`\delta\Phi_{min}`
                - If **returnStatus =  1**: Registration **did not converge**. Maximum number of iterations is reached. Retry with a higher ``-it`` or higher ``-dp``?
                - If **returnStatus = -1**: Registration **diverged**. Singular matrix M cannot be inverted. Retry with a mask ``-mf1``? Or different binning level ``-bb``? Or perhaps updating the gradient ``-ug``?
                - If **returnStatus = -2**: Registration **diverged**. Error is more than 80% of previous iteration error. Retry with different binning level ``-bb`` or higher margin ``-m``?
                - If **returnStatus = -3**: Registration **diverged** on displacement or volumetric change condition. Retry with a higher margin ``-m``?
262

Olga Stamati's avatar
Olga Stamati committed
263
264
265
266
267
268
            - Compare the initial difference (im1.tif - im2.tif) and the new difference (im1Def.tif - im2.tif). It should look like this:

                .. figure:: images/tutorial/scripts/regResult.png
                        :align: center 


269
                Has the difference been decreased?
Olga Stamati's avatar
Olga Stamati committed
270

271
        Are you satisfied with this **overall** deformation?
Olga Stamati's avatar
Olga Stamati committed
272

273
            If yes, and you wish to go and look **locally**, **keep this Φ** and move towards :ref:`pixelSearch` or :ref:`ldic`.
274

Olga Stamati's avatar
Olga Stamati committed
275
276
277
278
279
|
|
|
|
|
280

Olga Stamati's avatar
Olga Stamati committed
281
.. _pixelSearch:
282

Olga Stamati's avatar
Olga Stamati committed
283
284
Pixel search script (spam-pixelSearch)
---------------------------------------
285

286
This script performs a point-by-point "pixel search" which computes a **cross-correlation** of a subvolume extracted in im1 based on a **brute-force search** in a given search range in im2.
Olga Stamati's avatar
Olga Stamati committed
287

Olga Stamati's avatar
Olga Stamati committed
288
It measures a **displacement field** with a **1-pixel sensitivity** that maps im1 into im2.
289

Olga Stamati's avatar
Olga Stamati committed
290
.. tip:: When to use this script?
291

292
            This step can be very useful when the deformation between the two configurations is **not homogeneous**, meaning that it can **not** be captured only by a single Φ (coming from :ref:`register` for example).
Olga Stamati's avatar
Olga Stamati committed
293
294
            Such a deformation could look like this:

295
            
296
        If you notice that there is also a **homogeneous** deformation (a rigid rotation for example), run :ref:`ereg`, or :ref:`register` and pass Φ as an initial guess to this script
297

298
.. IMPORTANT::
Olga Stamati's avatar
Olga Stamati committed
299
        This script can run in **two** different **modes**. It can measure a displacement field of:
Olga Stamati's avatar
Olga Stamati committed
300

301
302
            1. Regularly-spaced points laid on a **structured grid**. See :ref:`grid` for tips regarding the grid geometry. This mode is activated with ``-ns``, ``-hws`` input options
            2. **Discrete** points. For example centres of mass of particles, if your objective is to perform particle tracking. See :ref:`labelToolkitTutorial` for more information. This mode is activated with ``-lab1`` input option
Olga Stamati's avatar
Olga Stamati committed
303
304

``Input:``
Olga Stamati's avatar
Olga Stamati committed
305
.............
Olga Stamati's avatar
Olga Stamati committed
306
307
308
309
310

    **Required**

        - **Two TIFF greyscale images**, representing the initial (im1) and the deformed (im2) configuration:

311
                1. If **structured grid** mode: The images can be 2D or 3D
312
                2. If **discrete** mode: The **labelled** image, which defines zones to correlate, **must be given** through the ``-lab1`` input parameter. Only for 3D images for the moment. If ``-lab1`` passed: ``-ns``, ``-hws`` are disactivated
Olga Stamati's avatar
Olga Stamati committed
313
314

    **Optional**
315
316
    
        - ``-np``: Number of **parallel processes** to use. If not passed, all the CPUs available in the system will be used
317
        - ``-pf``: *".tsv"* file containing an initial guess of the transformation between im1 and im2
Olga Stamati's avatar
Olga Stamati committed
318
319
320

            .. HINT:: This can be either:

321
                        1. A **single** deformation function (one-line file). This is typically the result of :ref:`ereg` or :ref:`register`. If this guess corresponds to downscaled images, use ``-pfb`` to set the binning ratio
322
323
324
                        2. A **field** of deformation functions (multiple-line file). For example, the result of a previous :ref:`pixelSearch`. Make sure that points' positions at input file are the same as points' positions at the current computation. Consider using :ref:`passPhi`

        - ``-F``: If a ``-pf`` is passed. Choose if the **F part** of Φ (see :ref:`imageCorrelationTheory`) will be **applied** to the subvolume. If "rigid" only rotations are applied. If "all" everything is applied. If "no" nothing is applied.
Olga Stamati's avatar
Olga Stamati committed
325
326
        - ``-sr``: **Search range** [-Z Z -Y Y -X X] (in pixels). How far away from the point's position in im1 should we look in im2?
        - ``-od``:  Defines the **output** directory
327
        - ``-pre``: Defines the prefix for the **output** files
Olga Stamati's avatar
Olga Stamati committed
328
329
        - ``-vtk``: Ask for a vtk **output** for **visualisation** of the displacement field

330
    For **structured grid** mode:
Olga Stamati's avatar
Olga Stamati committed
331
332
        - ``-ns``: **Spacing** between the grid **points** (in pixels). Can be set different in each direction (Z, Y, X) with ``-ns3``
        - ``-hws``: "Half window **size**" (in pixels) of the **correlation window**. It results in a correlation window of 1+2xhws in each direction, centered at each point's position. Can be set different in each direction (Z, Y, X) with ``-hws3``
333
        - ``-mf1``: **Mask** of **im1** defining the zones to correlate in im1. Should be a **binary** image (same size as im1), where *False* pixels are not correlated
Olga Stamati's avatar
Olga Stamati committed
334
335
336
337
        - ``-mc``:  If *-mf1* is passed. It defines the tolerance for a subvolume's pixels to be masked before it is skipped
        - ``-mf2``: **Mask** of **im2** defining the zones to correlate in im2. Should be a **binary** image (same size as im2), where *False* pixels are not correlated
        - ``-glt``: Grey **threshold** on mean values of correlation window **below** which the correlation is not performed (window is **skipped**)
        - ``-ght``: Grey **threshold** on mean values of correlation window **above** which the correlation is not performed (window is **skipped**)
Olga Stamati's avatar
Olga Stamati committed
338
        - ``-tif``: Ask for a tif output for **visualisation** of the structured displacement field
Olga Stamati's avatar
Olga Stamati committed
339
340
341

    For **discrete** mode:
        - ``-ld``: Number of times to dilate labels before extracting them (see :ref:`labelToolkitTutorial`)
342
        - ``-vt``: **I thought that was an option!!**
Olga Stamati's avatar
Olga Stamati committed
343
344
345


``Output:``
Olga Stamati's avatar
Olga Stamati committed
346
.............
Olga Stamati's avatar
Olga Stamati committed
347
348
349

    - A *".tsv"* file containing:

350
        - The points **position** (Z, Y, X). For a **structured grid** mode positions are defined by the input grid. For a **discrete** mode positions are the centre of mass of each label
Olga Stamati's avatar
Olga Stamati committed
351
        - The **displacement** measured at each point
352
        - The **correlation coefficient** (*pixelSearchCC*) of each subvolume
353
        - The return status of each subvolume (*returnStatus*). If it is -5 the subvolume has been skipped (due to ``-glt``, ``-ght`` or ``-mf1``)
354
355
        - For compatibility reasons, the F part of Φ is written as the identity matrix (see :ref:`imageCorrelationTheory` for explanation of the components of Φ). This is **totally irrelevant** to the result of this script
        - For compatibility reasons, information related to the iterative algorithm in ``spam.DIC.register()`` (*iterations*, *error*, *deltaPhiNorm*). These are **totally irrelevant** to the result of this script
356
    - If asked with ``-tif`` or ``-vtk``: Files for the visualisation of the displacement and correlation coefficient fields
Olga Stamati's avatar
Olga Stamati committed
357

358
For the **structured grid** mode, the script can be run like this:
359

360
.. code-block:: console
361

Olga Stamati's avatar
Olga Stamati committed
362
363
364
    $ source /path/to/spam/venv/bin/activate
    (spam) $ spam-pixelSearch \                                      # the script
                /path/to/my/data/im1.tif /path/to/my/data/im2.tif \  # the two 3D or 2D images (tiff files) to correlate
Olga Stamati's avatar
Olga Stamati committed
365
                -mf1 /path/to/my/data/mask1.tif -mc 0.5 \            # the mask of im1 and the coverage to skip the subvolumes
366
367
                -pf /path/to/my/data/initialPhi.tsv -pfb 2 \         # initial guess, which was run in half-sized images
                -ns X -hws X \                                       # node spacing and half window size of the grid
Olga Stamati's avatar
Olga Stamati committed
368
369
                -sr -Z Z -Y Y -X X \                                 # search range
                -tif -vtk                                            # ask for tif and vtk output
370

Olga Stamati's avatar
Olga Stamati committed
371
For the **discrete** mode, the script can be run like this:
372

Olga Stamati's avatar
Olga Stamati committed
373
.. code-block:: console
374

Olga Stamati's avatar
Olga Stamati committed
375
376
377
    $ source /path/to/spam/venv/bin/activate
    (spam) $ spam-pixelSearch \                                      # the script
                /path/to/my/data/im1.tif /path/to/my/data/im2.tif\   # the two 3D images (tiff files) to correlate
378
379
380
381
                -lab1 /path/to/my/data/lab1.tif \                    # the labelled image 1
                -pf /path/to/my/data/initialPhi.tsv -pfb 2 \         # initial guess, which was run in half-sized images
                -F rigid \                                           # apply the rigid part of the initial guess to each label
                -sr -Z Z -Y Y -X X \                                 # search range
Olga Stamati's avatar
Olga Stamati committed
382
                -vtk                                                 # ask for a vtk output
383

Olga Stamati's avatar
Olga Stamati committed
384
.. tip:: How to evaluate the result of this script?
385

386
        - Check the **pixelSearchCC** values. A cross-correlation procedure is **successful** when the **correlation coefficient** is **close to 1**. Are these values low? Perhaps think of a coarser field (increase ``-ns``)? And then pass this to the finer one through `spam-passField`
Olga Stamati's avatar
Olga Stamati committed
387
388
389
        - Check the measured **displacements**. If you notice that the displacement values hit the limits of your input search range, retry by increasing this range
        - Check the measured **displacement fields**. Do the points move as you would expect? For example you are pulling your material from the bottom edge. Do the displacement vectors point downwards increasing from the top to the bottom of your field?
        - Check the **residual fields**. Use the measured displacement field to deform im1. Check the initial difference (im2.tif - im1.tif) and the new difference (im2.tif - im1Def.tif). 
390

Olga Stamati's avatar
Olga Stamati committed
391
    Are you satisfied with the measured displacement field?
392

393
        If yes, and you wish to refine it by measuring subpixel displacements, **keep this Φ field** and move towards :ref:`ldic` for a **structured grid** mode or :ref:`ddic` for a **discrete** mode.
394

Olga Stamati's avatar
Olga Stamati committed
395
396
397
398
399
400
|
|
|
|
|

Olga Stamati's avatar
Olga Stamati committed
401
402
403
404
405
.. _pixelSearchPropagate:

Pixel search propagate script (spam-pixelSearchPropagate)
----------------------------------------------------------

406
407
408
This script performs a **sequential** point-by-point "pixel search" which computes a **cross-correlation** of a subvolume extracted in im1
based on a **gaussian weighted good neighbours guess** in a given search range in im2.
It is based on https://gricad-gitlab.univ-grenoble-alpes.fr/DVC/pt4d.
Olga Stamati's avatar
Olga Stamati committed
409
410
411
412
413

It measures a **displacement field** with a **1-pixel sensitivity** that maps im1 into im2.

.. tip:: When to use this script?

414
            This can be seen as a *special* case of :ref:`pixelSearch`.
Olga Stamati's avatar
Olga Stamati committed
415
416
417
418
            It can be very useful when:

                - There are **large** and **not homogeneous** deformations between the two configurations and 
                - The **texture** of the material is **weak**
Olga Stamati's avatar
Olga Stamati committed
419
            Such a challenging case could look like this:
Olga Stamati's avatar
Olga Stamati committed
420
421
422
423
424

        .. figure:: images/tutorial/scripts/psPropInput.png
                    :align: center

.. ATTENTION::
425
        This script requires a **starting point** (``-sp`` input option) which should be **easily tracked by eye** between the two configurations.
Olga Stamati's avatar
Olga Stamati committed
426
427
        This is recommended to be either:

428
            - A point that **does not move** (for example a point close to the top border for the set of images above), or
429
430
431
            - A point that moves, but its **displacement** is easily measured **by hand** and given as an input

        If the **cross-correlation** of this point:
432

433
434
            - **Fails** (*ie*, correlation coefficient **below** given threshold ``-cct``), the **script stops**.  Try to give a better estimation of its displacement? Or perhaps try with another point?
            - **Succeeds**, continuing with processing its **neighbours** (see ``-nr`` and ``-gwd``) the motion is **progressively** propagated to the selected points (see note below) in im1
Olga Stamati's avatar
Olga Stamati committed
435
436
437
438

.. IMPORTANT::
        This script can run in **two** different **modes**. It can measure a displacement field of:

439
            1. Regularly-spaced points laid on a **structured grid**. See :ref:`grid` for tips regarding the grid geometry. This mode is activated with ``-ns``, ``-hws`` input options
Olga Stamati's avatar
Olga Stamati committed
440
441
            2. **Discrete** points. For example:

442
                - `Guiding points` selected based on their **strong** texture. This mode is activated with ``-gp`` input option
443
                - Centres of mass of particles, if your objective is to perform particle tracking. See :ref:`labelToolkitTutorial` for more information. This mode is activated with ``-lab1`` input option
Olga Stamati's avatar
Olga Stamati committed
444

445
        No matter the selected mode, points are **ranked** based on their distance to the **starting point** and there are processed **sequentially**.
446
        This means that this script can **not** run **in parallel**.
Olga Stamati's avatar
Olga Stamati committed
447
448

``Input:``
Olga Stamati's avatar
Olga Stamati committed
449
.............
Olga Stamati's avatar
Olga Stamati committed
450
451
452
453
454
455

    **Required**

        - **Two TIFF greyscale images**, representing the initial (im1) and the deformed (im2) configuration:

                1. If **regularly-spaced mode**: The images can be 2D or 3D
456
                2. If **discrete mode**: The **labelled** image, which defines zones to correlate, **must be given**, through the ``-lab1`` input parameter. Only for 3D images for the moment. If ``-lab1`` passed: ``-ns``, ``-hws``, ``-gp`` are disactivated
Olga Stamati's avatar
Olga Stamati committed
457
458
459
460
461
462
463
464

        - ``-sp``: **Starting point** [Z Y X Zdisp Ydisp Xdisp] (in pixels). The position and the displacement of the starting point

    **Optional**

        - ``-sr``: **Search range** [-Z Z -Y Y -X X] (in pixels). How far away from the point's position in im1 should we look in im2?
        - ``-nr``: **Radius** (in pixels) inside which to extract neighbours
        - ``-gwd``: **Distance** (in pixels) over which the neighbour's distance is weighted
465
        - ``-cct``: Pixel search **correlation coefficient** threshold **below** which the point is considered **badly** correlated (not taken as an neighbour). Recommended value above 0.9
Olga Stamati's avatar
Olga Stamati committed
466
        - ``-od``:  Defines the **output** directory
467
        - ``-pre``: Defines the prefix for the **output** files
Olga Stamati's avatar
Olga Stamati committed
468
469
        - ``-vtk``: Ask for a vtk **output** for **visualisation** of the displacement field

470
    For **structured grid** mode:
471
        - ``-ns``: **Spacing** between the grid **points** (in pixels). Can be set different in each direction (Z, Y, X) with ``-ns3``. These grid points will be **ranked** and then tracked based on their distance to the **starting point**
Olga Stamati's avatar
Olga Stamati committed
472
473
474
475
476
477
478
479
480
        - ``-hws``: "Half window **size**" (in pixels) of the **correlation window**. It results in a correlation window of 1+2xhws in each direction, centered at each point's position. Can be set different in each direction (Z, Y, X) with ``-hws3``
        - ``-mf1``: **Mask** of **im1** defining the zones to correlate in im1. Should be a **binary** image (same size as im1), where *False* pixels are not correlated. 
        - ``-mc``:  If *-mf1* is passed. It defines the tolerance for a subvolume's pixels to be masked before it is skipped
        - ``-mf2``: **Mask** of **im2** defining the zones to correlate in im2. Should be a **binary** image (same size as im2), where *False* pixels are not correlated
        - ``-glt``: Grey **threshold** on mean values of correlation window **below** which the correlation is not performed (window is **skipped**)
        - ``-ght``: Grey **threshold** on mean values of correlation window **above** which the correlation is not performed (window is **skipped**)
        - ``-tif``: Ask for a tif output for **visualisation** of the structured displacement field

    For **discrete** mode:
481
        - ``-gp``: **Guiding points** file. A file containing points the displacement of which will be measured
Olga Stamati's avatar
Olga Stamati committed
482
483
484

                .. HINT:: Recommended to correspond to points with **large greylevel gradients**, meaning that they can be easily tracked.
                          Typically coming from a *3D Harris Corner detection* algorithm.
485
486
                          These points are **ranked** based on their distance to the **starting point**.
                          If passed: ``-ns``, ``-hws``, ``-lab1`` are disactivated
Olga Stamati's avatar
Olga Stamati committed
487
488
489
490
491

        - ``-ld``: Only if ``-lab1`` is passed. Number of times to dilate labels before extracting them (see :ref:`labelToolkitTutorial`)


``Output:``
Olga Stamati's avatar
Olga Stamati committed
492
.............
Olga Stamati's avatar
Olga Stamati committed
493
494
495

    - A *".tsv"* file containing:

496
        - The points **position** (Z, Y, X). For a **structured grid mode** positions are defined by the input grid. For a **discrete mode** positions are the centre of mass of each label (if ``-lab1``) or the guiding points (if ``-gp``)
Olga Stamati's avatar
Olga Stamati committed
497
        - The **displacement** measured at each point
498
499
500
501
        - The **correlation coefficient** (*pixelSearchCC*) of each subvolume
        - The return status of each subvolume ()*returnStatus*). If it is -5 the subvolume had been skipped (due to ``-glt``, ``-ght`` or ``-mf1``)
        - For compatibility reasons, the F part of Φ is written as the identity matrix (see :ref:`imageCorrelationTheory` for explanation of the components of Φ). This is **totally irrelevant** to the result of this script
        - For compatibility reasons, information related to the iterative algorithm in ``spam.DIC.register()`` (*iterations*, *error*, *deltaPhiNorm*). These are **totally irrelevant** to the result of this script
502
    - If asked with ``-tif`` or ``-vtk``: Files for the visualisation of the displacement and correlation coefficient fields
Olga Stamati's avatar
Olga Stamati committed
503

504
For the **structured grid** mode, the script can be run like this:
Olga Stamati's avatar
Olga Stamati committed
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524

.. code-block:: console

    $ source /path/to/spam/venv/bin/activate
    (spam) $ spam-pixelSearchPropagate \                             # the script
                /path/to/my/data/im1.tif /path/to/my/data/im2.tif \  # the two 3D or 2D images (tiff files) to correlate
                -sp Z Y X w v u                                      # the position and displacement of the starting point
                -mf1 /path/to/my/data/im1.tif -mc 0.5 \              # the mask of im1, and the coverage to skip the subvolumes
                -nr X -gdw X \                                       # neighbour radius and gaussian distance
                -ns X -hws X \                                       # node spacing and window size of the grid
                -sr -Z Z -Y Y -X X \                                 # search range
                -cct X \                                             # correlation coefficient threshold
                -tif -vtk                                            # ask for tif and vtk output

For a **discrete** mode with a **labelled** image, the script can be run like this:

.. code-block:: console

    $ source /path/to/spam/venv/bin/activate
    (spam) $ spam-pixelSearchPropagate \                             # the script
525
                /path/to/my/data/im1.tif /path/to/my/data/im2.tif \  # the two 3D images (tiff files) to correlate
Olga Stamati's avatar
Olga Stamati committed
526
527
528
                -lab1 /path/to/my/data/lab1.tif                      # the labelled image 1
                -sp Z Y X w v u                                      # the position and displacement of the starting point
                -nr X -gdw X \                                       # neighbour radius and gaussian distance
529
                -sr -Z Z -Y Y -X X \                                 # search range
Olga Stamati's avatar
Olga Stamati committed
530
531
532
533
534
535
536
537
538
                -cct X \                                             # correlation coefficient threshold
                -vtk                                                 # ask for a vtk output

For a **discrete** mode with a **guiding points** file, the script can be run like this:

.. code-block:: console

    $ source /path/to/spam/venv/bin/activate
    (spam) $ spam-pixelSearchPropagate \                             # the script
539
540
541
                /path/to/my/data/im1.tif /path/to/my/data/im2.tif \  # the two 3D images (tiff files) to correlate
                -gp /path/to/my/data/guidingPoints.txt \             # the postions of the guiding points
                -sp Z Y X w v u \                                    # the position and displacement of the starting point
Olga Stamati's avatar
Olga Stamati committed
542
                -nr X -gdw X \                                       # neighbour radius and gaussian distance
543
                -sr -Z Z -Y Y -X X \                                 # search range
Olga Stamati's avatar
Olga Stamati committed
544
545
546
547
548
549
550
551
552
553
554
555
                -cct X \                                             # correlation coefficient threshold
                -vtk                                                 # ask for a vtk output

.. tip:: How to evaluate the result of this script?

        - Check the **pixelSearchCC** values. A cross-correlation procedure is **successful** when the **correlation coefficient** is close to 1. Are these values low? Perhaps thinking of a coarser field (increase ``-ns``)? And then pass this to the finer one through `spam-passField`
        - Check the measured **displacements**. If you notice that the displacement values hit the limits of your input search range, retry by increasing this range
        - Check the measured **displacement fields**. Do the points move as you would expect? For example you are pulling your material from the bottom edge. Do the displacement vectors point downwards increasing from the top to the bottom of your field?
        - Check the **residual fields**. Use the measured displacement field to deform im1. Check the initial difference (im2.tif - im1.tif) and the new difference (im2.tif - im1Def.tif). 

    Are you satisfied with the measured displacement field?

556
        If yes, and you wish to refine it by measuring subpixel displacements, **keep this Φ field** and move towards :ref:`ldic` for a **structured grid** mode or :ref:`ddic` for a **discrete** mode
Olga Stamati's avatar
Olga Stamati committed
557
558
559
560
561
562
563

|
|
|
|
|

Olga Stamati's avatar
Olga Stamati committed
564
565
566
567
568
.. _ldic:

Regular grid local DIC script (spam-ldic)
---------------------------------------------

569
570
571
This script performs **local** non-rigid correlations on a **structured grid**.
It defines a regular grid of points in im1 and runs independent correlations (calling ``spam.DIC.register()``) for small subvolumes (or "correlation windows") centred on each point.
See :ref:`grid` for tips regarding the grid geometry.
Olga Stamati's avatar
Olga Stamati committed
572
573
574
575
576

It measures a **field of deformation functions** that map each subvolume in im1 to im2.

.. tip:: When to use this script?

577
        This is the **final** step of the procedure to get a deformation field between two configurations, where we suppose we are already **close enough** to the actual **solution**.
Olga Stamati's avatar
Olga Stamati committed
578

579
        Such a state could look like this:
Olga Stamati's avatar
Olga Stamati committed
580
581
582
583

            .. figure:: images/tutorial/scripts/regInput.png
                    :align: center

584
585
.. IMPORTANT:: This script actually runs (in parallel) ``spam.DIC.register()`` (see function_) for every subvolume defined in the grid.
               For each **local** iterative **algorithm** to **converge** it is important to **start close** to the right **solution** (see :ref:`imageCorrelationTheory` for more details).
586
587
               It is, thus, **highly recommended** to give an **initial guess** of the deformation as **input** to this script.

588
               This guess can be either:
589
590

                    1. A **single** deformation function, when the **deformation** between im1 and im2 looks **smooth** and **homogeneous**. This is typically the result of :ref:`ereg` or :ref:`register`
591
592
593
                    2. A **field** of deformation functions, when the **deformation** between im1 and im2 does **not** look **homogeneous**. This field can be the result of :ref:`pixelSearch` or a previous computation of :ref:`ldic`.
                       Make sure that points' positions at input file are the same as points' positions at the current computation
                       Consider using :ref:`passPhi`
594
595

.. _function: https://ttk.gricad-pages.univ-grenoble-alpes.fr/spam/DIC.html#DIC.correlate.register
Olga Stamati's avatar
Olga Stamati committed
596

Olga Stamati's avatar
Olga Stamati committed
597
``Input:``
Olga Stamati's avatar
Olga Stamati committed
598
.............
Olga Stamati's avatar
Olga Stamati committed
599
600
601
602
603
604

    **Required**

        - **Two 2D or 3D TIFF greyscale images**, representing the initial (im1) and the deformed (im2) configuration

    **Optional**
605
        - ``-np``: Number of **parallel processes** to use. If not passed, all the CPUs available in the system will be used
Olga Stamati's avatar
Olga Stamati committed
606

607
        - ``-pf``: *".tsv"* file containing an initial guess of the transformation between im1 and im2
Olga Stamati's avatar
Olga Stamati committed
608

609
                   .. HINT:: This can be either:
Olga Stamati's avatar
Olga Stamati committed
610

611
612
                                1. A **single** deformation function (one-line file). This is typically the result of :ref:`ereg` or :ref:`register`. If this guess corresponds to downscaled images, use ``-pfb`` to set the binning ratio
                                2. A **field** of deformation functions (multiple-line file). The result of :ref:`pixelSearch`, or :ref:`pixelSearchPropagate` or a previous :ref:`ldic` computation. Make sure that points' positions at input file are the same as points' positions at the current computation. Consider using :ref:`passPhi`
Olga Stamati's avatar
Olga Stamati committed
613
614
615
616
617
618
619
        - ``-ns``: **Spacing** between the grid **points** (in pixels). Can be set different in each direction (Z, Y, X) with ``-ns3``
        - ``-hws``: "Half window **size**" (in pixels) of the **correlation window**. It results in a correlation window of 1+2xhws in each direction, centered at each point's position. Can be set different in each direction (Z, Y, X) with ``-hws3``
        - ``-mf1``: **Mask** of **im1** defining the zones to correlate in im1. Should be a **binary** image (same size as im1), where *False* pixels are not correlated. 
        - ``-mc``:  If *-mf1* is passed. It defines the tolerance for a subvolume's pixels to be masked before it is skipped
        - ``-glt``: Grey **threshold** on mean values of correlation window **below** which the correlation is not performed (window is **skipped**)
        - ``-ght``: Grey **threshold** on mean values of correlation window **above** which the correlation is not performed (window is **skipped**)
        - ``-m``: **Margin** (in pixels) to allow **space for interpolation** for each subvolume. Since we suppose we are close to the right solution, this is typically a **small** value. Can be set different in each direction (Z, Y, X) with ``-m3``
620
621
622
        - ``-ug``: Updates **gradient** during newton iterations inside ``spam.DIC.register()``. More computation time but sometimes more robust and possibly fewer iterations
        - ``-it``: Regulates **convergence options** for ``spam.DIC.register()``: The number of maximum iterations
        - ``-dp``: Regulates **convergence options** for ``spam.DIC.register()``: The smallest change in the norm of Φ (:math:`\delta\Phi_{min}`)
Olga Stamati's avatar
Olga Stamati committed
623
624
625
        - ``-o``:  Defines the order of greylevel **interpolation**; trilinear (``-o 1``) or cubic (``-o 3``)
        - ``-od``:  Defines the **output** directory
        - ``-pre``:  Defines the prefix for the **output** files
626
627
        - ``-vtk``: Ask for a vtk **output** for **visualisation** of the displacement and the iterative algorithm information fields
        - ``-tif``: Ask for a tif **output** for **visualisation** of the displacement and the iterative algorithm information fields
Olga Stamati's avatar
Olga Stamati committed
628
629

``Output:``
Olga Stamati's avatar
Olga Stamati committed
630
.............
Olga Stamati's avatar
Olga Stamati committed
631
632
633
634
635
636

    - A *".tsv"* file containing:

        - The points **position** (Z, Y, X), as defined by the input grid. These are the centres of each correlation window
        - The components of the measured deformation function inside each window (*Fzz*, *Fzy*, *Fzx*, *Zdisp*, *etc*). See :ref:`imageCorrelationTheory` for explanation of these components
        - Information related to the iterative algorithm status (*iterations*, *error*, *returnStatus*, *deltaPhiNorm*)
637
    - If asked with ``-tif`` or ``-vtk``: Files containing the displacement and the iterative algorithm's information as structured fields
Olga Stamati's avatar
Olga Stamati committed
638
639
640
641
642
643
644
645
646
647

The script can be run like this:

.. code-block:: console

    $ source /path/to/spam/venv/bin/activate
    (spam) $ spam-ldic \                                             # the script
                /path/to/my/data/im1.tif /path/to/my/data/im2.tif\   # the two 3D or 2D images (tiff files) to correlate
                -pf /path/to/my/data/initialPhi.tsv\                 # initial guess
                -ns X -hws X \                                       # node spacing and window size of the grid
648
649
                -glt X \                                             # set low greylevel threshold
                -dp X -it X \                                        # set convergence value for the norm of δΦ and maximum number of iterations
Olga Stamati's avatar
Olga Stamati committed
650
                -tif -vtk                                            # ask for tif and vtk output
651

Olga Stamati's avatar
Olga Stamati committed
652
653
.. tip:: How to evaluate the result of this script?

654
        - Each **local** computation is **successful** when the **iterative algorithm** has **converged**. Check the **returnStatus** field:
Olga Stamati's avatar
Olga Stamati committed
655

656
657
658
659
660
661
            - If **returnStatus =  2**: Registration **converged** reaching desired precision in :math:`\delta\Phi_{min}`
            - If **returnStatus =  1**: Registration **did not converge**. Maximum number of iterations is reached. Retry with a higher ``-it`` or higher ``-dp``?
            - If **returnStatus = -1**: Registration **diverged**. Singular matrix M cannot be inverted. Perhaps your window size is too small?
            - If **returnStatus = -2**: Registration **diverged**. Error is more than 80% of previous iteration error. Perhaps your window size is too small?
            - If **returnStatus = -3**: Registration **diverged** on displacement or volumetric change condition. Perhaps your window size is too small? Or retry with a higher ``-m``?
            - If **returnStatus = -5**: Registration **skipped** due to greylevel threshold condition or to mask
Olga Stamati's avatar
Olga Stamati committed
662
        - Check the measured **displacement fields**. Do the points move as you would expect? For example you are pulling your material from the bottom edge. Do the displacement vectors point downwards increasing from the top to the bottom of your field?
663
        - Check the **greylevel residual fields**. Create the deformed image 1 by running ``spam-deformImageFromField``. Compare the initial difference (im1.tif - im2.tif) and the new difference (im1Def.tif - im2.tif). It should look like this:
Olga Stamati's avatar
Olga Stamati committed
664

Olga Stamati's avatar
Olga Stamati committed
665
666
667
668
                .. figure:: images/tutorial/scripts/ldicResult.png
                        :align: center 

    Are you satisfied with the measured deformation field?
Olga Stamati's avatar
Olga Stamati committed
669

670
671
672
        - If *mostly* yes, but there are some **crazy points** here and there that you wish to get rid of, **keep this Φ field** and move towards :ref:`filterPhi`
        - If yes, and you wish directly to proceed into measuring the **strains**, **keep this Φ field** and move towards :ref:`regularStrainScript`

673
.. ATTENTION:: Remember to say that strains can be directly computed from decomposing the Φ of each correlation window.
674
675
676
677
678
679
680
681

|
|
|
|
|
.. _passPhi:

682
683
Passing deformation script (spam-passPhiField)
-------------------------------------------------
684

685
This script facilitates the manipulation of different Φ fields, applying a single deformation function to a series of points or passing a deformation field to a new basis.
686
687
688

.. tip:: When to use this script?

689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
                When you have a **single** deformation function (measured at one point) or **field** of deformation functions (measured at a set of points) and you want to **pass** them to a set of **new points**.

                Such a case could look like this:

                    .. figure:: images/tutorial/scripts/passInput.png
                        :align: center

                    Passing of a guiding points displacement field coming from :ref:`pixelSearchPropagate` (with ``-gp`` option) to a structured grid


.. IMPORTANT::
        This script defines **output positions** based on **two** different **modes**:

            1. Regularly-spaced points laid on a **structured grid**. See :ref:`grid` for tips regarding the grid geometry. This mode is activated with ``-ns`` input option
            2. **Discrete** points. For example centres of mass of particles, if your objective is to perform particle tracking. See :ref:`labelToolkitTutorial` for more information. This mode is activated with ``-lab1`` input option

        If the input file is:

            - A **single** deformation function: Φ is **applied** to each output position
            - A **field** of deformation functions: An **inverse distance interpolation** of the closest neighbours (see ``-nr``, ``-nn``) Φ is performed

        In both cases, you can choose which part of Φ (see :ref:`imageCorrelationTheory`) will be passed setting the input option ``-m``.
711
712

``Input:``
Olga Stamati's avatar
Olga Stamati committed
713
.............
714
715
716
717
718
719
720
721
722
723

    **Required**

        -  ``-pf``: *".tsv"* file containing the input deformation field

            .. HINT:: This can be either:

                        1. A **single** deformation function (one-line file). This is typically the result of :ref:`ereg` or :ref:`register`
                        2. A **field** of deformation functions (multiple-line file). For example, the result of :ref:`pixelSearch`,  :ref:`pixelSearchPropagate`, :ref:`ldic`, :ref:`ddic`

724
                      If this file refers to downscaled images, use ``-pfb`` to set the correct binning ratio
725
726
727
728
    **Optional**

        - ``-np``: Number of **parallel processes** to use. If not passed, all the CPUs available in the system will be used
        - ``-m``:  Choose which part of Φ will be passed (see :ref:`imageCorrelationTheory`) . If "all" all deformation components are passed. If "rigid" the rigid body motion is passed. If "disp" only displacements are passed 
729
730
731
        - ``-nr``: Only if input ``-pf`` is a **field**. *Radius** (in pixels) inside which to extract **neighbours** for the interpolation
        - ``-nn``: Only if input ``-pf`` is a **field**. **Number** of **neighbours** to extract for the interpolation. Disactivates ``-nr``
        - ``-rst``: Only if input ``-pf`` is a **field**. Return status **threshold below** which input points are **not** taken as **neighbours**
732
733
        - ``-od``: Defines the **output** directory
        - ``-pre``: Defines the prefix for the **output** files
734
        - ``-vtk``: Ask for a *".vtk"* **output** for **visualisation** of the displacement field
735
736
737

    For **structured grid** mode:

738
739
740
741
        - ``-ns``: **Spacing** between the grid **output points** (in pixels). Can be set different in each direction (Z, Y, X) with ``-ns3``
        - ``-im1``: The image on which to define the grid. If passed, ``-im1shape`` is not needed
        - ``-im1shape``: The shape of the image [Z Y X] (in pixels) on which to define the grid. If passed, ``-im1`` is not needed
        - ``-tif``: Ask for a *".tif"* output for **visualisation** of the displacement field
742
743
744

    For a **discrete** mode:

745
        - ``-lab1`` The labelled image (see :ref:`labelToolkitTutorial`) from which the **output positions** are calculated (centres of mass of labels)
746
747

``Output:``
Olga Stamati's avatar
Olga Stamati committed
748
749
750
.............

    - A *".tsv"* file containing:
751

Olga Stamati's avatar
Olga Stamati committed
752
        - New **positions** (Z, Y, X), as defined by the input grid (``-ns``) or the labelled image (``-lab1``)
753
754
        - The components of the applied or interpolated deformation function (*Fzz*, *Fzy*, *Fzx*, *Zdisp*, *etc*). Option ``-m`` defined which part of Φ was passed
        - For compatibility reasons, information related to the iterative algorithm in ``spam.DIC.register()`` (*iterations*, *error*, *deltaPhiNorm*, *returnStatus*). These are **totally irrelevant** to the result of this script
755

756
    - If asked with ``-tif`` or ``-vtk``: Files containing the displacement field
757

758
For **output** points defined by a **labelled** image, the script can be run like this:
759

760
761
762
763
764
765
766
767
768
769
770
771
.. code-block:: console

    $ source path/to/spam/venv/bin/activate
    (spam) $ spam-passPhiField                       # the script
                -pf /path/to/my/data/Phi.tsv\        # the file containing the correlation result (let's assume it's a single-line registration)
                -m "all"                             # pass all components of Φ 
                -lab1 /path/to/my/data/lab1.tif\     # path to labelled image to define the output points
                -vtk                                 # ask for a VTK output

...and it simply applies the result of a registration to the centres of mass of a labelled image.

For **output** points defined in a **structured grid**, the script can be run like this:
772
773
774
775
776

.. code-block:: console

    $ source path/to/spam/venv/bin/activate
    (spam) $ spam-passPhiField                       # the script
777
778
779
780
781
782
                -pf /path/to/my/data/PhiField.tsv\   # the file containing the correlation result (let's assume it's a field)
                -m "disp"                            # pass only disp
                -im1 /path/to/my/data/im1.tif\       # path to image to define the output grid
                -ns X \                              # node spacing of the output grid
                -nr X \                              # radius to extract neighbours for the interpolation (since we passed a field)
                -tif -vtk                            # ask for TIFF file and VTK output
783
784


785
786
787
788
.. tip:: The result of this script could look like this:

            .. figure:: images/tutorial/scripts/passResult.png
                :align: center
Olga Stamati's avatar
Olga Stamati committed
789
790
791
792
793
794

|
|
|
|
|
795

796
.. _filterPhi:
Olga Stamati's avatar
Olga Stamati committed
797
798
799
800

Filtering script (spam-filterPhiField)
---------------------------------------------

801
This script **corrects** or **filters** a deformation **field**.
Olga Stamati's avatar
Olga Stamati committed
802
803
804
805


.. tip:: When to use this script?

806
807
            - When you are **generally satisfied** with your deformation field, but there are some **crazy points** that you would like to get rid of.
              Such a field could look like this:
808

809
810
                .. figure:: images/tutorial/scripts/filterInput.png
                            :align: center
811

812
813
814
815
816
817
818
819
820
                            Displacement and return status fields coming from :ref:`ldic`

            - It is **recommended** to use this script **before** calculating **strains**.

.. ATTENTION:: Only **one operation** is allowed at a time. This means that you can either ask for a **correction** **OR** a **filtering** of your input field. 
               You can for example:

                1. run a first computation for correcting badly correlated points, check this result, and
                2. if satisfied with the correction, give the resulted field as a new input to this script asking now for an overall filtering of the corrected field.
Olga Stamati's avatar
Olga Stamati committed
821

822
.. IMPORTANT:: 
823

824
            If the **correction** of the field is asked, this script first selects bad points based on different metrics and then corrects them.
825

826
                The **selection** of bad points can be based on **one** of the following:
827

828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
                    - the **correlation coefficient**, activated with ``-scct``
                       A point's correlation coefficient is the result of :ref:`pixelSearch` or :ref:`pixelSearchPropagate`.
                       Typically a point with a coefficient **greater** than **0.99** is considered as **successfully** correlated

                    - the **return status**, activated with ``-srs``
                       A point's return status is the result of :ref:`ldic` or :ref:`ddic`. A return status equals to **2** means that the local algorithm inside ``spam.DIC.register()`` has **converged**

                    - the **local coherency**, activated with ``-slqc``
                       As per per Masullo and Theunissen 2016, a point's local coherency is the average residual between the point's displacement and a second-order parabolic surface,
                       fitted to the point's closest N neighbours and evaluated at the point's position.
                       Typically, a point with a coherency value **smaller** than **0.1** is classified as **coherent**.
                       The estimation of the local coherency is implemented in ``spam.DIC.estimateLocalQuadraticCoherency()`` and is based on https://gricad-gitlab.univ-grenoble-alpes.fr/DVC/pt4d.

                The **correction** of bad points can be done with **one** of the following operations:

                    - inverse distance weighting of closest good neighbours, activated with ``-cint``
                    - gaussian distance weighting of closest good neighbours, activated with 
                    - local quadratic fit of closest good neighbours, activated with ``-clqf``
846

Olga Stamati's avatar
Olga Stamati committed
847
848
                Closest neighbours are selected based on distance ``-nr`` or number ``-nn``.

849
            This script **filters** the input field based on:
850

851
                - an overall median filter, activated with ``-fm`` with a given radius ``-fmr``
Olga Stamati's avatar
Olga Stamati committed
852

853
            For both correction and filtering, this script by default **ignores background points** based their return status (*i.e, returnStatus=-5*). If you want to change this behaviour see ``-nomask`` input option.
Olga Stamati's avatar
Olga Stamati committed
854

Olga Stamati's avatar
Olga Stamati committed
855
856

``Input:``
Olga Stamati's avatar
Olga Stamati committed
857
.............
Olga Stamati's avatar
Olga Stamati committed
858
859
860

    **Required**

861
        - ``-pf``: *".tsv"* file containing the deformation **field**. This file can come either from :ref:`pixelSearch`, :ref:`pixelSearchPropagate`, :ref:`ldic`, :ref:`passPhi` or :ref:`filterPhi`. If this file refers to downscaled images, use ``-pfb`` to set the correct binning ratio
Olga Stamati's avatar
Olga Stamati committed
862
863
864

    **Optional**

865
866
867
868
869
870
871
872
873
874
875
876
877
878
        - ``-np``: Number of **parallel processes** to use. If not passed, all the CPUs available in the system will be used
        - ``-nomask``: If activated, correlation points in **background** (*i.e,* with *returnStatus*=-5) will **not** be **ignored** from the correction/filtering operations. It is recommended **not** to activate this option
        - ``-nr``: **Radius** (in pixels) inside which to extract **neighbours** for field interpolation. Disactivates ``-nn``
        - ``-nn``: **Number** of **neighbours** to extract for field interpolation. Disactivates ``-nr``
        - ``-srs``: Selects **bad** points based on their correlation **return status**. See ``-srst`` for setting the threshold. Disactivates ``-scc``, ``-slqc``, and ``-fm``
        - ``-srst``: Return status **threshold below** which points are considered **bad**. Default behaviour selects points with RS<=1
        - ``-scc``: Selects **bad** points based on their **correlation coefficient**. This is for fields coming from :ref:`pixelSearch`, :ref:`pixelSearchPropagate`. See ``-scct`` for setting the threshold. Disactivates ``-slqc``
        - ``-scct``: Correlation coefficient **threshold below** which points are considered **bad**. Default behaviour selects points with CC<=0.99
        - ``-slqc``: Selects **bad** points based on their local quadratic coherency value. Default behaviour selects points with coherency **bigger** or equal to 0.1. This threshold is **not** modifiable
        - ``-cint``: **Corrects** selected **bad** points by an **interpolation** of the extracted neighbours' Φ with weights equal to the **inverse** of the **distance**. See ``-m`` to select the interpolated components
        - ``-m``:  Choose which **part of Φ** (see :ref:`imageCorrelationTheory`) will be corrected/filtered. If **"all"** all deformation components are considered (default behaviour). If **"rigid"** the rigid body motion is considered. If **"disp"** only displacements are considered
        - ``-clqf``: **Corrects** selected **bad** points based on local **quadratic fit** of the extracted neighbours. The filt only applies to the **displacement** vector
        - ``-fm``: Activates an overall **median filter** on the deformation field. ``-m`` can be set either as **"all"** or as **"disp"**. The filtering operation is allowed only if **none** of ``-scc``, ``-slqc`` or ``-slqc`` is passed
        - ``-fmr``: Sets the radius (in pixels) of the median filter. Default behaviour is a radius of 1 pixel
Olga Stamati's avatar
Olga Stamati committed
879
880
881
        - ``-od``: Defines the **output** directory
        - ``-pre``: Defines the prefix for the **output** files
        - ``-vtk``: Ask for a vtk **output** for **visualisation** of the displacement field
882
        - ``-tif``: Ask for a tif **output** for **visualisation** of the displacement field
Olga Stamati's avatar
Olga Stamati committed
883
884
885


``Output:``
Olga Stamati's avatar
Olga Stamati committed
886
.............
Olga Stamati's avatar
Olga Stamati committed
887

Olga Stamati's avatar
Olga Stamati committed
888
    - A *".tsv"* file containing:
Olga Stamati's avatar
Olga Stamati committed
889

Olga Stamati's avatar
Olga Stamati committed
890
        - The points position (Z, Y, X) which are the **same** as the **input** field
891
892
        - The components of the points deformation functions (*Fzz*, *Fzy*, *Fzx*, *Zdisp*, *etc*). See :ref:`imageCorrelationTheory` for explanation of these components. Deformation components of **good** and **background** points are **not modified**
        - Information related to the iterative algorithm status (*returnStatus*, *iterations*, *etc*). Values of **good** and **background** points are **not modified**. *returnStatus* of corrected/filtered **bad** points is set equal to 1
Olga Stamati's avatar
Olga Stamati committed
893

894
    - If asked with *-tif* or *-vtk*: Files containing the corrected/filtered displacement fields
895
896


897
For a **correction** operation the script can be run like this:
Olga Stamati's avatar
Olga Stamati committed
898
899
900
901

.. code-block:: console

    $ source path/to/spam/venv/bin/activate
902
903
904
905
906
907
    (spam) $ spam-filterPhiField                     # the script
                -pf /path/to/my/data/PhiField.tsv\   # the file containing the deformation field
                -srs -srst 1\                        # select bad correlation points based on return status equal or less than 1
                -nn 27\                              # extract closest 27 good neighbours
                -cint -m "disp"\                     # correct displacements of bad points based on an inverse weighted distance interpolation of extracted good neighbours displacements
                -tif -vtk                            # ask for TIFF file and VTK outputs
Olga Stamati's avatar
Olga Stamati committed
908

909
910
911
912
913
                The script can be run like this:

For a **filtering** operation the script can be run like this:

.. code-block:: console
Olga Stamati's avatar
Olga Stamati committed
914

915
916
917
918
919
920
921
922
923
924
    $ source path/to/spam/venv/bin/activate
    (spam) $ spam-filterPhiField                     # the script
                -pf /path/to/my/data/PhiField.tsv\   # the file containing the deformation field
                -fm -fmr 2 -m "disp"\                # ask for a median filter of the input displacements with a radius of 2 px
                -tif -vtk                            # ask for TIFF file and VTK outputs

.. tip:: The result of this script could look like this:

            .. figure:: images/tutorial/scripts/filterOutput.png
                        :align: center
Olga Stamati's avatar
Olga Stamati committed
925

926
        Are you satisfied with the corrected/filtered displacement field?
927

928
929
930
            - If no, retry by increasing ``-nr`` or ``-nn`` or ``-fmr``?
            - If yes, and you wish to run a **last correlation** step, **keep this Φ field** and move towards :ref:`ldic`
            - If yes, and you wish to proceed into measuring the **strains**, **keep this Φ field** and move towards :ref:`regularStrainScript`
Olga Stamati's avatar
Olga Stamati committed
931
932
933
934
935
936
937
|
|
|
|
|


938
.. _regularStrainScript:
939

940
Regular strain script (spam-regularStrain)
941
942
-----------------------------------------------

Olga Stamati's avatar
Olga Stamati committed
943
944
945
946
947
948
949
950
951
952
This scripts calculates the strain field between im1 and im2, based on a displacement field measured at a **regular grid**.

.. IMPORTANT:: This script calculates strains based on two modes:

                    1. **Geers** (see :ref:`strainTutorial`)
                    2. **Q8** (see :ref:`strainTutorial`), activated with ``-cub``

                For further details regarding the strain calculation in spam and the output of this script please see the detailed tutorial here :ref:`strainTutorial`.

``Input:``
Olga Stamati's avatar
Olga Stamati committed
953
.............
Olga Stamati's avatar
Olga Stamati committed
954
955
956
957

    **Required**

        - *-pf*: *".tsv"* file containing the deformation field. This file can come either from `spam-pixelSearch` or `spam-ldic`
958

Olga Stamati's avatar
Olga Stamati committed
959
960
961
962
963
964
965
966
967
968
969
    **Optional**
        - ``-r``: Only for **Geers** mode. The radius (in units of the grid spacing) inside which to select neighbours
        - ``-cub``: Activates **Q8** mode
        - ``-mask``: Mask the background? If the correlation of this point was skipped (**returnStatus**=-5), strain calculation will be skipped too 
        - ``-comp``: Defines which strain components to return (see )
        - ``-od``:  Defines the **output** directory
        - ``-pre``  Defines the prefix for the **output** files
        - ``-vtk``: Ask for a vtk **output** for **visualisation** of the strain field
        - ``-tif``: Ask for a tif output for **visualisation** of the strain field

``Output:``
Olga Stamati's avatar
Olga Stamati committed
970
.............
Olga Stamati's avatar
Olga Stamati committed
971
972
973
974
975
976
977
978

    - A *".tsv"* file containing:

        - The points **position** (Z, Y, X), where the strain was measured. It is the centre of the correlation window for **Geers**
        - The strain components
    - If asked with *-tif* or *-vtk*: Files containing the strain components as structured fields

The script can be run like this:
979
980
981
982

.. code-block:: console

    $ source path/to/spam/venv/bin/activate
Olga Stamati's avatar
Olga Stamati committed
983
984
985
986
987
988
989
990
991
992
993
994
    (spam) $ spam-regularStrain                    # the script
                /path/to/my/data/PhiField.tsv\     # the file containing the correlation result
                -mask \                            # do not calculate any strain in the background
                -comp vol dev\                     # ask for first and second invariants of strain
                -tif -vtk                          # ask for TIFF file and VTK output of the strain field

|
|
|
|
|

995
996
.. _grid:

Olga Stamati's avatar
Olga Stamati committed
997
998
999
1000
1001
1002
A note on the correlation grid
..............................

A small note on the choice of the input options *-ns* and *-hws* would be useful.
Generally, the size of the correlation window (2× *hws* +1) and the spacing of the measurement points (*ns*) can be chosen independently, by selecting a spacing smaller than the window size (overlapping windows), equal to the window size (contiguous windows) or bigger than the window size (separate windows).
Note that contiguous correlation windows will ensure statistical independence of the corresponding error.
1003

Olga Stamati's avatar
Olga Stamati committed
1004
1005
1006
As for the choice of the number of the measurement points (defined by *-ns*), it should be kept in mind that there is a clear trade-off between the spatial resolution of the measurement (node spacing) and the level of the measurement uncertainty.
A complex heterogeneous kinematic field requires many measurement points in order to be accurately described at the expense, though, of higher uncertainties which can dominate the signal for very small windows.
A compromise is therefore needed, which is strongly related to the particular experiment conducted, the material studied and the scale of the mechanisms of interest.
1007

Olga Stamati's avatar
Olga Stamati committed
1008
1009
1010
1011
1012
1013
1014
1015
1016
1017
1018
1019
1020
1021
1022
1023
1024
1025
1026
1027
1028
An illustration of a regular grid (in 2D) with contiguous correlation windows is given below.
The image is the reference configuration of the V4 data set -- an x-ray tomography of a sandstone performed before and after straining done by E. Charalampidou and used as an example in `TomoWarp 2`_ [TUD2017a]_.
You can download `VEC4.zip` from `spam tutorial archive on zenodo`_.

.. _TomoWarp 2: https://doi.org/10.1016/j.softx.2017.10.002
.. _spam tutorial archive on zenodo: https://doi.org/10.5281/zenodo.3888347

.. figure:: images/tutorial/scripts/regularGridSmall.png
    :align: center

    Example of a regular grid with spacing of the measurement points (*ns*) double of the half-window size (*hws*), so as not to have overlapping correlation windows.
    A linear deformation function Φ is calculated independently at the centre of each correlation window

Note that in this case a *-glt 20000* is recommended to avoid correlating the windows that fall on the sides of the specimen (surrounding air).
To calculate the deformation field between `V4_01.tif` and `V4_02.tif` measured on the regular grid shown in the image above, only very simple lines are needed:

|
|
|
|
|
1029

1030
.. _ddic:
1031

1032
Discrete local DIC script (spam-ddic)
1033
1034
1035
1036
1037
----------------------------------------

A script for performing kinematical measurements on discretely defined objects is provided with ``spam-ddic``.
The objects to correlate are defined in a labelled volume as individually-numbered objects (see :ref:`labelToolkitTutorial` and :ref:`discreteImageCorrelationTutorial`).
By default, the extent of each label is used to mask the greylevels that are correlated (see `spam.label.boundingBoxes`).
1038
The result of running this script is an independent measurement of a linear deformation function :math:`\Phi` (see :ref:`imageCorrelationTheory`) at the centre of mass of each object.
1039
1040
1041
1042

Inputs
......

1043
At the very least this script must be run with **three TIFF images in this order**:
1044
1045
1046
1047
1048

  1. A greyscale image of state 1
  2. A labelled image of state 1
  3. A greyscale image of state 2

1049
1050
1051
1052
1053
1054
Here is an example of the vertical slices through the "Martian Simulant" M2EA05 dataset from [Kawamoto2016]_ that will be used for discrete correlation.

.. figure:: images/tutorial/scripts/discrete.png
    :align: center

    Example of input images for a discrete correlation.
Olga Stamati's avatar
Olga Stamati committed
1055
    A linear deformation function Φ is calculated independently at the centre of mass of each labelled object
1056
1057
1058
1059

Other key input options are:

    - *-ld* sets how many iterations of dilation will be applied to each object to extract the greylevels to correlate
Edward Andò's avatar
Edward Andò committed
1060
    - *-regbb* and *-regbe* to perform an *initial multiscale registration* of the two provided images -- **this is highly recommended**. In many cases (without too complicated and large displacement fields), this step can avoid the need to perform a computationally expensive pixel search. *-regbb* defines the initial binning level *-regbe* defines the final binning level.
1061
    - *-vt* the volume (in voxels) of the smallest object to correlate.
Edward Andò's avatar
Edward Andò committed
1062
    - *-lci* and *-lcp* regulate convergence options for each grid point `spam.DIC.correlate.register` run in an object: the number of maximum iterations and the smallest change in the norm of Φ (:math:`\delta\Phi_{min}`) respectively.
1063
1064
    - *-od* and *-pre* set the output directory and the prefix for the output files respectively.

Olga Stamati's avatar
Olga Stamati committed
1065
As per the ``spam-ldic`` script, a large number of other options exist and can be seen with **- -help**.
1066
1067

For 3D images, only a single 3D TIFF per state file is supported for the moment.
1068
This script can **run in parallel** using the MPI multiprocessing library as above.
1069
1070
1071
1072
1073
1074
1075
1076
1077

Outputs
.......

By default the outputs of a discrete correlation are:

- a TSV file with the transformation and correlation status for each object
- a VTK file with the displacement vectors of each object

Edward Andò's avatar
Edward Andò committed
1078
1079
1080
1081
1082
1083
1084
1085
1086
1087
1088
1089
1090
1091
1092
1093
1094
1095
1096
1097
1098
The code will return, for each label, the following information:

    - The label number
    - The label's Z,Y,X position (centre of mass)
    - The measured label's Z,Y,X displacement
    - The measured label's components of F matrix
    - The correlation error for this label: final difference in greylevels between im1 and im2(:math:`\Phi`.x) subvolumes with a normalisation applied.
    - The number of  iterations until convergence (equal to *-lci* if convergence was not reached)
    - The return status (**this is important!**):

        -  2 = **Convergence**: reached desired :math:`\delta\Phi_{min}` (*-lcp* input)
        -  1 = **Not Convergence**: stopped by hitting the maximum number of iterations (*-lci* input) without diverging
        - -1 = **Divergence**: error is more than 80% of previous iteration error (see `spam.DIC.correlate.register`)
        - -2 = Iterative procedure stopped: singular matrix M cannot be inverted (see `spam.DIC.correlate.register`)
        - -3 = Divergence: diverged on displacement or volumetric change condition (see `spam.DIC.correlate.register`)
        - -4 = Margin alignment for im1 and im2 subvolumes failed (the iterative procedure never started)
        - -5 = Failed margin alignment (too close to edge?) or failed volume condition
        - -6 = Not used right now
        - -7 = Label missing or NaNs/infinite in Phi


Olga Stamati's avatar
Olga Stamati committed
1099
As per ``spam-ldic`` script, to perform a discrete correlation between these two M2EA05 states, again only very simple lines are needed:
1100
1101
1102
1103
1104
1105
1106
1107
1108
1109
1110

.. code-block:: console

    $ source path/to/spam/venv/bin/activate
    (spam) $ spam-ddic                                            # the script
                data/M2EA05/M2EA05-quart-01.tif \                 # reference greyscale image
                data/M2EA05/M2EA05-quart-01-bin-watershed.tif \   # reference labelled image
                data/M2EA05/M2EA05-quart-02.tif \                 # deformed greyscale image

To see the output of this script please go to the detailed tutorial for discrete image correlation here :ref:`discreteImageCorrelationTutorial`.