CHANGELOG.md 6.27 KB
Newer Older
Loic Huder's avatar
Loic Huder committed
1
2
# Changelog of GeDACCMU

Loic Huder's avatar
Loic Huder committed
3
4
5
6
7
8
9
10
11
12
0.7
---
New features:
* PCA_U can now performed with energy normalisation (PCA is performed on core flow energy coefficents rather than directly on core flow coefficients)
* CoreState can now be initialised in several fashions:
    * *constant*: equal to the average prior (for MF, U and ER)
    * *normal*: normal draw around the average prior (variance = deviation of the prior) (for MF, U and ER)
    * *from_file*: equal to the corestate of a given file (*init_file*) at a given date (*init_date*) (for all measures)
* Added higher resolution COVOBS files (COVOBS_hd). The COVOBS loading method now computes the error matrix R directly from the files rather than using a `var_*` file.
* Configuration file can now be generated from a hdf5 file of a previous calculation
13

Loic Huder's avatar
Loic Huder committed
14
15
16
17
18
Other improvements:
* Matrix inversions required by misfits computation were refactored
* Misfits of the PCA_U are now saved
* Added tests
* Updated README.md
19

Loic Huder's avatar
Loic Huder committed
20
21
22
23
24
0.6
---
**Important bug fixes:**
- Fixed the time sampling for dense AR matrices that led to no variation in forecasts (#54)
- Observation error matrices are now properly set for COVOBS (#57)
25
26

In addition:
Loic Huder's avatar
Loic Huder committed
27
28
29
30
31
- Seed arg is now working (#56): it allows to have reproducible stochastic processes **when using the same number of MPI processes**. Noising of GO_VO observations does not use this global seed but prints the seeds used in debug logs.
- Misfits between the analysed states and the observations are now saved for analysis times (CoreState-like behaviour) (#58)
- Updated GO_VO data with 2018 data
- Updated guide for advanced users
- Added some tests (coverage: 79%)
32

Loic Huder's avatar
Loic Huder committed
33
0.5
34
---
Loic Huder's avatar
Loic Huder committed
35
36
:warning: **Important bug fix**:
* Fixed the covariance matrices of analysed states Z that was not scaled (led to inconsistent results when using GO/VO)
37

Loic Huder's avatar
Loic Huder committed
38
39
40
41
42
43
Improvements towards a usable package:
* Explained the arguments of `run_algo.py` in README.md
* Added an in-depth guide in `doc` to explain how to change the input data (priors/observations) and how the low-level features (forecast/analysis/_CoreState_) work (#50)
* Refactored the reading of observations/priors in separate functions that are called dynamically according to the types asked in config (#49)
* Redesigned the observations to have a single object _Observation_ handling data, operator and errors.
* Renamed _Forecasts_/_Analysis_ objects in _Forecaster_/_Analyser_
44

Loic Huder's avatar
Loic Huder committed
45
46
47
Testing:
* Implemented the use of hypothesis.strategies and composite strategies as input for several tests
* Added basic functional tests of `run_algo.py` and other tests (coverage: 78% !)
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70

0.4
---
:warning: **This version introduced a bug in analysis_step (P_zz not scaled) ! Use version 0.5 instead (same features but with bug fixed).**

Scientific improvements of the code:
* Implementation of a AR-1 process using dense drift matrices (#42 and #48)
* Implementation of several scaling methods for dense matrices: v1 is the same for diagonal and dense AR, v2 is the recommended for dense AR.


This was possible thanks to the following improvements in the code:
* Added high-resolution midpath data as hdf5 file
* Redesigned CoreState to be able to dynamically set the measures in it (#46)
* Implemented the possibility to do the computation on a principal componenent analysis of U (PCA_U)
* The covariance of analysed states Z is computed at each analyse and now enters in the expression of the covariance matrice P_zz used for the augmented state Kalman filter.

Other minor improvements:
* Changed magnetic field key from 'B' to 'MF'
* Spectral observations are now truncated by the number of asked coefficients for computation (Nb, Nsv)
* Asking for max degrees higher than the degree of priors will raise a ValueError (#41)
* Fixed docstrings of functions decorated by with_core_state_of_dimensions
* Added instructions to run the code on IST-OAR

Loic Huder's avatar
Loic Huder committed
71
0.3
72
---
Loic Huder's avatar
Loic Huder committed
73
74
75
76
77
78
Oops ! Computation of diffusion using cross-covariances only works for samples of the same statistical ensemble... As a consequence :
* Deprecation of the AugKF algorithm with diffusion computation (now 'legacy.augkf' in deprecated module)
* Deprecation of the CorDE algorithm (now in deprecated module as well)
* New implementation of the AugKF algorithm (formely AugKF_dpe) using master equation: $`\dot{b} = A(b)u + e`$ (DIFF is taken as a contribution of ER in:
  - AR-1 process on ER
  - Analysis on augmented state $`z= [u^Te^T] `$
79

Loic Huder's avatar
Loic Huder committed
80
81
82
Changes in observations:
* VO and GO data were added to the repository and can be used as observations for analyses.
* Dates in the algorithm are now handled with [NumPy datetime64](https://docs.scipy.org/doc/numpy/reference/arrays.datetime.html#arrays-dtypes-dateunits). This allows notably to have $`\Delta t`$ in months in the config files rather than floating dates that can have infinite decimals.
83
84


Loic Huder's avatar
Loic Huder committed
85
86
87
88
89
90
91
92
93
94
95
96
In addition:
* $`\Delta t`$ for Euler schemes and $`\Delta t_{forecasts}`$  are no longer different.
* Covariance matrices can be supplied as files in the prior folders. They will be computed from priors if not found.
* Deprecated files were put in a dedicated module that triggers a warning if imported.
* Various code improvements: clean-ups, docs written and tests added.

v0.2
----
Numerous improvements aiming to optimise the code, the most important being:
* The possibility to run forecasts in parallel using `mpi4py`
* The possibility to save results during computation in `hdf5` or `ASCII` format.
* Number of forecasts and analysis are now determined **ONLY** from the config file
97
98

In addition:
Loic Huder's avatar
Loic Huder committed
99
100
101
102
103
* Possibility to run analysis in parallel (not very interesting as the algorithm used in this case is way slower and scientifically equivalent...)
* Some clean-up: 
    - All raw data are now in the data folder
    - *forecast* folder containing the Fortran sources was renamed to *fortran*
* Launching an algorithm can now be done in one line in `run_algo.py` thanks to the encapsulation of the `run` module.
104
105


Loic Huder's avatar
Loic Huder committed
106
107
108
109
110
111
112
113
114
115
116
117
v0.1
----

Important milestones comprise:
- Par_forecast returns scientifically correct results.
- Fortran and Python implementation are interchangeable for the setup, the diffusion computation and the forecasts.

In addition:
- Implemented visualisation tools for trajectories and Lowes spectra.
- Possibility to save files in a format readable by [pygeodyn](https://gricad-gitlab.univ-grenoble-alpes.fr/Geodynamo/pygeodyn)
- Unitary and functional tests on around 35% of the code.
- Generated doc using Sphinx