jmmc-jmdcui-docker issueshttps://gricad-gitlab.univ-grenoble-alpes.fr/OSUG/JMMC/jmmc-jmdcui-docker/-/issues2023-03-01T14:48:57+01:00https://gricad-gitlab.univ-grenoble-alpes.fr/OSUG/JMMC/jmmc-jmdcui-docker/-/issues/24Some exceptions are not properly handled during the user data feeding.2023-03-01T14:48:57+01:00Guillaume MellaSome exceptions are not properly handled during the user data feeding.more info to come before working on solving it...more info to come before working on solving it...https://gricad-gitlab.univ-grenoble-alpes.fr/OSUG/JMMC/jmmc-jmdcui-docker/-/issues/23Add a validation step for the user before moderation request in async mode2019-11-21T10:17:00+01:00Guillaume MellaAdd a validation step for the user before moderation request in async modeMoving to async mode, the submission workflow swallowed the user verification test.
Please add a user stage to display status of validation (the link can be sent after whole check).
If errors occured then display report to fix and submi...Moving to async mode, the submission workflow swallowed the user verification test.
Please add a user stage to display status of validation (the link can be sent after whole check).
If errors occured then display report to fix and submit again the data else send a moderaion request and display report.
Then moderators should only receive well formed data and they will be happy.Raphael JacquotRaphael Jacquot2019-11-28https://gricad-gitlab.univ-grenoble-alpes.fr/OSUG/JMMC/jmmc-jmdcui-docker/-/issues/22Send error report of first JMDC to Gilles2019-07-19T17:29:44+02:00Guillaume MellaSend error report of first JMDC to Gilleshere comes the well formated csv file :
http://jmmc.fr/~mellag/files/JMDC-ascii.2019-07-18.csvhere comes the well formated csv file :
http://jmmc.fr/~mellag/files/JMDC-ascii.2019-07-18.csvRaphael JacquotRaphael Jacquot2019-07-24https://gricad-gitlab.univ-grenoble-alpes.fr/OSUG/JMMC/jmmc-jmdcui-docker/-/issues/21a number of bands overlap each other2019-07-19T16:23:42+02:00Raphael Jacquota number of bands overlap each otheryet there is only one possible band_code
which should be selected ? n or n+1 ?yet there is only one possible band_code
which should be selected ? n or n+1 ?https://gricad-gitlab.univ-grenoble-alpes.fr/OSUG/JMMC/jmmc-jmdcui-docker/-/issues/20Add bootstrap toolkit - to add a spinner display during submission process2019-11-21T10:18:43+01:00Guillaume MellaAdd bootstrap toolkit - to add a spinner display during submission processhttps://gricad-gitlab.univ-grenoble-alpes.fr/OSUG/JMMC/jmmc-jmdcui-docker/-/issues/19Add full width for moderation message field2019-07-18T16:29:28+02:00Guillaume MellaAdd full width for moderation message fieldhttps://gricad-gitlab.univ-grenoble-alpes.fr/OSUG/JMMC/jmmc-jmdcui-docker/-/issues/18Ask to check emails after submission action instead of main page display2019-07-19T16:23:25+02:00Guillaume MellaAsk to check emails after submission action instead of main page displayhttps://gricad-gitlab.univ-grenoble-alpes.fr/OSUG/JMMC/jmmc-jmdcui-docker/-/issues/17Put back moderation list page (hidden from menu)2019-07-18T16:47:16+02:00Guillaume MellaPut back moderation list page (hidden from menu)Add stats for each of them : #accepted / #rejected
This page will covered by authentication on the frontend (HAProxy)Add stats for each of them : #accepted / #rejected
This page will covered by authentication on the frontend (HAProxy)Raphael JacquotRaphael Jacquothttps://gricad-gitlab.univ-grenoble-alpes.fr/OSUG/JMMC/jmmc-jmdcui-docker/-/issues/16Support every test files2019-09-25T15:59:06+02:00Guillaume MellaSupport every test filesGuillaume MellaGuillaume Mellahttps://gricad-gitlab.univ-grenoble-alpes.fr/OSUG/JMMC/jmmc-jmdcui-docker/-/issues/15Add validation rule to detect ID1/ID2 inconsistency2019-07-18T12:03:07+02:00Guillaume MellaAdd validation rule to detect ID1/ID2 inconsistencyhttps://gricad-gitlab.univ-grenoble-alpes.fr/OSUG/JMMC/jmmc-jmdcui-docker/-/issues/14Fix submission using textarea2019-07-18T16:43:39+02:00Guillaume MellaFix submission using textareahttps://gricad-gitlab.univ-grenoble-alpes.fr/OSUG/JMMC/jmmc-jmdcui-docker/-/issues/13add link to ID1 and BIBCODE2019-07-18T15:35:41+02:00Raphael Jacquotadd link to ID1 and BIBCODEhttps://gricad-gitlab.univ-grenoble-alpes.fr/OSUG/JMMC/jmmc-jmdcui-docker/-/issues/12Implement ID1 computation from user ID2019-07-18T11:57:37+02:00Guillaume MellaImplement ID1 computation from user IDfirst version could use main ID returned by sesame.
May change in the future using HD//// to be checked with Gillesfirst version could use main ID returned by sesame.
May change in the future using HD//// to be checked with GillesRaphael JacquotRaphael Jacquot2019-07-16https://gricad-gitlab.univ-grenoble-alpes.fr/OSUG/JMMC/jmmc-jmdcui-docker/-/issues/11commit submit transaction after notification email2019-07-24T17:15:12+02:00Guillaume Mellacommit submit transaction after notification emailThis would improve the reliability and consistency of database and avoid sleeping dataThis would improve the reliability and consistency of database and avoid sleeping dataRaphael JacquotRaphael Jacquothttps://gricad-gitlab.univ-grenoble-alpes.fr/OSUG/JMMC/jmmc-jmdcui-docker/-/issues/10currenty not checking if multiple lines in the current submission are identical2019-09-25T16:00:16+02:00Raphael Jacquotcurrenty not checking if multiple lines in the current submission are identicalwe are only checking if a new line already exists in the database itselfwe are only checking if a new line already exists in the database itselfhttps://gricad-gitlab.univ-grenoble-alpes.fr/OSUG/JMMC/jmmc-jmdcui-docker/-/issues/9need to find better subject for emails ;)2019-07-18T08:46:36+02:00Raphael Jacquotneed to find better subject for emails ;)https://gricad-gitlab.univ-grenoble-alpes.fr/OSUG/JMMC/jmmc-jmdcui-docker/-/issues/8should be sending mail to administrator after submission2019-07-17T10:32:55+02:00Raphael Jacquotshould be sending mail to administrator after submissionhttps://gricad-gitlab.univ-grenoble-alpes.fr/OSUG/JMMC/jmmc-jmdcui-docker/-/issues/7JMDC.csv generated by alx script contains illegal input sequence characters2019-07-18T12:08:42+02:00Guillaume MellaJMDC.csv generated by alx script contains illegal input sequence charactersThis first throw an exception which break the injecst process
```
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xa0 in position 6:
invalid start byte
'utf-8' codec can't decode byte 0xa0 in position 6: invalid start byte
something...This first throw an exception which break the injecst process
```
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xa0 in position 6:
invalid start byte
'utf-8' codec can't decode byte 0xa0 in position 6: invalid start byte
something bad happened on line 773
```https://gricad-gitlab.univ-grenoble-alpes.fr/OSUG/JMMC/jmmc-jmdcui-docker/-/issues/6source data incoherence, bibcode and somme comment smashed all into one field2019-07-11T13:07:18+02:00Raphael Jacquotsource data incoherence, bibcode and somme comment smashed all into one fieldThe spec (issue #3) states there is bibcode and a separate comment
the csv files smash those 2 fields together in one large field, with a space separator in between, in a field named "reference"
for now I have code that splits that th...The spec (issue #3) states there is bibcode and a separate comment
the csv files smash those 2 fields together in one large field, with a space separator in between, in a field named "reference"
for now I have code that splits that thing into the bibcode and comment
is this the right thing to do ?https://gricad-gitlab.univ-grenoble-alpes.fr/OSUG/JMMC/jmmc-jmdcui-docker/-/issues/5Track follow up of july sprint2019-09-25T16:00:38+02:00Guillaume MellaTrack follow up of july sprint## Main user functions of the tool
### Provide doc page
- [ ] add injest data format ( header )
- [ ] provide units for each numerical values
### Provide release notes page
- [x] a single page will provide main change
- [x] the release...## Main user functions of the tool
### Provide doc page
- [ ] add injest data format ( header )
- [ ] provide units for each numerical values
### Provide release notes page
- [x] a single page will provide main change
- [x] the release note must display a version number (take image version if easy to do)
### Display catalogue content
- [x] a table must help to navigate in the data
- [x] columns must be sortable
### Record a single user record
At first no specific form will be developped for a single record. Off course the multi record input will be compatible with a single line ;)
- [x] User input will not provide UD1 UD2 BANDCODE
- [x] some input should be enum or more precise specific values (band)
mandatory field:
- [x] STARNAME (replaced by main_id for ID1 and kept original in ID2)
- [ ] UD or LD
- [ ] E_MEAS (excepted for old values injected as -1)
- [x] BAND (letter or numerical valie) - see https://svn.jmmc.fr/jmmc-sw/MCS/tags/ASPRO2_V0_9_9_6/jmal/src/main/java/fr/jmmc/jmal/ as example
- [x] REFERENCE (BIBCODE should be checked)
optional:
- [x] MU_LAMBDA ( helps when UD and LD )
- [x] METHOD (1 for defaut (interferometry))
Constraints:
- [x] NOTES : have fixed size (TBD)
- [x] decimal values accepts only N.N (except for original dataset injest)
- [ ] ( -1, NAN, +inf, -inf values could be removed / considered as empty )
### CSV file header format
- [x] Header must be provided
- [x] for initial injest `ID1,ID2,UD_MEAS,LD_MEAS,E_LD_MEAS,BAND,MU_LAMBDA,METHOD,BANDCODE,NOTES,REFERENCE`
- [x] for user submissions `ID,UD_MEAS,LD_MEAS,E_LD_MEAS,BAND,MU_LAMBDA,METHOD,NOTES,REFERENCE`
- [x] insensitive to column order
- [x] ignore or warn extra columns (TBD)
see doc.
### Keep housekeeping data
Next information should be stored:
- [x] user name (email)
- [x] submission date
### Record a multi measurements user record
- [x] First version should accept the following spreadsheet extract of previous catalogue as user input
```
ID1,ID2,UD_MEAS,LD_MEAS,E_LD_MEAS,BAND,MU_LAMBDA,METHOD,BANDCODE,NOTES,REFERENCE
HD18884,ALF CET,"9,4",-1,-1,,,1,,,"................... Carnegie Year Books for 1933, 1935, 1937"
HD29139,ALF TAU,24,-1,-1,,,1,,,"................... Carnegie Year Books for 1933, 1935, 1937"
HD124897,ALF BOO,19,-1,-1,,,1,,,"................... Carnegie Year Books for 1933, 1935, 1937"
CCDM J16294-2626 A,ALF SCO,28,-1,-1,,,1,,,"................... Carnegie Year Books for 1933, 1935, 1937"
HD186791,GAM AQL,"8,4",-1,-1,,,1,,,"................... Carnegie Year Books for 1933, 1935, 1937"
HD206778,EPS PEG,"8,4",-1,-1,,,1,,,"................... Carnegie Year Books for 1933, 1935, 1937"
HD29139,ALF TAU,20,-1,-1,,,1,,,"1931............... Pease, F.G.: 1931, Ergebn. Exacten Naturwiss. 10, 84"
HD39801,ALF ORI,34,-1,-1,,,1,,,"1931............... Pease, F.G.: 1931, Ergebn. Exacten Naturwiss. 10, 84"
HD124897,ALF BOO,20,-1,-1,,,1,,,"1931............... Pease, F.G.: 1931, Ergebn. Exacten Naturwiss. 10, 84"
CCDM J16294-2626 A,ALF SCO,40,-1,-1,,,1,,,"1931............... Pease, F.G.: 1931, Ergebn. Exacten Naturwiss. 10, 84"
HD156014,ALF HER A,30,-1,-1,,,1,,,"1931............... Pease, F.G.: 1931, Ergebn. Exacten Naturwiss. 10, 84"
HD217906,BET PEG,21,-1,-1,,,1,,,"1931............... Pease, F.G.: 1931, Ergebn. Exacten Naturwiss. 10, 84"
HD28307,THE 1 TAU,"2,3","2,39","0,1",,,2,,,"1946............... Wood, F.B.: 1946, Contrib.Princeton Univ.Obs. 21, 18"
HD102212,NU VIR,8,-1,-1,,,2,,,"1951............... Jager, F.W.: 1951, Die Sterne 27, 51"
HD110014,CHI VIR,5,-1,-1,,,2,,,"1951............... Jager, F.W.: 1951, Die Sterne 27, 51"
```
### Validation:
validate a batch or a single line.
purgatoir et enfer.
- [x] Error should be displayed properly ( because staging area will only contains valid entries.)
#### Validate input according following rules
* [x] target names are resolved by simbad (or already present in the catalogue)
Note GD a traduire ;
>>>
regarder si STILTS ou équivalent permettrait d'accepter plusieurs autres formats de manière transparente.
la table csv doit comporter les colonnes de JMDC, soit:
ID1 ID2 UD_MEAS LD_MEAS E_LD_MEAS BAND MU_LAMBDA METHOD BANDCODE NOTES REFERENCE
ID1 devra être renseigné. Il sera validé par le CDS, qui retourne une liste de synonymes. ID1 sera préférentiellement le synonyme de type HDxxxxx. En l'absence de HDxxxx, utiliser dans l'ordre HIPxxxx, GAIAxxxxxx. ID2 sera mis à la valeur de ID1 originale si ID2 est non renseigné. Si ID2 est renseigné, vérfier avec le CDS que ID2 est bien un synonyme de ID1.
UD_MEAS et LD_MEAS ne peuvent être tous deux vides. Ce sont des rééls.
E_LD_MEAS doit etre renseigné et être un réel.
BAND peut être vide
MU_LAMBDA peut être vide.
METHOD est 1 par défaut et on accepte 2 et 3, avec warning. Le Readme du JSDC explique ce que c'est
BANDCODE est présent et vaut 1..12 correspondant aux bandes UBVRIJKLMNQ
NOTES est un texte de < 200 caractères
REFERENCE est un BiBCode VALIDE ou rien, auquel cas ce sera "YYYY ..............."+ adresse mail de la personne qui soumet.
>>>
#### Clean / relive pending submission which would not have been validated by user email
### Follow status of submissions
- [x] User receives an email after submission
- [x] Application should approve the entry when user click confirmation link
- [x] The administrator receive an email
- [x] Pending submissions should be moderated before display.
### Approval/confirmation by moderator
- [x] Moderator can confirm all record with some exceptions
### Export content through VO TAP interface
This block will be on top of [existing library](https://github.com/gmantele/vollt) used for OiDB service at JMMC.
Standards supported formats will be fine for the administrator of the database and input for futur update at CDS. The datamodel may be setup on top of [ObsCore](http://ivoa.net/documents/ObsCore/)Guillaume MellaGuillaume Mella