Skip to content
GitLab
Menu
Projects
Groups
Snippets
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in
Toggle navigation
Menu
Open sidebar
OSUG
RESIF
datadump
Commits
9a536c1a
Commit
9a536c1a
authored
Aug 19, 2019
by
Jonathan Schaeffer
Browse files
bugfix
parent
15eddfdc
Changes
1
Hide whitespace changes
Inline
Side-by-side
src/resifdatadump
View file @
9a536c1a
...
...
@@ -70,7 +70,7 @@ irods_push(){
format_report
$KEY
$SIZEMB
$(
date
+%Y-%m-%dT%H:%M:%S
--date
=
@
$start
)
- -
'Error 011. Transfer cancelled'
>>
$LOCAL_REPORT
# Roll back
irods_rollback
$KEY
rm
$RESIFDD_WORKDIR
/
${
KEY
}
.tar
rm
$RESIFDD_WORKDIR
/
${
KEY
}
.tar
# Alert to zabbix
zabbix_err
"
${
KEY
}
:Error 011"
else
...
...
@@ -79,8 +79,8 @@ irods_push(){
echo
"[
$KEY
] staging.tar data sent, let's commit everything on irods server"
format_report
$KEY
$SIZEMB
$(
date
+%Y-%m-%dT%H:%M:%S
--date
=
@
$start
)
$duration
$throughput
'OK'
>>
$LOCAL_REPORT
irods_commit
$KEY
zabbix_ok
"
$KEY
|
${
SIZEMB
}
MB|
${
duration
}
s|
${
throughput
}
MB/s"
rm
$RESIFDD_WORKDIR
/
${
KEY
}
.
*
zabbix_ok
"
$KEY
|
${
SIZEMB
}
MB|
${
duration
}
s|
${
throughput
}
MB/s"
rm
$RESIFDD_WORKDIR
/
${
KEY
}
.
*
fi
# Send report to irods. Do some locking here
(
...
...
@@ -166,7 +166,7 @@ pack_and_send() {
STATION
=
${
YNS
[-1]
}
KEY
=
${
YEAR
}
_
${
NETWORK
}
_
${
STATION
}
if
[[
$YEAR
-lt
$RESIFDD_START_AT
]]
;
then
return
0
return
0
fi
echo
"[
$KEY
] Starting job
$2
"
# Test if in recovery mode, we should send or not
...
...
@@ -183,7 +183,7 @@ pack_and_send() {
if
[[
$?
-ne
0
]]
;
then
# Something went wrong creating archive. Exit
echo
"[
$KEY
] Error 007 creating tar"
rm
-f
$RESIFDD_WORKDIR
/
$KEY
.tar
rm
-f
$RESIFDD_WORKDIR
/
$KEY
.tar
# Send key to zabbix_err
zabbix_err
"
$KEY
:Error 007"
return
1
...
...
@@ -199,9 +199,9 @@ pack_and_send() {
# If the hashes differs, then move distant file and push this one
if
[[
"
$local_sha
"
=
"
$irods_sha
"
]]
;
then
echo
"[
$KEY
] The archive on irods is the same as our version. Skipping."
SIZE
=
$(
stat
-c
%s
$RESIFDD_WORKDIR
/
$KEY
.tar
)
SIZE
=
$(
stat
-c
%s
$RESIFDD_WORKDIR
/
$KEY
.tar
)
format_report
$KEY
$((
$SIZE
/
1024
/
1024
))
$(
date
+%Y-%m-%dT%H:%M:%S
)
"-"
"-"
"Skipped"
>>
$LOCAL_REPORT
rm
$RESIFDD_WORKDIR
/
$KEY
*
rm
$RESIFDD_WORKDIR
/
$KEY
*
return
0
fi
fi
...
...
@@ -260,8 +260,8 @@ fi
IRODS_REPORT
=
reports/
$(
date
+%Y%m%d-%H%M
)
.csv
imkdir
-p
reports
iput
-f
$LOCAL_REPORT
$IRODS_REPORT
rm
$RESIFDD_WORKDIR
/report.lock
rm
$RESIFDD_WORKDIR
/
*
.tar
rm
-f
$RESIFDD_WORKDIR
/report.lock
rm
-f
$RESIFDD_WORKDIR
/
*
.tar
##################
#
...
...
@@ -269,12 +269,12 @@ rm $RESIFDD_WORKDIR/*.tar
#
##################
KEY
=
"validated-seismic-metadata"
MONTH
=
$(
date
+%Y-%m
)
if
[[
-r
$RECOVERY_FILE
]]
&&
egrep
-q
-e
".*(
$KEY
).*( OK | Skipped ).*"
$RECOVERY_FILE
;
then
format_report
$KEY
"-"
$(
date
+%Y-%m-%dT%H:%M:%S
)
"-"
"-"
"Skipped"
>>
$LOCAL_REPORT
else
# Get the snapshot name for this month
MONTH
=
$(
date
+%Y-%m
)
SNAPSHOT_DIR
=
$(
ls
-d
$RESIFDD_DATADIR
/validated_seismic_metadata/.snapshot/monthly.
${
MONTH
}*
|tail
-1
)
if
[[
!
-d
$SNAPSHOT_DIR
]]
;
then
echo
"Error 000 Snapshot directory
$SNAPSHOT_DIR
does not exist"
...
...
Write
Preview
Supports
Markdown
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment