CamiTK Release Management
As a | CamiTK developer |
I would like to | Release new CamiTK version |
So that | A new stable version is available to users and ready to be packaged in Debian |
Description / Overview
This issue is meant to be reused for each release, it is a direct translation of an internal document with the list of task to carry out for each release. It might be a bit technical for non CamiTK developer, but the targeted audience is only the maintainers (although remarks and suggestions are always welcomed!)
Forewords
A release:
- cannot have blocking bugs
- must have no compilation error on supported platforms and no failed test on CDash
Normally all these requirements should be match thanks to the git branching model, the agile process and the continuous integration system.
Some extra steps are recommended here in order to improve the quality of the release, reduce technological debt along the way, and ensure a smoother debian packaging process.
Steps
This is the detailed procedure of each step. Tick the corresponding task below when a step is achieved.
Step 1. Create a release management MR
To do so:
- Re-open this issue
- Create a corresponding branch + merge request but change the default name to
camitk-x.y-release-management
Step 2. Prepare the environment
This step installs the Debian sid image and prepare everything needed to check the current develop branch.
You need to have docker installed, if you do not:
# Install docker package
sudo apt install docker.io
# Add yourself to the docker group
sudo usermod -aG docker $(whoami)
# log in a different terminal to be in the docker group
# set the version
export version="x.y"
# build the docker image
git clone git@gricad-gitlab.univ-grenoble-alpes.fr:CamiTK/camitk-docker.git
cd camitk-docker
docker build --rm -t camitk:unstable-release-test unstable-release-test
# create the docker network to have static ip for container
docker network create --subnet=172.20.0.0/16 camitk-docker-subnet
# create the homedir with the code
cd where/you/want/to/work
export workspace=$PWD
cd $workspace
git clone git@gricad-gitlab.univ-grenoble-alpes.fr:CamiTK/CamiTK.git -b camitk-$version-release-management src
# create the "permanent" container
# ⚠ the docker image is built for user id/group id 1000:1000
# use --user "$(id -u):$(id -g)" if your id/group id is not 1000:1000
docker container run \
--privileged=true \
--net camitk-docker-subnet \
--ip 172.20.0.33 \
--name camitk-release-$version \
--restart unless-stopped \
-d \
--ipc=host \
-e DISPLAY=$DISPLAY \
-v /tmp/.X11-unix:/tmp/.X11-unix \
-v $workspace:/home/camitk/workspace \
-w /home/camitk/workspace \
-ti camitk:unstable-release-test
# get inside the docker
docker container exec -it camitk-release-$version bash
Step 3. Fix clang tidy warnings
(or some of them)
# Note: add -G Ninja to use ninja build system
cmake -S src -B build -DCEP_IMAGING:BOOL=TRUE -DCEP_MODELING:BOOL=TRUE -DCEP_TUTORIALS:BOOL=TRUE -DAPIDOC_SDK:BOOL=TRUE -DCMAKE_EXPORT_COMPILE_COMMANDS=ON -G Ninja
cmake --build build -j20
run-clang-tidy -header-filter='src/**/.*' -p=build -export-fixes build/clang-tidy-fixes-$(date +"%m-%d-%H:%M") 2>&1 | tee build/clang-error-$(date +"%m-%d-%H:%M").log
It is recommended to fix "invoking virtual functions from constructors or destructors", using one of the following three solutions
- solution 1: When methods are private, just remove the unjustified virtual keyword
- solution 2: declare virtual method called in constructor as final, see OOP50-CPP-EX2 of SEI CERT C++ Coding Standard
- solution 3: call the virtual function with an explicitly qualified ID (e.g, Application::eventFilter), see OOP50-CPP-EX1 of SEI CERT C++ Coding Standard as well
Leave qtpropertybrowser source code intact to facilitate keeping it sync with other forks.
Note that sometimes the "Potential memory leak" warning should not be fixed (e.g, new QEvent(...)
are used in Application
, InteractiveViewer
, ActionState
are used to force updates)
Step 4. Fix clang compiler warnings
rm -rf build
cmake -S src -B build -DCEP_IMAGING:BOOL=TRUE -DCEP_MODELING:BOOL=TRUE -DCEP_TUTORIALS:BOOL=TRUE -DAPIDOC_SDK:BOOL=TRUE -DCMAKE_CXX_COMPILER=$(which clang++) -DCMAKE_C_COMPILER=$(which clang) -G Ninja
cmake --build build -j20
Step 5. Fix source code format using astyle
!!! Warning
recent astyle version propertly manages C++ raw string, but the version installed on ubuntu 24.04 LTS (3.1) does not. Either use a more recent version or check the raw string formatting afterwards (especially in qtest file and regexp, see R"(...)"
R"json(...)json"
R"re(..)re"
, grep for something like R"\(\S
)
Run the following to reformat all code (see CamiTK Programming Guidelines):
cd src
astyle --style=java \
--break-closing-brackets \
--add-brackets \
--unpad-paren \
--pad-oper \
--pad-header \
--align-pointer=type \
--indent-switches \
--suffix=none \
$(find . -name "*.cpp" -or -name "*.h")
Step 6. Update source code comment with the current year
To check the file that have to be updated if last release was done in 20xx:
# Get the last tag and extract its year using git
lastReleaseYear=$(git show --format=%ai $(git describe --tags $(git rev-list --tags --max-count=1)) | cut -d ' ' -f 1 | cut -d '-' -f 1 | tail -2 | head -1)
grep -rH "2001-$lastReleaseYear"
To update all files to release in 20yy:
# update
newReleaseYear=$(date +"%Y")
if [ "$newReleaseYear" != "$lastReleaseYear" ]; then
filesToUpdate=$(grep -rHl "2001-$lastReleaseYear")
for i in $filesToUpdate ; do
sed -i "s/(c) 2001-$lastReleaseYear Univ/(c) 2001-$newReleaseYear Univ/g" $i;
done
fi
Step 7. Update packaging
warning: That's probably the hardest of all steps!
cd ..
rm -rf build
sudo apt update
sudo apt upgrade
PROJECT_LOG_DIR=log bash src/.gitlab/package.sh
Process
- execute this script a first time, and then you can work on the debian/ directory
- if the package is not produced:
-
prepare release CamiTKVersion.h and recreate orig
-
cd src
(that's where camitk-x.y-release-management branch sits) -
cmake -S . -B build -DCEP_IMAGING:BOOL=TRUE -DCEP_MODELING:BOOL=TRUE -DCEP_TUTORIALS:BOOL=TRUE -DAPIDOC_SDK:BOOL=TRUE
-
edit
./build/sdk/libraries/core/CamiTKVersion.h
so thatCore::version
is equal toCamiTK ${version}.0
exactly -
cp ./build/sdk/libraries/core/CamiTKVersion.h ../camitk-debian/sdk/libraries/core/
-
check autopkgtest files in
../camitk-debian/debian/tests/*.sh
:cepgenerator-test.sh
config-test.sh
generate-coreschema-files.sh
andgenerate-cpp-files.sh
- if any change were done on debian git and not integrated upstream, merge with corresponding files in
sdk/applications/config/testing
andsdk/applications/cepgenerator/testing
- then copy the new versions to
camitk-debian/debian/tests
: cp sdk/applications/config/testing/config-test.sh sdk/applications/cepgenerator/testing/*.sh ../camitk-debian/debian/tests/
- if any change were done on debian git and not integrated upstream, merge with corresponding files in
-
cd ../camitk-debian
-
dpkg-source --commit
This creates a
debian/patches
file that contains the addition of the generatedCamiTKVersion.h
.⚠ Do not push back this to salsa. BEWARE : this specific patch SHOULD NOT be commited back to salsa. As this patch is now listed indebian/patches/series
it will be automatically applied during the package build.(TODO find a better way to include CamiTKVersion.h before first tarball is produced)
-
-
fiddle with the debian/ files, start with:
-
check the packaging after a change with:
{ time gbp buildpackage \
--git-no-create-orig \
--git-ignore-new \
--git-postbuild='~/.gbp-postbuild.sh' 2>&1 ; } 2>&1
This produces two logs:
- build log: camitk_version_amd64.build
- dependencies list: camitk_version_amd64.buildinfo
Note if things have to be modifid in the CamiTK source code, report all modification to the camitk-x.y-release-management branch
- back to fiddling if it did not work
- commit the changes and push to salsa
When everything looks good, create a new docker from naked sid and run the autotest from the camitk-debian
directory:
docker run -v.:/home/camitk/workspace -ti debian:sid bash
# in docker container
cd /home/camitk/workspace
apt update
apt install -y autopkgtest
autopkgtest -l autopkgtest-sid.log camitk_5.1.202307271147-1_amd64.changes -- null
Step 8. Merge release management branch in develop
Now that this branch is merge to develop, it is time to create the actual new release.
See the Release New Version.md
document on the CamiTK developer cloud.
Tasks (see corresponding step)
-
1. Create a release management MR -
2. Prepare the environment -
3. Fix clang tidy warnings -
4. Fix clang compiler warnings -
5. Fix source code format using astyle -
6. Update source code comment with the current year -
7. Update packaging -
8. Merge release management branch in develop
TODO
The following TODO list requires attention so that they can more reproducible / systematically applied. They should be worked on and then added to the previous steps as a release management step in their full right.
camitk-${version}-release-management
tarball
Packaging should use Script .gitlab/package.sh
uses the current develop
branch tarball to test the packaging. It should use the current state of camitk-${version}-release-management
instead, as this branch is closer to the release branch than develop.
Fix clazy warnings (or some of them)
(see this list for details. It is recommended to fix: fully-qualified-moc-types, qstring-args, qenums and most (but not all as there was some false positive or some that required more complete rewrite)container-anti-pattern and range-loop. Note that: non-pod-global-static, connect-3arg-lambda, qtpropertybrowser and source code generated by xsdcxx are not recommended.
Fix cppcheck warnings (or some of them)
cppcheck can be ran with --enable=performance --enable=portability --library=qt -i ./CMakeFiles
It is recommended to fix for instance:
- (performance) Prefer prefix ++/-- operators for non-primitive types.
- (performance) Variable assigned in constructor body moved to initialization list.
The "(performance) Function parameter 'xxx' should be passed by const reference" should not be change (at this stage anyway) as it will change the API
see this page for an example of set up
Memory leak
A systematic recommended way to check memory leak is to use heaptrack on Linux.
An easy way to do this for CamiTK is to run the itkfilter integration test (or any other integration tests).
You can list all integration tests with ctest -N | grep integration
You can get the line that is executed by a test by adding -VV
to ctest e.g. ctest -VV -R action-itkfilters-integration-test
heaptrack bin/camitk-actionstatemachine -f /home/promayon/Dev/CamiTK/build/camitk/Testing/Temporary/action-itkfilters-integration-test/asm-input.scxml -o /home/promayon/Dev/CamiTK/build/camitk/Testing/Temporary/action-itkfilters-integration-test -a
View and check the resulting gz
file using heaptrack_gui
generate new API documentation
Test and generate new API documentation with doxygen and update doxyfile accordingly → set APIDOC_SDK
option in CMake
configure to TRUE
, run target camitk-ce-api-doc
Test deployment on windows
- Create the user archive (see script in Appendix 1)
- test the user archive on a bare windows machine