Commit 64a6f44b authored by Emmanuel Promayon's avatar Emmanuel Promayon

UPDATED camitk test requirement / disable tests message format

Using a clearer message when enabling/disabling test for a
given requirement or unconditionnally
Also remove the empty line due to trailing newline in XSD version string
parent 939d26eb
......@@ -27,25 +27,26 @@ camitk_disable_tests(TESTS action-itkfilters-level1-6
action-itkfilters-level1-8
action-itkfilters-level1-14
REASON " Test failure due to ITK exception.
For now disable these tests until this can be taken care of in the save() method
For input file: biorad.pic, cthead1.lsm, mini-complex-slow1.nrrd
When applying action Gradient Magnitude With Smoothing, the following exception is raised:
\"camitk-testcomponents aborted by std exception: /usr/include/ITK-4.10/itkRecursiveSeparableImageFilter.hxx:245:
itk::ERROR: RecursiveGaussianImageFilter(0x55b3b21bc700): The number of pixels along direction 2 is less than 4. This filter requires a minimum of four pixels along the dimension to be processed...\""
For now disable these tests until this can be taken care of in the save() method
For input file: biorad.pic, cthead1.lsm, mini-complex-slow1.nrrd
When applying action Gradient Magnitude With Smoothing, the following exception is raised:
\"camitk-testcomponents aborted by std exception: /usr/include/ITK-4.10/itkRecursiveSeparableImageFilter.hxx:245:
itk::ERROR: RecursiveGaussianImageFilter(0x55b3b21bc700): The number of pixels along direction 2 is less than 4.
This filter requires a minimum of four pixels along the dimension to be processed...\""
)
camitk_tests_requirement(TESTS action-itkfilters-integration-test
REQUIRES "NOT WIN32"
REASON "WIN32 OpenGL failure on a VM
This test will always fail when run inside a VM due to OpenGL crash.
This test will pass when run directly from a physical windows machine."
This test will always fail when run inside a VM due to OpenGL crash.
This test will pass when run directly from a physical windows machine."
)
camitk_tests_requirement(TESTS action-itkfilters-integration-test
REQUIRES "${VTK_VERSION} VERSION_EQUAL 6.3"
REASON "VTK version is not equals than 6.3
The default test files are written using the currently supported version of VTK (VTK 6.3).
This test will therefore fail when comparing the input to the output if another version of VTK is used.
E.g., Ubuntu 16.04 LTS comes with VTK 6.2 and Debian buster comes with VTK 7.1
Although everything should work well on both OS, this test will unconditionnaly fail."
The default test files are written using the currently supported version of VTK (VTK 6.3).
This test will therefore fail when comparing the input to the output if another version of VTK is used.
E.g., Ubuntu 16.04 LTS comes with VTK 6.2 and Debian buster comes with VTK 7.1
Although everything should work well on both OS, this test will unconditionnaly fail."
)
......@@ -18,11 +18,12 @@ camitk_disable_tests(TESTS component-itkimage-level3-3
component-itkimage-level3-19
component-itkimage-level3-20
REASON " Test failure due to ITK exception.
For now disable these tests until this can be dealt with (if possible)
For input file: biorad.pic, cthead1.lsm, BigEndian.spr, LittleEndianZ.hdr and LittleEndianZ.img.gz
When applying action Gradient Magnitude With Smoothing, the following exception is raised:
\"camitk-testcomponents aborted by std exception: /usr/include/ITK-4.10/itkRecursiveSeparableImageFilter.hxx:245:
itk::ERROR: RecursiveGaussianImageFilter(0x55b3b21bc700): The number of pixels along direction 2 is less than 4. This filter requires a minimum of four pixels along the dimension to be processed...\""
For now disable these tests until this can be dealt with (if possible)
For input file: biorad.pic, cthead1.lsm, BigEndian.spr, LittleEndianZ.hdr and LittleEndianZ.img.gz
When applying action Gradient Magnitude With Smoothing, the following exception is raised:
\"camitk-testcomponents aborted by std exception: /usr/include/ITK-4.10/itkRecursiveSeparableImageFilter.hxx:245:
itk::ERROR: RecursiveGaussianImageFilter(0x55b3b21bc700): The number of pixels along direction 2 is less than 4.
This filter requires a minimum of four pixels along the dimension to be processed...\""
)
# TODO confirm diagnostic: check why BigEndianCompressed.img.gz, LittleEndianCompressed.img.gz and minimal.nii.gz does not work on Windows (maybe because of double suffix .im.gz?)
......@@ -31,7 +32,7 @@ camitk_tests_requirement(TESTS component-itkimage-level3-6
component-itkimage-level3-28
REQUIRES "NOT WIN32"
REASON "WIN32 failure (probably) due to double suffix .im.gz or .nii.gz
For input file: BigEndianCompressed.img.gz, LittleEndianCompressed.img.gz and minimal.nii.gz"
For input file: BigEndianCompressed.img.gz, LittleEndianCompressed.img.gz and minimal.nii.gz"
)
camitk_tests_requirement(TESTS component-itkimage-level3-1
......@@ -45,8 +46,8 @@ camitk_tests_requirement(TESTS component-itkimage-level3-1
component-itkimage-level3-28
REQUIRES "${ITK_VERSION} VERSION_LESS 4.12"
REASON " ITK version is greater than 4.11
The default test files are written using the currently supported version of ITK (ITK 4.11).
This test will therefore fail when comparing the input to the output if another version of ITK is used."
The default test files are written using the currently supported version of ITK (ITK 4.11).
This test will therefore fail when comparing the input to the output if another version of ITK is used."
)
# Additional test for SDK actions: this will add auto test for the given actions using the itkimage component
......
......@@ -16,15 +16,15 @@ set(SHIBOKEN_CAMITK_SDK_PATH ${SHIBOKEN_CAMITK_SDK_PATH}:${CMAKE_CURRENT_SOURCE_
camitk_tests_requirement(TESTS action-reconstruction-integration-test
REQUIRES "${VTK_VERSION} VERSION_EQUAL 6.3"
REASON "VTK version is not equals than 6.3
The default test files are written using the currently supported version of VTK (VTK 6.3).
This test will therefore fail when comparing the input to the output if another version of VTK is used.
E.g., Ubuntu 16.04 LTS comes with VTK 6.2 and Debian buster comes with VTK 7.1
Although everything should work well on both OS, this test will unconditionnaly fail."
The default test files are written using the currently supported version of VTK (VTK 6.3).
This test will therefore fail when comparing the input to the output if another version of VTK is used.
E.g., Ubuntu 16.04 LTS comes with VTK 6.2 and Debian buster comes with VTK 7.1
Although everything should work well on both OS, this test will unconditionnaly fail."
)
camitk_tests_requirement(TESTS action-reconstruction-integration-test
REQUIRES "NOT WIN32"
REASON "WIN32 OpenGL failure on a VM
This test will always fail when run inside a VM due to OpenGL crash.
This test will pass when run directly from a physical windows machine."
This test will always fail when run inside a VM due to OpenGL crash.
This test will pass when run directly from a physical windows machine."
)
......@@ -59,7 +59,8 @@ if(XSD_FOUND)
endif()
if(NOT XSD_FIND_QUIETLY)
message(STATUS "Found XSD executable: ${XSD_EXECUTABLE} version ${XSD_VERSION}")
string(REGEX REPLACE "\n$" "" XSD_VERSION_WITHOUT_TRAILING_NEWLINE "${XSD_VERSION}")
message(STATUS "Found XSD executable: ${XSD_EXECUTABLE} version ${XSD_VERSION_WITHOUT_TRAILING_NEWLINE}")
endif()
else()
if(XSD_FIND_REQUIRED)
......
......@@ -47,18 +47,22 @@ macro(camitk_disable_tests)
set(multiValueArgs TESTS)
cmake_parse_arguments(CAMITK_DISABLE_TESTS "${options}" "${oneValueArgs}" "${multiValueArgs}" ${ARGN})
set(DISABLE_MESSAGE "")
# CMake < 3.9 did not have the DISABLED test property. Use WILL_FAIL instead
foreach(TEST_NAME ${CAMITK_DISABLE_TESTS_TESTS})
# CMake < 3.9 did not have the DISABLED test property. Use WILL_FAIL instead
if(${CMAKE_VERSION} VERSION_LESS "3.9")
set(DISABLE_MESSAGE ${DISABLE_MESSAGE} " Unconditionally disabling test ${TEST_NAME} (using WILL_FAIL property)\n")
set_tests_properties(${TEST_NAME} PROPERTIES WILL_FAIL true)
else()
set(DISABLE_MESSAGE ${DISABLE_MESSAGE} " Unconditionally disabling test ${TEST_NAME}\n")
set_tests_properties(${TEST_NAME} PROPERTIES DISABLED true)
endif()
endforeach()
set(DISABLE_MESSAGE ${DISABLE_MESSAGE} " Reason: ${CAMITK_DISABLE_TESTS_REASON}\n")
# message to the developper
string(REPLACE ";" " " CAMITK_DISABLE_TESTS_TESTS_STRING "${CAMITK_DISABLE_TESTS_TESTS}")
set(DISABLE_MESSAGE "Unconditionally disabling tests:\n Disabled tests: ${CAMITK_DISABLE_TESTS_TESTS_STRING}\n")
if(${CMAKE_VERSION} VERSION_LESS "3.9")
set(DISABLE_MESSAGE ${DISABLE_MESSAGE} " (using WILL_FAIL property because CMake < 3.9)\n")
endif()
set(DISABLE_MESSAGE ${DISABLE_MESSAGE} " Reason: ${CAMITK_DISABLE_TESTS_REASON}")
message(STATUS ${DISABLE_MESSAGE})
endmacro()
......@@ -168,24 +168,24 @@ macro(camitk_tests_requirement)
endif()
# -- disable tests if the requirement is not met
set(DISABLE_MESSAGE "")
string(REPLACE ";" " " CAMITK_TESTS_REQUIREMENT_TESTS_STRING "${CAMITK_TESTS_REQUIREMENT_TESTS}")
if(NOT ${REQUIRES_EVAL})
foreach(TEST_NAME ${CAMITK_TESTS_REQUIREMENT_TESTS})
set(DISABLE_MESSAGE ${DISABLE_MESSAGE} " Disabling test ${TEST_NAME}")
# CMake < 3.9 did not have the DISABLED test property. Use WILL_FAIL instead
if(${CMAKE_VERSION} VERSION_LESS "3.9")
set(DISABLE_MESSAGE ${DISABLE_MESSAGE} " (using WILL_FAIL property)")
set_tests_properties(${TEST_NAME} PROPERTIES WILL_FAIL true)
else()
set_tests_properties(${TEST_NAME} PROPERTIES DISABLED true)
endif()
set(DISABLE_MESSAGE ${DISABLE_MESSAGE} "\n")
endforeach()
set(DISABLE_MESSAGE ${DISABLE_MESSAGE} " Reason: ${CAMITK_TESTS_REQUIREMENT_REASON}\n")
set(DISABLE_MESSAGE ${DISABLE_MESSAGE} " Requirement statement unmet: ${CAMITK_TESTS_REQUIREMENT_REQUIRES}.\n")
set(DISABLE_MESSAGE "Test requirement \"${CAMITK_TESTS_REQUIREMENT_REQUIRES}\": Failed\n Disabled tests: ${CAMITK_TESTS_REQUIREMENT_TESTS_STRING}\n")
if(${CMAKE_VERSION} VERSION_LESS "3.9")
set(DISABLE_MESSAGE ${DISABLE_MESSAGE} " (using WILL_FAIL property because CMake < 3.9)\n")
endif()
set(DISABLE_MESSAGE ${DISABLE_MESSAGE} " Reason: ${CAMITK_TESTS_REQUIREMENT_REASON}\n")
set(DISABLE_MESSAGE ${DISABLE_MESSAGE} " Requirement statement unmet: ${CAMITK_TESTS_REQUIREMENT_REQUIRES}")
message(STATUS ${DISABLE_MESSAGE})
else()
string(REPLACE ";" " " CAMITK_TESTS_REQUIREMENT_TESTS_STRING "${CAMITK_TESTS_REQUIREMENT_TESTS}")
message(STATUS "Test requirement \"${CAMITK_TESTS_REQUIREMENT_REQUIRES}\": Passed\n Enabled tests: ${CAMITK_TESTS_REQUIREMENT_TESTS_STRING}")
endif()
endmacro()
......@@ -38,8 +38,8 @@ camitk_tests_requirement(TESTS component-vtkmesh-level3-1
component-vtkmesh-level3-16
REQUIRES "${VTK_VERSION} VERSION_EQUAL 6.3"
REASON "VTK version is not equals than 6.3
The default test files are written using the currently supported version of VTK (VTK 6.3).
This test will therefore fail when comparing the input to the output if another version of VTK is used.
E.g., Ubuntu 16.04 LTS comes with VTK 6.2 and Debian buster comes with VTK 7.1
Although everything should work well on both OS, this test will unconditionnaly fail."
The default test files are written using the currently supported version of VTK (VTK 6.3).
This test will therefore fail when comparing the input to the output if another version of VTK is used.
E.g., Ubuntu 16.04 LTS comes with VTK 6.2 and Debian buster comes with VTK 7.1
Although everything should work well on both OS, this test will unconditionnaly fail."
)
......@@ -9,6 +9,6 @@ camitk_extension(ACTION_EXTENSION
camitk_tests_requirement(TESTS action-hardcodedpipeline-integration-test
REQUIRES "NOT WIN32"
REASON "WIN32 OpenGL failure on a VM
This test will always fail when run inside a VM due to OpenGL crash.
This test will pass when run directly from a physical windows machine."
This test will always fail when run inside a VM due to OpenGL crash.
This test will pass when run directly from a physical windows machine."
)
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment