From 2ffb0afecbf4dcef5899100bbd723fd4d0699c22 Mon Sep 17 00:00:00 2001 From: Jon Cruz Date: Thu, 22 Oct 2015 13:25:54 -0700 Subject: [PATCH] zunitc: Clarify documentation on return behavior. * Clarify documentation on ZUC_ASSERT_* behavior in regards to return vs. abort() * Added overview section on return behavior. * Fixed spelling * Removed outdated reference to tap function. Changes since v1: * Incorporated grammatical feedback. Signed-off-by: Jon A. Cruz Reviewed-by: Bryce Harrington --- tools/zunitc/doc/zunitc.dox | 77 +++++++++++++++++++++++--- tools/zunitc/inc/zunitc/zunitc.h | 94 +++++++++++++++++++++----------- 2 files changed, 132 insertions(+), 39 deletions(-) diff --git a/tools/zunitc/doc/zunitc.dox b/tools/zunitc/doc/zunitc.dox index cef6c340..6fc68588 100644 --- a/tools/zunitc/doc/zunitc.dox +++ b/tools/zunitc/doc/zunitc.dox @@ -27,6 +27,7 @@ @page zunitc zunitc - @ref zunitc_overview + - @ref zunitc_overview_return - @ref zunitc_execution - @ref zunitc_execution_commandline - @ref zunitc_execution_matching @@ -38,12 +39,14 @@ @section zunitc_overview Overview -A simple test framework in plain C suitable for basic unit and integration testing. +A simple test framework in plain C suitable for basic unit and integration +testing. -The main rationale for creating this framework was to have a simple to use testing -framework with tests implemented in C using common patterns and under a +The main rationale for creating this framework was to have a simple to use +testing framework with tests implemented in C using common patterns and under a compatible license. The structure of the test code and macro use is intended to -follow common patterns established by frameworks such as Boost Test and Google Test. +follow common patterns established by frameworks such as Boost Test and Google +Test. To get started, one or more tests should be defined via ZUC_TEST() and/or @@ -53,9 +56,10 @@ To actually execute tests, ZUC_RUN_TESTS() should be called. Tests can use several ZUC_ASSERT_* or ZUC_ASSERTG_* checks to validate conditions. The ZUC_ASSERT_* ones upon failure will mark the current test -as failing and immediatly execute a return. On the other hand, the +as failing and immediately execute a return. On the other hand, the ZUC_ASSERTG_* tests will mark the current test as failed and then execute a -'goto' targeting the specified label. +'goto' targeting the specified label. See @ref zunitc_overview_return for more +detail. The set of fatal test checks are @@ -94,12 +98,66 @@ Unconditional test values for logging and termination are Unconditional message logging for failure cases only is - ZUC_TRACEPOINT() +@subsection zunitc_overview_return Test Termination and Return + +One key point where this framework differs from some others (especially many +common ad hoc test programs) is that it does not use assert() nor exceptions. + +- does not use assert() +- can not use throw + +One main implication of this is that when a check fails the current function +will be terminated via a 'return' statement and control passed back to the +calling function. If the check fails in an individual ZUC_TEST() or ZUC_TEST_F() +test function then control is returned to the framework and the current test is +deemed completed (either successfully or with failure). + +On the other hand, if a check fails that is being executed in a function called +by an individual test, then control will stay in the current test. In order to +successfully exit the current test a follow-up check needs to be done after +calling functions that might cause a failure. + +@code{.c} +... + + /* Call a function that might contain ZUC_ASSERT_* use. */ + some_helper_function(); + + /* Stop the current test if something failed. */ + ZUC_ASSERT_FALSE(zuc_has_failure()); + + /* execution will only reach this point if no failures occurred. */ + +... +@endcode + +Use of the ZUC_ASSERTG_*() variants can help in certain situations as check +failure will trigger a 'goto' statement as opposed to a general 'return'. + +@code{.c} +... + + /* Call a function that might contain ZUC_ASSERT_* use. */ + some_helper_function(); + + /* Stop the current test if something failed. */ + ZUC_ASSERTG_FALSE(zuc_has_failure(), err); + + /* execution will only reach this point if no failures occurred. */ +... + +err: + free(str_a); +} +... +@endcode + @section zunitc_execution Controlling Execution To control execution, the various zuc_set_* functions can be called before invoking ZUC_RUN_TESTS(). -@subsection zunitc_execution_commandline Commandline Parameters +@subsection zunitc_execution_commandline Command-line Parameters The current implementation defers processing of command-line parameters to the main application hosting the testing. It is possible that a @@ -118,6 +176,10 @@ following two special characters: - '*' matches any number of characters including zero. - '?' matches any single character. +This pattern matching is consistent with other testing frameworks and +has been chosen in order to make it easier for developers to move +between different testing frameworks. + Calling zuc_list_tests() after zuc_set_filter() can be done to show the effects of the matching without needing to actually run tests. @@ -151,7 +213,6 @@ parameter to ZUC_TEST_F(). - zuc_set_filter() - zuc_set_random() - zuc_set_spawn() -- zuc_set_output_tap() - zuc_set_output_junit() - zuc_has_skip() - zuc_has_failure() diff --git a/tools/zunitc/inc/zunitc/zunitc.h b/tools/zunitc/inc/zunitc/zunitc.h index d8c3cb00..d04b599a 100644 --- a/tools/zunitc/inc/zunitc/zunitc.h +++ b/tools/zunitc/inc/zunitc/zunitc.h @@ -299,6 +299,7 @@ zuc_set_output_junit(bool enable); * @return true if there is currently a test executing and it has * encountered any skips. * @see zuc_has_failure + * @see ZUC_SKIP() */ bool zuc_has_skip(void); @@ -314,7 +315,10 @@ bool zuc_has_failure(void); /** - * Terminates the current test without marking it as failed. + * Marks the running test as skipped without marking it as failed, and returns + * from the current function. + * + * For details on return and test termination see @ref zunitc_overview_return. * * @param message the message to log as to why the test has been skipped. */ @@ -326,7 +330,9 @@ zuc_has_failure(void); while (0) /** - * Terminates the current test and marks it as failed. + * Marks the running test as failed and returns from the current function. + * + * For details on return and test termination see @ref zunitc_overview_return. * * @param message the message to log as to why the test has failed. */ @@ -395,7 +401,9 @@ zuc_has_failure(void); /** * Verifies that the specified expression is true, marks the test as failed - * and terminates the test if it is not. + * and exits the current function via 'return' if it is not. + * + * For details on return and test termination see @ref zunitc_overview_return. * * @param condition the expression that is expected to be true. * @note it is far better to use a more specific check when possible @@ -407,7 +415,9 @@ zuc_has_failure(void); /** * Verifies that the specified expression is false, marks the test as - * failed and terminates the test if it is not. + * failed and exits the current function via 'return' if it is not. + * + * For details on return and test termination see @ref zunitc_overview_return. * * @param condition the expression that is expected to be false. * @note it is far better to use a more specific check when possible @@ -419,7 +429,9 @@ zuc_has_failure(void); /** * Verifies that the specified expression is NULL, marks the test as failed - * and terminates the test if it is not. + * and exits the current function via 'return' if it is not. + * + * For details on return and test termination see @ref zunitc_overview_return. * * @param condition the expression that is expected to be a NULL pointer. * @see ZUC_ASSERTG_NULL() @@ -429,7 +441,9 @@ zuc_has_failure(void); /** * Verifies that the specified expression is non-NULL, marks the test as - * failed and terminates the test if it is not. + * failed and exits the current function via 'return' if it is not. + * + * For details on return and test termination see @ref zunitc_overview_return. * * @param condition the expression that is expected to be a non-NULL pointer. * @see ZUC_ASSERTG_NOT_NULL() @@ -439,7 +453,9 @@ zuc_has_failure(void); /** * Verifies that the values of the specified expressions match, marks the - * test as failed and terminates the test if they do not. + * test as failed and exits the current function via 'return' if they do not. + * + * For details on return and test termination see @ref zunitc_overview_return. * * @param expected the value the result should hold. * @param actual the actual value seen in testing. @@ -450,7 +466,9 @@ zuc_has_failure(void); /** * Verifies that the values of the specified expressions differ, marks the - * test as failed and terminates the test if they do not. + * test as failed and exits the current function via 'return' if they do not. + * + * For details on return and test termination see @ref zunitc_overview_return. * * @param expected the value the result should not hold. * @param actual the actual value seen in testing. @@ -461,8 +479,10 @@ zuc_has_failure(void); /** * Verifies that the value of the first expression is less than the value - * of the second expression, marks the test as failed and terminates the - * test if it is not. + * of the second expression, marks the test as failed and exits the current + * function via 'return' if it is not. + * + * For details on return and test termination see @ref zunitc_overview_return. * * @param lesser the expression whose value should be lesser than the other. * @param greater the expression whose value should be greater than the other. @@ -474,7 +494,9 @@ zuc_has_failure(void); /** * Verifies that the value of the first expression is less than or equal * to the value of the second expression, marks the test as failed and - * terminates the test if it is not. + * exits the current function via 'return' if it is not. + * + * For details on return and test termination see @ref zunitc_overview_return. * * @param lesser the expression whose value should be lesser than or equal to * the other. @@ -487,8 +509,10 @@ zuc_has_failure(void); /** * Verifies that the value of the first expression is greater than the - * value of the second expression, marks the test as failed and terminates - * the test if it is not. + * value of the second expression, marks the test as failed and exits the + * current function via 'return' if it is not. + * + * For details on return and test termination see @ref zunitc_overview_return. * * @param greater the expression whose value should be greater than the other. * @param lesser the expression whose value should be lesser than the other. @@ -499,8 +523,10 @@ zuc_has_failure(void); /** * Verifies that the value of the first expression is greater than or equal - * to the value of the second expression, marks the test as failed and - * terminates the test if it is not. + * to the value of the second expression, marks the test as failed and exits + * the current function via 'return' if it is not. + * + * For details on return and test termination see @ref zunitc_overview_return. * * @param greater the expression whose value should be greater than or equal to * the other. @@ -514,7 +540,9 @@ zuc_has_failure(void); /** * Verifies that the values of the specified expressions match when * compared as null-terminated C-style strings, marks the test as failed - * and terminates the test if they do not. + * and exits the current function via 'return' if they do not. + * + * For details on return and test termination see @ref zunitc_overview_return. * * @param expected the value the result should hold. * @param actual the actual value seen in testing. @@ -526,7 +554,9 @@ zuc_has_failure(void); /** * Verifies that the values of the specified expressions differ when * compared as null-terminated C-style strings, marks the test as failed - * and terminates the test if they do not. + * and exits the current function via 'return' if they do not. + * + * For details on return and test termination see @ref zunitc_overview_return. * * @param expected the value the result should not hold. * @param actual the actual value seen in testing. @@ -537,7 +567,7 @@ zuc_has_failure(void); /** * Verifies that the specified expression is true, marks the test as failed - * and terminates the test via a 'goto' if it is not. + * and interrupts the current execution via a 'goto' if it is not. * * @param condition the expression that is expected to be true. * @note it is far better to use a more specific check when possible @@ -550,7 +580,7 @@ zuc_has_failure(void); /** * Verifies that the specified expression is false, marks the test as - * failed and terminates the test via a 'goto' if it is not. + * failed and interrupts the current execution via a 'goto' if it is not. * * @param condition the expression that is expected to be false. * @note it is far better to use a more specific check when possible @@ -563,7 +593,7 @@ zuc_has_failure(void); /** * Verifies that the specified expression is NULL, marks the test as failed - * and terminates the test via a 'goto' if it is not. + * and interrupts the current execution via a 'goto' if it is not. * * @param condition the expression that is expected to be a NULL pointer. * @param label the target for 'goto' if the assertion fails. @@ -574,7 +604,7 @@ zuc_has_failure(void); /** * Verifies that the specified expression is non-NULL, marks the test as - * failed and terminates the test via a 'goto' if it is not. + * failed and interrupts the current execution via a 'goto' if it is not. * * @param condition the expression that is expected to be a non-NULL pointer. * @param label the target for 'goto' if the assertion fails. @@ -585,7 +615,8 @@ zuc_has_failure(void); /** * Verifies that the values of the specified expressions match, marks the - * test as failed and terminates the test via a 'goto' if they do not. + * test as failed and interrupts the current execution via a 'goto' if they + * do not. * * @param expected the value the result should hold. * @param actual the actual value seen in testing. @@ -597,7 +628,8 @@ zuc_has_failure(void); /** * Verifies that the values of the specified expressions differ, marks the - * test as failed and terminates the test via a 'goto' if they do not. + * test as failed and interrupts the current execution via a 'goto' if they + * do not. * * @param expected the value the result should not hold. * @param actual the actual value seen in testing. @@ -609,8 +641,8 @@ zuc_has_failure(void); /** * Verifies that the value of the first expression is less than the value - * of the second expression, marks the test as failed and terminates the - * test if it is not. + * of the second expression, marks the test as failed and interrupts the + * current execution via a 'goto' if it is not. * * @param lesser the expression whose value should be lesser than the other. * @param greater the expression whose value should be greater than the other. @@ -623,7 +655,7 @@ zuc_has_failure(void); /** * Verifies that the value of the first expression is less than or equal * to the value of the second expression, marks the test as failed and - * terminates the test via a 'goto' if it is not. + * interrupts the current execution via a 'goto' if it is not. * * @param lesser the expression whose value should be lesser than or equal to * the other. @@ -637,8 +669,8 @@ zuc_has_failure(void); /** * Verifies that the value of the first expression is greater than the - * value of the second expression, marks the test as failed and terminates - * the test if it is not. + * value of the second expression, marks the test as failed and interrupts the + * current execution via a 'goto' if it is not. * * @param greater the expression whose value should be greater than the other. * @param lesser the expression whose value should be lesser than the other. @@ -651,7 +683,7 @@ zuc_has_failure(void); /** * Verifies that the value of the first expression is greater than or equal * to the value of the second expression, marks the test as failed and - * terminates the test via a 'goto' if it is not. + * interrupts the current execution via a 'goto' if it is not. * * @param greater the expression whose value should be greater than or equal to * the other. @@ -666,7 +698,7 @@ zuc_has_failure(void); /** * Verifies that the values of the specified expressions match when * compared as null-terminated C-style strings, marks the test as failed - * and terminates the test via a 'goto' if they do not. + * and interrupts the current execution via a 'goto' if they do not. * * @param expected the value the result should hold. * @param actual the actual value seen in testing. @@ -679,7 +711,7 @@ zuc_has_failure(void); /** * Verifies that the values of the specified expressions differ when * compared as null-terminated C-style strings, marks the test as failed - * and terminates the test via a 'goto' if they do not. + * and interrupts the current execution via a 'goto' if they do not. * * @param expected the value the result should not hold. * @param actual the actual value seen in testing.