tests: thread-based client harness
This replaces the old test harness with a new one.
The old harness relied on fork()'ing each test which makes tests independent,
but makes debugging them harder. The new harness runs client code in a thread
instead of a new process. A side-effect of not fork()'ing anymore is that any
failure will stop running a test series short. Fortunately we do not have any
tests that are expected to crash or fail.
The old harness executed 'weston' from Meson, with lots of setup as both
command line options and environment variables. The new harness executes
wet_main() instead: the test program itself calls the compositor main function
to execute the compositor in-process. Command line arguments are configured in
the test program itself, not in meson.build. Environment variables aside, you
are able to run a test by simply executing the test program, even if it is a
plugin test.
The new harness adds a new type of iteration: fixtures. For now, fixtures are
used to set up the compositor for tests that need a compositor. If necessary, a
fixture setup may include a data array of arbitrary type for executing the test
series for each element in the array. This will be most useful for running
screenshooting tests with both Pixman- and GL-renderers.
The new harness outputs TAP formatted results into stdout. Meson is not
switched to consume TAP yet though, because it would require a Meson version
requirement bump and would not have any benefits at this time. OTOH outputting
TAP is trivial and sets up a clear precedent of random test chatter belonging
to stderr.
This commit migrates only few tests to actually make use of the new features:
roles is a basic client test, subsurface-shot is a client test that
demonstrates the fixture array, and plugin-registry is a plugin test. The rest
of the tests will be migrated later.
Once all tests are migrated, we can remove the test-specific setup from
meson.build, leaving only the actual build instructions in there.
The not migrated tests and stand-alone tests suffer only a minor change: they
no longer fork() for each TEST(), otherwise they keep running as before.
Signed-off-by: Pekka Paalanen <pekka.paalanen@collabora.com>
5 years ago
|
|
|
/*
|
|
|
|
* Copyright 2019 Collabora, Ltd.
|
|
|
|
*
|
|
|
|
* Permission is hereby granted, free of charge, to any person obtaining
|
|
|
|
* a copy of this software and associated documentation files (the
|
|
|
|
* "Software"), to deal in the Software without restriction, including
|
|
|
|
* without limitation the rights to use, copy, modify, merge, publish,
|
|
|
|
* distribute, sublicense, and/or sell copies of the Software, and to
|
|
|
|
* permit persons to whom the Software is furnished to do so, subject to
|
|
|
|
* the following conditions:
|
|
|
|
*
|
|
|
|
* The above copyright notice and this permission notice (including the
|
|
|
|
* next paragraph) shall be included in all copies or substantial
|
|
|
|
* portions of the Software.
|
|
|
|
*
|
|
|
|
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
|
|
|
|
* EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
|
|
|
|
* MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
|
|
|
|
* NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
|
|
|
|
* BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
|
|
|
|
* ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
|
|
|
|
* CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
|
|
* SOFTWARE.
|
|
|
|
*/
|
|
|
|
|
|
|
|
#ifndef WESTON_TEST_FIXTURE_COMPOSITOR_H
|
|
|
|
#define WESTON_TEST_FIXTURE_COMPOSITOR_H
|
|
|
|
|
|
|
|
#include <wayland-client-protocol.h>
|
tests: thread-based client harness
This replaces the old test harness with a new one.
The old harness relied on fork()'ing each test which makes tests independent,
but makes debugging them harder. The new harness runs client code in a thread
instead of a new process. A side-effect of not fork()'ing anymore is that any
failure will stop running a test series short. Fortunately we do not have any
tests that are expected to crash or fail.
The old harness executed 'weston' from Meson, with lots of setup as both
command line options and environment variables. The new harness executes
wet_main() instead: the test program itself calls the compositor main function
to execute the compositor in-process. Command line arguments are configured in
the test program itself, not in meson.build. Environment variables aside, you
are able to run a test by simply executing the test program, even if it is a
plugin test.
The new harness adds a new type of iteration: fixtures. For now, fixtures are
used to set up the compositor for tests that need a compositor. If necessary, a
fixture setup may include a data array of arbitrary type for executing the test
series for each element in the array. This will be most useful for running
screenshooting tests with both Pixman- and GL-renderers.
The new harness outputs TAP formatted results into stdout. Meson is not
switched to consume TAP yet though, because it would require a Meson version
requirement bump and would not have any benefits at this time. OTOH outputting
TAP is trivial and sets up a clear precedent of random test chatter belonging
to stderr.
This commit migrates only few tests to actually make use of the new features:
roles is a basic client test, subsurface-shot is a client test that
demonstrates the fixture array, and plugin-registry is a plugin test. The rest
of the tests will be migrated later.
Once all tests are migrated, we can remove the test-specific setup from
meson.build, leaving only the actual build instructions in there.
The not migrated tests and stand-alone tests suffer only a minor change: they
no longer fork() for each TEST(), otherwise they keep running as before.
Signed-off-by: Pekka Paalanen <pekka.paalanen@collabora.com>
5 years ago
|
|
|
#include <libweston/libweston.h>
|
|
|
|
|
|
|
|
#include "weston-testsuite-data.h"
|
|
|
|
|
|
|
|
/** Weston renderer type
|
|
|
|
*
|
|
|
|
* \sa compositor_setup
|
|
|
|
* \ingroup testharness
|
|
|
|
*/
|
|
|
|
enum renderer_type {
|
|
|
|
/** Dummy renderer that does nothing. */
|
|
|
|
RENDERER_NOOP = 0,
|
|
|
|
/** Pixman-renderer */
|
|
|
|
RENDERER_PIXMAN,
|
|
|
|
/** GL-renderer */
|
|
|
|
RENDERER_GL
|
|
|
|
};
|
|
|
|
|
|
|
|
/** Weston shell plugin
|
|
|
|
*
|
|
|
|
* \sa compositor_setup
|
|
|
|
* \ingroup testharness
|
|
|
|
*/
|
|
|
|
enum shell_type {
|
|
|
|
/** Desktop test-shell with predictable window placement and
|
|
|
|
* no helper clients */
|
|
|
|
SHELL_TEST_DESKTOP = 0,
|
|
|
|
/** The full desktop shell. */
|
|
|
|
SHELL_DESKTOP,
|
|
|
|
/** The ivi-shell. */
|
|
|
|
SHELL_IVI,
|
|
|
|
/** The fullscreen-shell. */
|
|
|
|
SHELL_FULLSCREEN
|
|
|
|
};
|
|
|
|
|
|
|
|
/** Weston compositor configuration
|
|
|
|
*
|
|
|
|
* This structure determines the Weston compositor command line arguments.
|
|
|
|
* You should always use compositor_setup_defaults() to initialize this, then
|
|
|
|
* override any members you need with assignments.
|
|
|
|
*
|
|
|
|
* \ingroup testharness
|
|
|
|
*/
|
|
|
|
struct compositor_setup {
|
|
|
|
/** The backend to use. */
|
|
|
|
enum weston_compositor_backend backend;
|
|
|
|
/** The renderer to use. */
|
|
|
|
enum renderer_type renderer;
|
|
|
|
/** The shell plugin to use. */
|
|
|
|
enum shell_type shell;
|
|
|
|
/** Whether to enable xwayland support. */
|
|
|
|
bool xwayland;
|
|
|
|
/** Default output width. */
|
|
|
|
unsigned width;
|
|
|
|
/** Default output height. */
|
|
|
|
unsigned height;
|
|
|
|
/** Default output scale. */
|
|
|
|
int scale;
|
|
|
|
/** Default output transform, one of WL_OUTPUT_TRANSFORM_*. */
|
|
|
|
enum wl_output_transform transform;
|
tests: thread-based client harness
This replaces the old test harness with a new one.
The old harness relied on fork()'ing each test which makes tests independent,
but makes debugging them harder. The new harness runs client code in a thread
instead of a new process. A side-effect of not fork()'ing anymore is that any
failure will stop running a test series short. Fortunately we do not have any
tests that are expected to crash or fail.
The old harness executed 'weston' from Meson, with lots of setup as both
command line options and environment variables. The new harness executes
wet_main() instead: the test program itself calls the compositor main function
to execute the compositor in-process. Command line arguments are configured in
the test program itself, not in meson.build. Environment variables aside, you
are able to run a test by simply executing the test program, even if it is a
plugin test.
The new harness adds a new type of iteration: fixtures. For now, fixtures are
used to set up the compositor for tests that need a compositor. If necessary, a
fixture setup may include a data array of arbitrary type for executing the test
series for each element in the array. This will be most useful for running
screenshooting tests with both Pixman- and GL-renderers.
The new harness outputs TAP formatted results into stdout. Meson is not
switched to consume TAP yet though, because it would require a Meson version
requirement bump and would not have any benefits at this time. OTOH outputting
TAP is trivial and sets up a clear precedent of random test chatter belonging
to stderr.
This commit migrates only few tests to actually make use of the new features:
roles is a basic client test, subsurface-shot is a client test that
demonstrates the fixture array, and plugin-registry is a plugin test. The rest
of the tests will be migrated later.
Once all tests are migrated, we can remove the test-specific setup from
meson.build, leaving only the actual build instructions in there.
The not migrated tests and stand-alone tests suffer only a minor change: they
no longer fork() for each TEST(), otherwise they keep running as before.
Signed-off-by: Pekka Paalanen <pekka.paalanen@collabora.com>
5 years ago
|
|
|
/** The absolute path to \c weston.ini to use,
|
|
|
|
* or NULL for \c --no-config .
|
|
|
|
* To properly fill this entry use weston_ini_setup() */
|
|
|
|
char *config_file;
|
tests: thread-based client harness
This replaces the old test harness with a new one.
The old harness relied on fork()'ing each test which makes tests independent,
but makes debugging them harder. The new harness runs client code in a thread
instead of a new process. A side-effect of not fork()'ing anymore is that any
failure will stop running a test series short. Fortunately we do not have any
tests that are expected to crash or fail.
The old harness executed 'weston' from Meson, with lots of setup as both
command line options and environment variables. The new harness executes
wet_main() instead: the test program itself calls the compositor main function
to execute the compositor in-process. Command line arguments are configured in
the test program itself, not in meson.build. Environment variables aside, you
are able to run a test by simply executing the test program, even if it is a
plugin test.
The new harness adds a new type of iteration: fixtures. For now, fixtures are
used to set up the compositor for tests that need a compositor. If necessary, a
fixture setup may include a data array of arbitrary type for executing the test
series for each element in the array. This will be most useful for running
screenshooting tests with both Pixman- and GL-renderers.
The new harness outputs TAP formatted results into stdout. Meson is not
switched to consume TAP yet though, because it would require a Meson version
requirement bump and would not have any benefits at this time. OTOH outputting
TAP is trivial and sets up a clear precedent of random test chatter belonging
to stderr.
This commit migrates only few tests to actually make use of the new features:
roles is a basic client test, subsurface-shot is a client test that
demonstrates the fixture array, and plugin-registry is a plugin test. The rest
of the tests will be migrated later.
Once all tests are migrated, we can remove the test-specific setup from
meson.build, leaving only the actual build instructions in there.
The not migrated tests and stand-alone tests suffer only a minor change: they
no longer fork() for each TEST(), otherwise they keep running as before.
Signed-off-by: Pekka Paalanen <pekka.paalanen@collabora.com>
5 years ago
|
|
|
/** Full path to an extra plugin to load, or NULL for none. */
|
|
|
|
const char *extra_module;
|
|
|
|
/** Debug scopes for the compositor log,
|
|
|
|
* or NULL for compositor defaults. */
|
|
|
|
const char *logging_scopes;
|
|
|
|
/** The name of this test program, used as a unique identifier. */
|
|
|
|
const char *testset_name;
|
|
|
|
};
|
|
|
|
|
|
|
|
void
|
|
|
|
compositor_setup_defaults_(struct compositor_setup *setup,
|
|
|
|
const char *testset_name);
|
|
|
|
|
|
|
|
/** Initialize compositor setup to defaults
|
|
|
|
*
|
|
|
|
* \param s The variable to initialize.
|
|
|
|
*
|
|
|
|
* The defaults are:
|
|
|
|
* - backend: headless
|
|
|
|
* - renderer: noop
|
|
|
|
* - shell: desktop shell
|
|
|
|
* - xwayland: no
|
|
|
|
* - width: 320
|
|
|
|
* - height: 240
|
|
|
|
* - scale: 1
|
|
|
|
* - transform: WL_OUTPUT_TRANSFORM_NORMAL
|
tests: thread-based client harness
This replaces the old test harness with a new one.
The old harness relied on fork()'ing each test which makes tests independent,
but makes debugging them harder. The new harness runs client code in a thread
instead of a new process. A side-effect of not fork()'ing anymore is that any
failure will stop running a test series short. Fortunately we do not have any
tests that are expected to crash or fail.
The old harness executed 'weston' from Meson, with lots of setup as both
command line options and environment variables. The new harness executes
wet_main() instead: the test program itself calls the compositor main function
to execute the compositor in-process. Command line arguments are configured in
the test program itself, not in meson.build. Environment variables aside, you
are able to run a test by simply executing the test program, even if it is a
plugin test.
The new harness adds a new type of iteration: fixtures. For now, fixtures are
used to set up the compositor for tests that need a compositor. If necessary, a
fixture setup may include a data array of arbitrary type for executing the test
series for each element in the array. This will be most useful for running
screenshooting tests with both Pixman- and GL-renderers.
The new harness outputs TAP formatted results into stdout. Meson is not
switched to consume TAP yet though, because it would require a Meson version
requirement bump and would not have any benefits at this time. OTOH outputting
TAP is trivial and sets up a clear precedent of random test chatter belonging
to stderr.
This commit migrates only few tests to actually make use of the new features:
roles is a basic client test, subsurface-shot is a client test that
demonstrates the fixture array, and plugin-registry is a plugin test. The rest
of the tests will be migrated later.
Once all tests are migrated, we can remove the test-specific setup from
meson.build, leaving only the actual build instructions in there.
The not migrated tests and stand-alone tests suffer only a minor change: they
no longer fork() for each TEST(), otherwise they keep running as before.
Signed-off-by: Pekka Paalanen <pekka.paalanen@collabora.com>
5 years ago
|
|
|
* - config_file: none
|
|
|
|
* - extra_module: none
|
|
|
|
* - logging_scopes: compositor defaults
|
|
|
|
* - testset_name: the test name from meson.build
|
|
|
|
*
|
|
|
|
* \ingroup testharness
|
|
|
|
*/
|
|
|
|
#define compositor_setup_defaults(s) do {\
|
|
|
|
compositor_setup_defaults_(s, THIS_TEST_NAME); \
|
|
|
|
} while (0)
|
|
|
|
|
|
|
|
int
|
|
|
|
execute_compositor(const struct compositor_setup *setup,
|
|
|
|
struct wet_testsuite_data *data);
|
|
|
|
|
|
|
|
/* This function creates and fills a Weston.ini file
|
|
|
|
*
|
|
|
|
* \param setup a compositor_setup
|
|
|
|
* \param ... Variadic number of strings that will be used to compose
|
|
|
|
* the Weston.ini
|
|
|
|
*
|
|
|
|
* This function receives a compositor_setup and a variable number of
|
|
|
|
* strings that will compose the weston.ini. And will create a
|
|
|
|
* <test-name>.ini and fill it with the entries.
|
|
|
|
* This function will assert if anything goes wrong, then it will not
|
|
|
|
* have a return value to indicate success or failure.
|
|
|
|
*
|
|
|
|
* \ingroup testharness
|
|
|
|
*/
|
|
|
|
#define weston_ini_setup(setup, ...) \
|
|
|
|
weston_ini_setup_(setup, __VA_ARGS__, NULL)
|
|
|
|
|
|
|
|
void
|
|
|
|
weston_ini_setup_(struct compositor_setup *setup, ...);
|
|
|
|
|
|
|
|
char *
|
|
|
|
cfgln(const char *fmt, ...);
|
|
|
|
|
tests: thread-based client harness
This replaces the old test harness with a new one.
The old harness relied on fork()'ing each test which makes tests independent,
but makes debugging them harder. The new harness runs client code in a thread
instead of a new process. A side-effect of not fork()'ing anymore is that any
failure will stop running a test series short. Fortunately we do not have any
tests that are expected to crash or fail.
The old harness executed 'weston' from Meson, with lots of setup as both
command line options and environment variables. The new harness executes
wet_main() instead: the test program itself calls the compositor main function
to execute the compositor in-process. Command line arguments are configured in
the test program itself, not in meson.build. Environment variables aside, you
are able to run a test by simply executing the test program, even if it is a
plugin test.
The new harness adds a new type of iteration: fixtures. For now, fixtures are
used to set up the compositor for tests that need a compositor. If necessary, a
fixture setup may include a data array of arbitrary type for executing the test
series for each element in the array. This will be most useful for running
screenshooting tests with both Pixman- and GL-renderers.
The new harness outputs TAP formatted results into stdout. Meson is not
switched to consume TAP yet though, because it would require a Meson version
requirement bump and would not have any benefits at this time. OTOH outputting
TAP is trivial and sets up a clear precedent of random test chatter belonging
to stderr.
This commit migrates only few tests to actually make use of the new features:
roles is a basic client test, subsurface-shot is a client test that
demonstrates the fixture array, and plugin-registry is a plugin test. The rest
of the tests will be migrated later.
Once all tests are migrated, we can remove the test-specific setup from
meson.build, leaving only the actual build instructions in there.
The not migrated tests and stand-alone tests suffer only a minor change: they
no longer fork() for each TEST(), otherwise they keep running as before.
Signed-off-by: Pekka Paalanen <pekka.paalanen@collabora.com>
5 years ago
|
|
|
#endif /* WESTON_TEST_FIXTURE_COMPOSITOR_H */
|