This replaces the old test harness with a new one.
The old harness relied on fork()'ing each test which makes tests independent,
but makes debugging them harder. The new harness runs client code in a thread
instead of a new process. A side-effect of not fork()'ing anymore is that any
failure will stop running a test series short. Fortunately we do not have any
tests that are expected to crash or fail.
The old harness executed 'weston' from Meson, with lots of setup as both
command line options and environment variables. The new harness executes
wet_main() instead: the test program itself calls the compositor main function
to execute the compositor in-process. Command line arguments are configured in
the test program itself, not in meson.build. Environment variables aside, you
are able to run a test by simply executing the test program, even if it is a
plugin test.
The new harness adds a new type of iteration: fixtures. For now, fixtures are
used to set up the compositor for tests that need a compositor. If necessary, a
fixture setup may include a data array of arbitrary type for executing the test
series for each element in the array. This will be most useful for running
screenshooting tests with both Pixman- and GL-renderers.
The new harness outputs TAP formatted results into stdout. Meson is not
switched to consume TAP yet though, because it would require a Meson version
requirement bump and would not have any benefits at this time. OTOH outputting
TAP is trivial and sets up a clear precedent of random test chatter belonging
to stderr.
This commit migrates only few tests to actually make use of the new features:
roles is a basic client test, subsurface-shot is a client test that
demonstrates the fixture array, and plugin-registry is a plugin test. The rest
of the tests will be migrated later.
Once all tests are migrated, we can remove the test-specific setup from
meson.build, leaving only the actual build instructions in there.
The not migrated tests and stand-alone tests suffer only a minor change: they
no longer fork() for each TEST(), otherwise they keep running as before.
Signed-off-by: Pekka Paalanen <pekka.paalanen@collabora.com>
When we move on to TAP, stdout will be reserved for TAP and stderr is for free
chatter. Set up an example that tests should use testlog() instead of fprintf
or printf to chat in the right place.
Most statements were already printing to stderr, so this just makes then a
little shorter. There are also some statements that printed to stdout and are
now corrected.
Signed-off-by: Pekka Paalanen <pekka.paalanen@collabora.com>
Using the test name for the reference images will stop working when the new
test harness starts using fixtures. Fixtures allow running the same tests in
varying environments, so the test results file names will include fixture
index. However the reference images will remain the same for all fixtures.
Signed-off-by: Pekka Paalanen <pekka.paalanen@collabora.com>
Enhance the existing pointer motion and button event tests to
additionally verify the event timestamps. This requires updating the
weston-test protocol to support passing motion and button event
timestamps.
Signed-off-by: Alexandros Frantzis <alexandros.frantzis@collabora.com>
Reviewed-by: Pekka Paalanen <pekka.paalanen@collabora.co.uk>
When a client changes the subsurfaces state, we need to damage
them so the result is visible. We do that by flagging the surfaces
when the state changes and causing damage when committing the
state. This prevents normal repaints from considering these changes
until a commit has happened, and allows the client to atomically
schedule several changes.
This fixes the subsurface_z_order test, which is now marked as expected
to succeed.
Signed-off-by: Emilio Pozuelo Monfort <emilio.pozuelo@collabora.co.uk>
Reviewed-by: Pekka Paalanen <pekka.paalanen@collabora.co.uk>
Reviewed-by: Micah Fedke <micah.fedke@collabora.co.uk>
This is marked as a FAIL_TEST, because the last image comparison fails
due to a bug in Weston.
Jointly authored by Pekka and Emilio.
Signed-off-by: Pekka Paalanen <pekka.paalanen@collabora.co.uk>
Signed-off-by: Emilio Pozuelo Monfort <emilio.pozuelo@collabora.co.uk>
Reviewed-by: Micah Fedke <micah.fedke@collabora.co.uk>
[Pekka: move weston-tests-env as terminator to EXTRA_DIST, change
ok/FAIL to PASS/FAIL, write diff image only on fail.]
Acked-by: Quentin Glidic <sardemff7+git@sardemff7.net>
Pekka Paalanen
Alexandros Frantzis
Emilio Pozuelo Monfort
Pekka Paalanen