The new field in struct scalar_stat allows recording all tested values
into a file. This is intended to replace ad hoc dumping code like in
alpha-blending-test.c.
To make it easy to set up, also offer a helper to open a writable file
whose name consists of a custom prefix and test name.
Signed-off-by: Pekka Paalanen <pekka.paalanen@collabora.com>
I will be needing it in a new test, so let's share it.
For convenience, this also adds client back-pointer in struct surface so
that I don't need to pass client explicitly.
Signed-off-by: Pekka Paalanen <pekka.paalanen@collabora.com>
skip() is a left-over from an old test harness design, the comment even
refers to automake. Calling skip() cannot do anything good anymore,
because it wouldn't print the skips in the TAP report, so it would
probably be considered a failure.
Delete this unused and nowadays incorrect function, so it doesn't
confuse people.
Signed-off-by: Pekka Paalanen <pekka.paalanen@collabora.com>
Store the pointer serial for events that provide one, so that it can be
used by tests to send requests that require it (e.g., setting the cursor
surface).
Signed-off-by: Alexandros Frantzis <alexandros.frantzis@collabora.com>
expect_protocol_error() ensures that the connection has failed in the
expected way. To satisfy ASan leak detection, we still need to tear down
everything, including call client_destroy().
client_destroy() needs to check that tear-down does not cause protocol
errors, so it does one last roundtrip that checks that is succeeds. But
if the connection is already down in an expected way, this roundtrip
cannot succeed and must be skipped.
Also moves the roundtrip under 'if (wl_display)', assuming the 'if' is
there for a reason - but obviously that reason was never used as it
would have crashed.
Signed-off-by: Pekka Paalanen <pekka.paalanen@collabora.com>
This refactors a new function verify_image() out of
verify_screen_content().
verify_image() will be useful with a test that verifies a screenshot
against a reference image but also wants to do additional testing on the
screenshot.
Signed-off-by: Pekka Paalanen <pekka.paalanen@collabora.com>
Const has documentary value saying the code will not modify the
parameter contents. Everything that can be const, should be const.
Signed-off-by: Pekka Paalanen <pekka.paalanen@collabora.com>
Until now we had two different protocol extensions: one for the
test suite screenshooter and other for the screenshooter client.
As they are identical and this won't change, make the test suite
use the same protocol that the screenshooter client uses.
Besides the cleanup, this change will also allow us to use the
DRM writeback screenshooter in the test suite, as the test suite
implementation is hardcoded to use a renderer based screenshooter.
Signed-off-by: Leandro Ribeiro <leandro.ribeiro@collabora.com>
The 'struct test' has a field 'int buffer_copy_done', but it
is in a fact a boolean. Change it to 'bool buffer_copy_done'.
Signed-off-by: Leandro Ribeiro <leandro.ribeiro@collabora.com>
The wl_drm protocol is not being used by the test client. So
remove 'bool has_wl_drm' from 'struct client' and also the
branch that initializes this variable in handle_global().
Signed-off-by: Leandro Ribeiro <leandro.ribeiro@collabora.com>
There will be a new test program using viewports and would like to share this
bit of code.
There are two behavioral changes:
- Compositor wp_viewporter interface version is no longer checked.
- client_create_viewport() does not leak the viewporter object.
test_viewporter_double_create needs to call bind_to_singleton_global() itself
so that the viewporter object still exists when the error event arrives.
Otherwise error verification fails.
Signed-off-by: Pekka Paalanen <pekka.paalanen@collabora.com>
This goes through all output transforms with two different buffer transforms
and verifies the visual output against reference images.
This commit introduces a new test input image 'basic-test-card.png'. It is a
small image with deliberately odd and indivisible dimensions to provoke bad
assumptions about image sizes. It contains red, green and blue areas which are
actually text that makes it very obvious if you have e.g. color channels
swapped. It has a white thick circle to highlight aspect ratio issues, and an
orange cross to show a mixed color. The white border is for contrast and a 1px
wide detail. The whole design makes it clear if the image happens to be rotated
or flipped in any way.
The image has one pixel wide transparent border so that bilinear sampling
filter near the edges of the image would produce the same colors with both
Pixman- and GL-renderers which handle the out-of-image samples fundamentally
differently: Pixman assumes (0, 0, 0, 0) samples outside of the image, while
GL-renderer clamps sample coordinates to the edge essentially repeating the
edge pixels.
It would have been "easy" to create a full matrix of
every output scale & transform x every buffer scale & transform, but that
would have resulted in 2 renderers * 8 output transforms * 3 output scales *
8 buffer transforms * 3 buffer scales = 1152 test cases that would have all
ran strictly serially because our test harness has no parallelism inside one
test program. That would have been slow to run, and need a lot more reference
images too.
Instead, I chose to iterate separately through all output scales & transforms
(this patch) and all buffer scales & transforms (next patch). This limits the
number of test cases in this patch to 56, and allows the two test programs to
run in parallel.
I did not even pick all possible scale & transform combinations here, but just
what I think is a representative sub-set to hopefully exercise all the code
paths.
https://gitlab.freedesktop.org/wayland/weston/issues/52
Signed-off-by: Pekka Paalanen <pekka.paalanen@collabora.com>
It turns out that if the client is not explicitly destroyed, it will remain
connected until the compositor shuts down because there is no more a client
process that would terminate.
Usually this is not a problem, but if a test file has multiple screenshooting
tests, the windows from earlier tests in the file will remain on screen. That
is not wanted, hence implement client destruction.
To properly destroy a client, we also need a list of outputs. They used to be
simply leaked. This does not fix wl_registry.global_remove for wl_outputs, that
is left for a time when a test will actually need that.
This patch makes only ivi-shell-app test use the new client_destroy() to show
that it actually works. The added log scopes prove it: destroy requests get
sent. Sprinkling client_destroy() around in all other tests is left for a time
when it is actually necessary.
ivi-shell-app is a nicely simple test doing little else, hence I picked it.
Signed-off-by: Pekka Paalanen <pekka.paalanen@collabora.com>
The old name felt too... short.
The return type is changed to bool; fits better for a success/failure.
Signed-off-by: Pekka Paalanen <pekka.paalanen@collabora.com>
This will be useful in more tests.
No changes to the code, aside from dropping one 'static'.
Copyright 2017 is taken from git-blame of the moved code.
Signed-off-by: Pekka Paalanen <pekka.paalanen@collabora.com>
The fuzzy range will be used with GL-renderer testing, as it may produce
slightly different images than Pixman-renderer yet still correct results.
Such allowed differences are due to different rounding.
Signed-off-by: Pekka Paalanen <pekka.paalanen@collabora.com>
Introduce helper test code to implement the client side of the
input_timestamps_unstable_v1 protocol. This helper will be used in
upcoming commits to test the server side implementation of the protocol
in libweston.
The input_timestamps_unstable_v1 protocol was introduced in version 1.13
of wayland-protocols, so this commit updates the version dependency in
configure.ac accordingly.
Signed-off-by: Alexandros Frantzis <alexandros.frantzis@collabora.com>
Reviewed-by: Pekka Paalanen <pekka.paalanen@collabora.co.uk>
The current test client code waits for all wl_seat globals to arrive
before checking them and deciding which one is the test seat global to
use for the input object. Test code that needs to add/remove test seats
would have to call the client_set_input() function for any seat changes
to take effect. Although we could allow this by making
client_set_input() public, we would be exposing unecessary
implementation details.
This commit applies any seat changes immediately upon arrival of the
seat name, freeing test code from needing to call extra functions like
client_set_input(). To achieve this the call to input_data_devices() is
moved from client_set_input() to the seat name event handler.
This commit also moves the check that all seats have names to an
explicit test. To support this test, inputs corresponding to non-test
seats are not destroyed (unless their seat global is removed), as
was previously the case.
Signed-off-by: Alexandros Frantzis <alexandros.frantzis@collabora.com>
Reviewed-by: Pekka Paalanen <pekka.paalanen@collabora.co.uk>
The current test client code completely ignores removal of globals.
This commit updates the code to properly handle removal of globals in
general, and of seat globals in particular. This ensures that the test
client objects are in sync with the server and any relevant resources
are released accordingly.
This update will be used by upcoming tests to check that seat removal
and re-addition is working properly.
Signed-off-by: Alexandros Frantzis <alexandros.frantzis@collabora.com>
Reviewed-by: Pekka Paalanen <pekka.paalanen@collabora.co.uk>
Add test to verify the server correctly emits pointer axis events. This
requires updating the weston-test protocol with a new request for
pointer axis events.
Signed-off-by: Alexandros Frantzis <alexandros.frantzis@collabora.com>
Reviewed-by: Pekka Paalanen <pekka.paalanen@collabora.co.uk>
Add test to verify that the server correctly sets the timestamps of
touch events. This requires updating the weston-test protocol with a new
request for touch events.
Signed-off-by: Alexandros Frantzis <alexandros.frantzis@collabora.com>
Reviewed-by: Pekka Paalanen <pekka.paalanen@collabora.co.uk>
Add test to verify that the server correctly sets the timestamps of
keyboard key events. This requires updating the weston-test protocol to
support passing key event timestamps.
simple_keyboard_test now uses the create_client_with_keyboard_focus()
helper function which changes the initial state of the surface to be
focused. This leads to one additional iteration of the test loop when
starting, during which the surface is deactivated, i.e., loses focus.
After this initial iteration the test continues as before.
Furthermore, simple_keyboard_test now uses the send_key() helper
function which performs a roundtrip internally. To account for this, the
client_roundtrip() function is now directly called in the loop only when
it is still required, i.e., when deactivating the surface.
Signed-off-by: Alexandros Frantzis <alexandros.frantzis@collabora.com>
Reviewed-by: Pekka Paalanen <pekka.paalanen@collabora.co.uk>
Enhance the existing pointer motion and button event tests to
additionally verify the event timestamps. This requires updating the
weston-test protocol to support passing motion and button event
timestamps.
Signed-off-by: Alexandros Frantzis <alexandros.frantzis@collabora.com>
Reviewed-by: Pekka Paalanen <pekka.paalanen@collabora.co.uk>
This doesn't attach a buffer to the surface. This is needed for the
next commit, where we have a test case with a surface that doesn't
have a buffer attached.
Signed-off-by: Emilio Pozuelo Monfort <emilio.pozuelo@collabora.co.uk>
Reviewed-by: Pekka Paalanen <pekka.paalanen@collabora.co.uk>
Reviewed-by: Daniel Stone <daniels@collabora.com>
buffer-count was introduced in line with a Mesa change which forced
an earlier block on frame events to try to enforce double-buffering
where available.
The Mesa change has since been reverted (Mesa commit 9ca6711faa), as
this had unpleasant interactions with buffer_age in particular, so this
test is no longer valid.
Additionally, it only worked on backends which initialised EGL (not
headless-backend, where tests generally run), which can be flaky due to
initialisation races. Not only that, but on the DRM backend, we can
legitimately enter triple-buffering due to promoting the surface to a
hardware plane, skipping GPU composition.
In light of all this, just remove the test.
Signed-off-by: Daniel Stone <daniels@collabora.com>
Reviewed-by: Derek Foreman <derekf@osg.samsung.com>
Useful for pointing out where the image comparisons fail.
Internal-screenshot-test is modified to save the visualization if the
test fails.
Signed-off-by: Pekka Paalanen <pekka.paalanen@collabora.co.uk>
Reviewed-by: Daniel Stone <daniels@collabora.com>
Screenshooting does not involve creating a wl_surface, so using struct
surface is superfluous.
Return a struct buffer instead. It could have been just a
pixman_image_t, but setting up proper destruction would be a bit more
work. Should not hurt to keep the wl_buffer around until the user is
ready to free the image.
Signed-off-by: Pekka Paalanen <pekka.paalanen@collabora.co.uk>
Reviewed-by: Daniel Stone <daniels@collabora.com>
This rewrites write_surface_as_png() into write_image_as_png(), which
operates on a pixman_image_t instead of a struct surface.
This is part of the migration to use pixman_image_t everywhere without
superfluous parameters/members.
Now the image saving handles more than just ARGB32 format, presumably.
At least it does not assume everything is always ARGB32.
Signed-off-by: Pekka Paalanen <pekka.paalanen@collabora.co.uk>
Reviewed-by: Daniel Stone <daniels@collabora.com>
This rewrites load_surface_from_png() to load_image_from_png(), to
return a pixman_image_t instead of a struct surface.
A loaded image has no need for wl_buffer or wl_surface or any of the
associated attributes. This is part of unifying to make everything use
pixman_image_t.
cairo_surface_flush() is added, because Cairo documentation for
cairo_image_surface_get_data() says you have to flush after drawing,
before using the data. It is unclear if loading a PNG counts as drawing,
so stay on the safe side.
load_image_from_png() now pays attention to the pixel format returned by
Cairo, which seems to come out as CAIRO_FORMAT_RGB24 in
internal-screenshot-test, not as CAIRO_FORMAT_ARGB32 as expected. I do
not know if Cairo actually guarantees the x8/a8 channel to be 0xff for
RGB24, but better to not trust it. Therefore the image is explicitly
converted to a8r8g8b8 as needed. This also adds support for loading A8
and RGB16_565 images, provided that Cairo delivers them.
The cairo surface is now wrapped directly into a pixman_image_t. If the
pixel format conversion is not needed, this eliminates a copy of the
image data. The Cairo surface will get automatically destroyed with the
Pixman image.
Signed-off-by: Pekka Paalanen <pekka.paalanen@collabora.co.uk>
Reviewed-by: Daniel Stone <daniels@collabora.com>
check_surfaces_geometry() is removed as it was not used by anything, and
unlikely would be.
check_surfaces_equal() is merged into check_surfaces_match_in_clip(),
passing a NULL clip means to compare whole images.
check_surfaces_match_in_clip() is converted to work on pixman_image_t
instead of struct surface. The function is only concerned about
comparing images in memory, and does not care about a wl_buffer or a
wl_surface.
The verbosity of image comparisons is greatly reduced. An image mismatch
no longer prints a flood of raw pixel values. This will be replaced
later with a function writing out an error image instead.
Degenerate comparisons are no longer accepted, be that clip outside
images or zero area. Those are an indication of a programmer error.
The pixel format assumptions are made more visible in the code.
A new internal helper image_check_get_roi() computes and verifies the
area to be compared. Image iterator helper makes it simpler to write
manual pixel-poking loops.
Signed-off-by: Pekka Paalanen <pekka.paalanen@collabora.co.uk>
Reviewed-by: Daniel Stone <daniels@collabora.com>
No users remain outside the file. This will allow to fix the assumptions
in the function.
Signed-off-by: Pekka Paalanen <pekka.paalanen@collabora.co.uk>
Reviewed-by: Daniel Stone <daniels@collabora.com>
We are growing more tests that need to handle buffers, both just images
and wl_buffers. Particularly the screenshooting facility needs these.
Currently everything is in struct surface, which contains more than we
need. It is a bit messy.
Create a new struct buffer to encapsulate the image representation, the
wl_buffer, and enough information to tear it all down (munmap) so we
don't have to leak everything. Some tests might start doing things in
loops, and leaking would accumulate.
Instead of inventing our own image representation, use pixman_image_t.
It is a well-tested library worth using, and we already rely on it in
other places.
This makes the tests depend on Pixman, which requires the fix for
building buffer-count, which would otherwise not find pixman.h.
The new create_shm_buffer_a8r8g8b8() creates an image with an explicit
format, and pixman_image_t keeps track of it. And stride and size and
data. This implementation is still a little hacky due to calling
create_shm_buffer().
A very new thing is buffer_destroy(). Previously we didn't really free
any buffers. It is not a problem when the process will exit soon anyway,
but it may become a problem if tests start iterating things.
Manual memset() on a image is converted to a pixman action, just to show
how to do it properly with pixman.
Stride and pixel format assumptions still linger all around, but those
are for another patch.
Signed-off-by: Pekka Paalanen <pekka.paalanen@collabora.co.uk>
Reviewed-by: Daniel Stone <daniels@collabora.com>
Direct fail_on_null calls now produce output like:
[weston-info] clients/weston-info.c:714: out of memory
xmalloc, et al produce output on failure like:
[weston-info] out of memory (-1)
Signed-off-by: Bryce Harrington <bryce@osg.samsung.com>
Reviewed-by: Pekka Paalanen <pekka.paalanen@collabora.co.uk>
These routines provide test cases an ability to capture screen images
for rendering verification.
This commit is a no-change refactoring, except for making the routines
non-static. Makefile rules are also updated; most notably, this links
test clients against the cairo libraries now.
v2: Fix pointer code styling, suggested in review
Signed-off-by: Bryce Harrington <bryce@osg.samsung.com>
Acked-by: Pekka Paalanen <pekka.paalanen@collabora.co.uk>
Reviewed-by: Derek Foreman <derekf@osg.samsung.com>
This seems like a good idea for consistency that the protocol header
is included for any protocols used by the code. This also means the
code will compile with headers generated by wayland-scanner -c.
Fixed to use angle brackets.
Reviewed-by: Daniel Stone <daniels@collabora.com>
Minor refactoring to simplify initial sanity checks of surfaces.
Conceivably useful for other basic checking.
Signed-off-by: Bryce Harrington <bryce@osg.samsung.com>
Reviewed-By: Derek Foreman <derekf@osg.samsung.com>
Introduce helper routines for testing surfaces against specific
conditions. These allow tests to validate screen captures as displaying
the correct rendering results.
Signed-off-by: Bryce Harrington <bryce@osg.samsung.com>
Reviewed-By: Derek Foreman <derekf@osg.samsung.com>
A more descriptive name to not be confused with create_client().
v2: Rebased: fix also devices-test.c.
Signed-off-by: Pekka Paalanen <pekka.paalanen@collabora.co.uk>
Reviewed-by: Marek Chalupa <mchqwerty@gmail.com>
Introduce a new helper create_client(), which creates and initializes
the client struct, but does not create a wl_surface.
This will be useful for ivi-shell tests.
v2: Rebased, and restored the dependency to weston-test.so, since seat
handling requires it.
Signed-off-by: Pekka Paalanen <pekka.paalanen@collabora.co.uk>
Reviewed-by: Marek Chalupa <mchqwerty@gmail.com>
Let the client bind to wl_touch. Since we have our own seat,
we know that the compositor will have wl_touch capability.
v2: rebased due to changes in previous commit
Signed-off-by: Marek Chalupa <mchqwerty@gmail.com>
Reviewed-by: Pekka Paalanen <pekka.paalanen@collabora.co.uk>
When running on different backends, we don't know what devices
the backend provides. Create new seat for tests that contains
everything what we need. This is also first step in adding
touch support for tests.
v2: do not add devices in wl_seat.name event. Collect first
all wl_seats and then pick the one that we need and
destroy the rest. The effect is the same, but this code
is better understandable.
Signed-off-by: Marek Chalupa <mchqwerty@gmail.com>
Reviewed-by: Pekka Paalanen <pekka.paalanen@collabora.co.uk>
We used hard-coded version 1 for all globals. For testing
newer methods and events we need use the current version
of global. This patch fixes this and adds missing
event handlers (for the events that were added in
versions > 1)
Signed-off-by: Marek Chalupa <mchqwerty@gmail.com>
Reviewed-by: Pekka Paalanen <pekka.paalanen@collabora.co.uk>
wayland-test isn't and will never be wayland protocol, it's weston internal.
Renamed wayland-test to weston-test, and wl_test to weston_test.
Also added a Big Fat Warning to the description of weston_test to try to
keep people from thinking it's a good idea to use some of these functions
outside of testing.
Signed-off-by: Derek Foreman <derekf@osg.samsung.com>
Acked-by: Bryce Harrington <bryce@osg.samsung.com>
Reviewed-by: Bryce Harrington <bryce@osg.samsung.com>
This skips the test when running on the headless backend.
Signed-off-by: Derek Foreman <derekf@osg.samsung.com>
Reviewed-by: Bryce Harrington <bryce@osg.samsung.com>
(Presumably) Reviewed-by: Daniel Stone <daniel@fooishbar.org>
Pekka Paalanen
Alexandros Frantzis
Leandro Ribeiro
Emilio Pozuelo Monfort
Daniel Stone
Jussi Kukkonen
Pekka Paalanen
Bryce Harrington
Bill Spitzak
Marek Chalupa
Derek Foreman