This runs the coverage tools to produce HTML pages listing the code lines /
functions / branches hit/totalled by the test suite.
Nowadays Gitlab has some Cobertura support itself:
https://docs.gitlab.com/ee/user/project/merge_requests/test_coverage_visualization.html
lcov is needed for the HTML report, gcovr is needed for the Cobertura
report. 'ninja clean' must be removed, otherwise it deletes the coverage
files before they are analysed.
Seeing the test suite code coverage is really interesting. It can guide
designing tests. If Gitlab MRs show the coverage in diff view, it shows
if new code actually gets executed in CI.
Signed-off-by: Pekka Paalanen <pekka.paalanen@collabora.com>
Doing it when the surface is being added would cause clients that
wait for frame callbacks to wait indefinitely as the surface being
activated is not yet, committed.
Fixes: #473
Signed-off-by: Marius Vlad <marius.vlad@collabora.com>
This adds a heuristic for freeing shader programs that have not been
needed for a while. The intention is to stop Weston accumulating shader
programs indefinitely, especially in the future when color management
will explode the number of possible different shader programs.
Shader programs that have not been used in the past minute are freed,
except always keep the ten most recently used shader programs anyway.
The former rule is to ensure we keep shader programs that are actively
used regardless of how many. The latter rule is to prevent freeing too
many shader programs after Weston has been idle for a long time and then
repaints just a small area. Many of the shader programs could still be
relevant even though not needed in the first repaint after idle.
The numbers ten and one minute in the above are arbitrary and not based
on anything.
These heuristics are simpler to implement than e.g. views taking
references on shader programs. Expiry by time allows shader programs to
survive a while even after their last user is gone, with the hope of
being re-used soon. Tracking actual use instead of references also
adapts to what is actually visible rather than what merely exists.
Keeping the shader list in most recently used order might also make
gl_renderer_get_program() more efficient on average.
last_repaint_start time is used for shader timestamp to avoid calling
clock_gettime() more often. Adding that variable is an ABI break, but
libweston major has already been bumped to 10 since last release.
Signed-off-by: Pekka Paalanen <pekka.paalanen@collabora.com>
This is useful for seeing that the shader program garbage collection
works in a future patch.
Signed-off-by: Pekka Paalanen <pekka.paalanen@collabora.com>
One more thing is coming to need this, so add the compositor pointer and
migrate existing places to use it where it simplifies things.
Signed-off-by: Pekka Paalanen <pekka.paalanen@collabora.com>
I have verified that the conversion here follows ITU-R BT.601 except for
the offsets 16/256 and 128/256 which should be 16/255 and 128/255
respectively.
I used to following octave script to verify this:
rf = 0.299;
gf = 0.587;
bf = 0.114;
crdiv = 1.402;
cbdiv = 1.772;
M = [ rf, gf, bf ;
-rf / cbdiv, -gf / cbdiv, (1 - bf) / cbdiv;
(1 - rf) / crdiv, -gf / crdiv, -bf / crdiv ];
YCbCr = [ 'Y'; 'Cb'; 'Cr' ];
RGB = [ 'R'; 'G'; 'B' ];
eq = [ ' '; '='; ' ' ];
l = [ ' [ '; ' [ '; ' [ ' ];
r = [ ' ] '; ' ] '; ' ] ' ];
mat = [
sprintf('%9f %9f %9f', M(1,:));
sprintf('%9f %9f %9f', M(2,:));
sprintf('%9f %9f %9f', M(3,:));
];
[ l YCbCr r eq l mat r l RGB r ]
R = inv(M);
mat = [
sprintf('%9f %9f %9f', R(1,:));
sprintf('%9f %9f %9f', R(2,:));
sprintf('%9f %9f %9f', R(3,:));
];
[ l RGB r eq l mat r l YCbCr r ]
[ R(:,1), R(:,2:3) .* (255/224) ]
The final matrix printed is what the shader uses down to +/- one digit,
so at least 7 correct decimals.
Signed-off-by: Pekka Paalanen <pekka.paalanen@collabora.com>
Sampling input texture has nothing to do with view alpha. This clarifies
the code structure.
Signed-off-by: Pekka Paalanen <pekka.paalanen@collabora.com>
Reading the input texture is just one part of the future color pipeline,
so separate it into a function of its own. This makes it easier to add
more steps to the pipeline, and shows the green tint is separate as
well.
Making use of early returns, reducing the if-else ladder should help
with readability. Sharing the call to yuva2rgba() likewise.
Setting yuva.w = alpha is not shared though, in case support for AYUV
format might be added in the future.
Signed-off-by: Pekka Paalanen <pekka.paalanen@collabora.com>
Do not call texture2D() in the shader when we already have the result.
Simpler code, maybe even a little bit faster?
Suggested-by: Harish Krupo <harishkrupo@gmail.com>
Signed-off-by: Pekka Paalanen <pekka.paalanen@collabora.com>
These same magic constants were used in all cases, so move them into a
common place.
While we are touching all these lines, also change from the four floats
into a vec4. This allows further clean-up in the next patch.
This makes the code easier to read.
Behavior and results are unchanged.
Signed-off-by: Pekka Paalanen <pekka.paalanen@collabora.com>
Mathematically the result is the same, while multiplying RGB with alpha
is easier to understand as correct than the earlier form.
Signed-off-by: Pekka Paalanen <pekka.paalanen@collabora.com>
A more unique name is easier to grep for. Using 'color' as a local
variable might be useful in the future.
Signed-off-by: Pekka Paalanen <pekka.paalanen@collabora.com>
This will hit the XYUV shader variant in GL-renderer that was not
covered in the test suite before.
Signed-off-by: Pekka Paalanen <pekka.paalanen@collabora.com>
This support is added so that the XYUV shader variant can be tested with
wl_shm from the test suite.
Libwayland version requirement is bumped to get WL_SHM_FORMAT_XYUV8888.
Libwayland is bumped to 1.18 too in the CI image. libwayland-dev package
is dropped, because we build wayland anyway.
Signed-off-by: Pekka Paalanen <pekka.paalanen@collabora.com>
This header is for sharing fallback definitions for drm_fourcc.h. A new
test in tests/yuv-buffer-test.c is going to be needing XYUV8888 format,
and more new formats will be expected with HDR supports.
Share these fallback definitions in one place instead of copying them
all over.
All users of drm_fourcc.h are converted to include weston-drm-fourcc.h
instead for consistency: have the same definitions available everywhere.
Signed-off-by: Pekka Paalanen <pekka.paalanen@collabora.com>
MOD_INVALID came with libdrm 2.4.83 and MOD_LINEAR came with libdrm
2.4.82. libweston unconditionally depends on libdrm >= 2.4.95, so the
fallback is not necessary.
Since linux-dmabuf.h itself has no use for these and also forgets to
include drm_fourcc.h, .c files including drm_fourcc.h after this header
would trigger compiler warnings.
linux-dmabuf.c does need these, so add the proper include.
Signed-off-by: Pekka Paalanen <pekka.paalanen@collabora.com>
These were introduced in libdrm 2.4.68, commit
268ae7cae5afd76462c3ef14ed9021a2d40c2e57. Weston unconditionally
requires libdrm >= 2.4.95, so these fallback definitions are
unnecessary now.
Signed-off-by: Pekka Paalanen <pekka.paalanen@collabora.com>
Extend the existing output-damage test to test
blit_shadow_to_output() specifically. This function had problems
originally, so make sure they can't reappear.
The added quirk is explained in the test.
An additional check of the quirk in gl_renderer_output_create() ensures
that the shadow framebuffer is really used. The test could false-pass if
the shadow is not used.
Signed-off-by: Pekka Paalanen <pekka.paalanen@collabora.com>
The version we used does not advertise GL_EXT_color_buffer_half_float
with llvmpipe even though the functionality seems to work.
If the extension is not advertised, the future commit
"tests: extend output-damage to GL shadow framebuffer"
will result in a test failure.
Upgrade Mesa, this gets the extension advertised and the test is happy.
Signed-off-by: Pekka Paalanen <pekka.paalanen@collabora.com>
This adds an output section option use-renderer-shadow in weston.ini.
This option is only recognized with headless and DRM backends, because
it requires GL-renderer and does not support resizing outputs.
The option is called use-renderer-shadow because this is what it does
right now. In the future the same setting will be used to turn on more
complex image processing when operational pieces required for color
management land. Once color management is implemented, this option is
expected to be removed. This option allows developer testing of features
to be used to implement color management.
This is a rewrite of "weston.ini: introduce use-shadow-fbo in output
config" by Harish Krupo. The main.c code is structured differently, the
weston.ini option is renamed, and the man page paragraph is moved to
weston.ini.man with different content.
Cc: Harish Krupo <harishkrupo@gmail.com>
Signed-off-by: Pekka Paalanen <pekka.paalanen@collabora.com>
Proper color management will need blending done with linear light pixel
values, that is, EOTF applied before blending, and then inverse-EOTF
applied for scanout after blending. The simplest way to set that up is
to use an intemediate framebuffer a.k.a shadow buffer containing the
composited image in linear light values, then blit from that to the
actual framebuffer.
This patch implements the shadow buffer, but the linear light
blending is left for another patch. This allows GL-renderer to turn
WESTON_CAP_COLOR_OPS on.
Half-float is chosen as the buffer format because linear light values
require more bits to encode with sufficient precision than the usual
non-linear pixel values.
v2: Use /* */ instead of // (Pekka)
Rename fbo and tex to shadow_{fbo,tex} (Pekka)
Check for OpenGLES capabilities before creating
shadow_{tex,fbo} (Pekka)
Signed-off-by: Harish Krupo <harishkrupo@gmail.com>
v3: Rebased.
Simplified GL version checks (Sebastian)
Apply changes from "libweston: add color ops cap and bool renderer
shadow buffer"
Renamed supports_half_float_texture to has_gl_half_float to
follow the existing naming pattern.
Introduce gl_renderer_create_shadow_16f().
Undo moving of glViewport() call.
Replace half_float_texture_enabled with shadow_exists().
Introduce struct gl_output_state_shadow.
Assert no resizing with shadow.
Fix triangle fan debug.
Rename repaint_from_texture() to blit_shadow_to_output().
Rewrite commit message because linear light blending is not
implemented in this patch.
Fix blit_shadow_to_output() for scaled/transformed outputs and
remove redundant code.
Fix has_gl_half_float determination.
v4: Disable blending in blit_shadow. (Daniel)
Port to gl_renderer_get_program().
Make a generic fbo-texture struct with parameterized format. (Daniel)
Change has_gl_half_float into gl_half_float_type.
Signed-off-by: Pekka Paalanen <pekka.paalanen@collabora.com>
Converting a region from global coordinates to output pixel coordinates
will become useful in GL-renderer soon, so move this function to be
shared. It is tricky to reinvent.
Signed-off-by: Pekka Paalanen <pekka.paalanen@collabora.com>
This adds the libweston capability bit for "color operations" which
refers to a renderer's support for operations needed for color
management. GL-renderer will grow the support while Pixman-renderer will
not, which is why the cap is needed.
To make an example use of the cap, this also adds new API:
weston_output_set_renderer_shadow_buffer(). This is a temporary API to
enable future experimental features. The first such feature will be the
renderer internal shadow buffer, the boolean variable for it taken from
Harish Krupo's "weston.ini: introduce use-shadow-fbo in output config".
Obviously this patch does not implement the renderer shadow buffer. No
renderer sets WESTON_CAP_COLOR_OPS yet so trying to enable it will fail.
The documentation here is deliberately vague, because the bits needed
for color management will come in trickling for a long time until we can
call it color management in any sense. Until then, the temporary API
shall remain, perhaps poorly named.
Cc: Harish Krupo <harishkrupo@gmail.com>
Signed-off-by: Pekka Paalanen <pekka.paalanen@collabora.com>
This helps accounting how many shaders live in the cache, what the
shader source code is, and when shaders are compiled.
Signed-off-by: Harish Krupo <harishkrupo@gmail.com>
v2: Resolved rebase conflicts.
Put shader_scope in struct gl_renderer, remove struct
gl_shader_generator.
Wrote commit message.
Rebased for "gl-renderer: rewrite fragment shaders" which completely
changed how shader sources are generated.
Added cache statistics to debug output on subscribe.
Signed-off-by: Pekka Paalanen <pekka.paalanen@collabora.com>
Various functions leave the current active texture as whatever. The
functions touched in this commit forgot to reset the active texture to
slot 0 before binding their textures. If not explicitly unbound, this
could leave textures lingering in unused slots, perhaps. Not sure if
that could cause any harm, but for consistency's sake, always use slot 0
when not multitexturing.
Found by code inspection.
Signed-off-by: Pekka Paalanen <pekka.paalanen@collabora.com>
If shader compiling on demand fails, then rather than using whatever
random shader happens to be current, use an explicit fallback shader
painting stuff brown.
The color is chosen dim enough to hopefully not cause problems even in
a HDR setting as it will be written verbatim into the fb/shadow.
This also prevents NULL dereference on shader->key.variant in
draw_view().
One way to test this shader is to hack fragment.glsl:
#if DEF_VARIANT == SHADER_VARIANT_EXTERNAL
#extension GL_OES_EGL_image_external : require
+#error haa haa
#endif
and then run e.g. weston-simple-dmabuf-v4l -f YUYV
with vivid kernel module loaded. This worked on Intel.
Signed-off-by: Pekka Paalanen <pekka.paalanen@collabora.com>
If we are trying to use a NULL shader, it is likely that the shader
compilation failed for some reason. Since we are trying this for a view,
the failure was probably triggered by a client. If there is a client,
get rid of it by sending it a protocol error. Hopefully the compositor
can then continue operation after a glitch on screen.
Signed-off-by: Pekka Paalanen <pekka.paalanen@collabora.com>
This patch modifies the shader generation code so that the shaders are
stitched together based on the requirement instead of creating them
during initialization. This is necessary for HDR use cases where each
surface would have different properties based on which different
de-gamma or tone mapping or gamma shaders are stitched together.
v2: Use /* */ instead of // (Pekka)
Move shader strings to gl-shaders.c file (Pekka)
Remove Makefile.am changes (Pekka)
Use a struct instead of uint32_t for storing requirements (Pekka)
Clean up shader list on destroy (Pekka)
Rename shader_release -> shader_destroy (Pekka)
Move shader creation/deletion into gl-shaders.c (Pekka)
Use create_shaders's multi string capbility instead of
concatenating (Pekka)
v3: Add length check when adding shader string (Pekka)
Signed-off-by: Harish Krupo <harishkrupo@gmail.com>
v4: Rebased, PROTECTION_MODE_ENFORCED converted.
Dropped unnecessary { }.
Ported setup_censor_overrides().
Split out moving code into gl-shaders.c.
Changed to follow "gl-renderer: rewrite fragment shaders",
no more shader source stitching.
Added SHADER_VARIANT_XYUV.
Const'fy function arguments.
Added gl_shader_requirements_cmp() and moved the early return in
use_gl_program().
Moved use_gl_program() before first use in file.
Split solid shader requirements by use case: requirements_censor and
requirements_triangle_fan.
Simplified fragment_debug_binding() since no need to force anything.
Ensure struct gl_shader_requirements has no padding. This allows us
to use normal C syntax instead of memset() and memcpy() when
initializing or assigning. See also:
https://gitlab.freedesktop.org/mesa/mesa/-/issues/2071
Make it also a bitfield to squeeze the size.
v5: Move wl_list_insert() into gl_shader_create() (Daniel)
Compare variant to explicit value. (Daniel)
Change functions to gl_renderer_get_program,
gl_renderer_use_program, and
gl_renderer_use_program_with_view_uniforms.
Use local variable instead of gr->current_shader. (Daniel)
Simplified gl_renderer_get_program.
Signed-off-by: Pekka Paalanen <pekka.paalanen@collabora.com>
Do not change in setup_censor_overrides() and then put back gs->shader
in draw_view() when the shader needs to be something else than what the
surface content calls for.
This makes the logic simpler, and makes following changes simpler as
well.
Signed-off-by: Pekka Paalanen <pekka.paalanen@collabora.com>
To help debugging shader compilation errors, print the shader source the
way it was given to the GLSL compiler and with line numbers that match
the compiler error messages.
This is necessary because some snippets are added at runtime to the
beginning, the source is not only what is in the respective .glsl file.
I did look into using #line directives, but you cannot put source file
names to it, only "source string numbers" which must be an integer
expression. If we used #line, the reader would need to know that string
number 0 is the version, string 1 is the config and string number 2 is
fragment.glsl. I think that would have been too cumbersome.
Signed-off-by: Pekka Paalanen <pekka.paalanen@collabora.com>
The main goal of this patch is to improve the readability of how and
what fragment shaders are generated.
Instead of having C code that assembles each shader variant from literal
string snippets, create one big fragment shader source that has
everything in it. This relies on a GLSL compiler to optimize statically
false conditions and unused uniforms away.
Having all the fragment shader code in one file, uncluttered by C string
literal syntax, improves readability significantly. A disadvantage is
that the code is more verbose, but it allows comments much better.
The actual shader code is kept unchanged except:
- FRAGMENT_CONVERT_YUV macro is now a proper function
- GLSL version is explicitly set to 1.00 ES
- RGBA and EXTERNAL use the same path, the difference is how the sampler
is declared
Further shader code consolidation is possible, but is left for another
time.
Signed-off-by: Pekka Paalanen <pekka.paalanen@collabora.com>
This patch adds the tooling for incorporating files as C data, so that
files can be built into the binaries. The tool is in Python to avoid
adding extra dependencies like xxd.
xxd.py is copied from Mesa as-is, from commit
b729cd58d76f97f3fc04a67569535ee5ef2f5278 (master branch on 2021-01-26),
a.k.a 21.0-branchpoint-635-gb729cd58d76.
Moving the GLSL vertex shader into a separate file is not that
interesting, the purpose of this commit is to provide a simple
demonstration of the tooling. The real benefits come in a following
patch where the fragment shaders are re-written and externalized.
Signed-off-by: Pekka Paalanen <pekka.paalanen@collabora.com>
This is purely moving code as is with no changes other than making the
three functions non-static.
Originally this was part of "gl-renderer: Requirement based shader
generation" by Harish Krupo, but that patch made also big changes to the
code at the same time. Patches are easier to review when code movement
is separate from behavioral changes, therefore I introduced this patch.
Cc: Harish Krupo <harishkrupo@gmail.com>
Signed-off-by: Pekka Paalanen <pekka.paalanen@collabora.com>
The main point here is to print "GL ES %d.%d" instead of "GL ES 2"
because GL-renderer can and will use GL ES 3 features when present.
Saying it's GL ES 2 renderer is not quite true.
To print that, I need to extract major, minor from gr->gl_version and
those didn't have ready made macros yet. While writing the extraction,
make all these trivial functions, so that the compiler might warn us if
one passes e.g. negative literal numbers to gr_gl_version(). Explicit
types help keeping the bit operations safe too.
The only purpose for GR_GL_VERSION_INVALID was to fall back to version
2.0. Moving the fallback and logging into get_gl_version() makes that
macro unnecessary.
Finally, just in case GL version string contained garbage, reject
negative version numbers.
Signed-off-by: Pekka Paalanen <pekka.paalanen@collabora.com>
This test ensures that client submitted damage goes to the screen
correctly, regardless of output scale or transform.
The added quirk is explained in the test that uses it.
Signed-off-by: Pekka Paalanen <pekka.paalanen@collabora.com>
For a pbuffer EGLSurface, assume that EGL swap behavior is "preserved"
which means buffer age is always 1 (after the very first
eglSwapBuffers()).
EGL pbuffers are always single-buffered.
Mesa EGL Surfaceless platform does not seem to expose EGL_EXT_buffer_age
that could have told us the same. Hence all repaints to pbuffer surfaces
used to need to repaint the whole output always. This patch makes
repainting only the latest damage possible.
Repainting only the latest damage is required for a future test on
output damage regions: "tests: add output damage test".
Technically, setting buffer_age to 1 is not correct before the very
first eglSwapBuffers(), but to keep the code simpler I chose to rely on
a newly enabled output always having full damage anyway.
Checking that EGL_SWAP_BEHAVIOR is EGL_BUFFER_PRESERVED would do too.
Unfortunately, Mesa seems to return EGL_BUFFER_DESTROYED, so I cannot
fail the headless-backend in that check. Even so, the output damage test
actually succeeds.
See also: https://gitlab.freedesktop.org/mesa/mesa/-/issues/4278
Signed-off-by: Pekka Paalanen <pekka.paalanen@collabora.com>
If full output damage is forced every time a screenshot is taken, the
test suite cannot take screenshots of damage as that would depend on
repainting only the damaged area.
Stop damaging the output and instead just schedule a repaint so the
screenshot can be taken. This is safe because:
- if any views were promoted to hw planes previously,
weston_output_disable_planes_incr() would force them to be demoted at
assing_planes() time causing damage via weston_view_move_to_plane()
- even when hardware primary plane has no damage, DRM-backend will not
skip calling to the renderer after commit
"drm-backend: do not skip renderer if capturing screen".
Signed-off-by: Pekka Paalanen <pekka.paalanen@collabora.com>
The renderer must be called for any pending screen capture to complete.
Previously this was guaranteed by weston_screenshooter_shoot() forcing
full output damage, so damage was never empty. If the future,
screenshooting stops inflicting damage, and the damage on the primary
plane even after disabling hardware planes may be empty.
This patch ensures that screenshots do not get stuck until damage
occurs.
Signed-off-by: Pekka Paalanen <pekka.paalanen@collabora.com>
I am working on adding a test to ensure Weston repaints damage
correctly, where I rely on Weston repainting exactly and only the damage
submitted by a client. That means I have to stop screenshooting from
damaging everything automatically. Doing that, I noticed that
screenshots on DRM-backend could theoretically get stuck if I do that.
So test for it.
Signed-off-by: Pekka Paalanen <pekka.paalanen@collabora.com>
This was prompted by a recent discussion on #dri-devel IRC channel,
where the use of pseudonyms to maintain anonymity was said to be a
normal and accepted practice. See:
https://people.freedesktop.org/~cbrill/dri-log/index.php?channel=dri-devel&highlight_names=&date=2021-02-09
and look for the discussion around pq and Lyude.
Until then, I was hesitant to accept Signed-off-by's with names that
looked very much not a real name.
Clarify our documentation that pseudonyms are ok.
Note, that is not what the Linux kernel documentation says today in
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/Documentation/process/submitting-patches.rst
which says that pseudonyms are not ok. According to the discussion, this
should probably be fixed in the kernel too.
Since we are now ok with pseudonyms in Signed-off-by, there is no reason
left to accept any patches without a Signed-off-by. This clarifies our
policy and takes the burden of case-by-case consideration away from
maintainers.
The wording about needing to use a personal email address is my
addition. The intention is to ensure a globally unique handle for a
person while that person remains anonymous, so that the Signed-off-by
cannot be mistaken for someone elses.
Signed-off-by: Pekka Paalanen <pekka.paalanen@collabora.com>
Pekka Paalanen
Vlad Zahorodnii
Marius Vlad
Harish Krupo
Maxime Roussin-Bélanger