compositor-drm: Disallow overlapping overlay planes

The scanout plane strictly stacks under all overlay planes, and the
cursor plane above. However, the stacking of overlay planes with respect
to each other is undefined.

We can control the stacking order of overlay planes with the zpos
property, though this significantly complicates plane assignment. In the
meantime, simply disallow assigning a view to an overlay, when it
overlaps another view which is already on an overlay. This ensures
stacking order is irrelevant, since the planes never intersect each
other.

Signed-off-by: Daniel Stone <daniels@collabora.com>
Reviewed-by: Pekka Paalanen <pekka.paalanen@collabora.co.uk>

libweston/compositor-drm.c

@@ -3324,6 +3324,16 @@ drm_output_propose_state(struct weston_output *output_base,
 					  &clipped_view);
 		if (pixman_region32_not_empty(&surface_overlap))
 			next_plane = primary;
+
+		/* We do not control the stacking order of overlay planes;
+		 * the scanout plane is strictly stacked bottom and the cursor
+		 * plane top, but the ordering of overlay planes with respect
+		 * to each other is undefined. Make sure we do not have two
+		 * planes overlapping each other. */
+		pixman_region32_intersect(&surface_overlap, &occluded_region,
+					  &clipped_view);
+		if (pixman_region32_not_empty(&surface_overlap))
+			next_plane = primary;
 		pixman_region32_fini(&surface_overlap);
 
 		if (next_plane == NULL)