You can not select more than 25 topics
Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
529 lines
19 KiB
529 lines
19 KiB
\documentclass{article}
|
|
\usepackage{palatino}
|
|
|
|
\author{Kristian Høgsberg\\
|
|
\texttt{krh@bitplanet.net}
|
|
}
|
|
|
|
\title{The Wayland Display Server}
|
|
|
|
\begin{document}
|
|
|
|
\maketitle
|
|
|
|
\section{Wayland Overview}
|
|
|
|
\begin{itemize}
|
|
\item wayland is a protocol for a new display server.
|
|
\item wayland is an implementation
|
|
\end{itemize}
|
|
|
|
\subsection{Replacing X11}
|
|
|
|
Over the last 10 years, a lot of functionality have slowly moved out
|
|
of the X server and into libraries or kernel drivers. It started with
|
|
freetype and fontconfig providing an alternative to the core X fonts
|
|
and direct rendering OpenGL as a graphics driver in a client side
|
|
library. Then cairo came along and provided a modern 2D rendering
|
|
library independent of X and compositing managers took over control of
|
|
the rendering of the desktop. Recently with GEM and KMS in the Linux
|
|
kernel, we can do modesetting outside X and schedule several direct
|
|
rendering clients. The end result is a highly modular graphics stack.
|
|
|
|
Wayland is a new display server building on top of all those
|
|
components. We’re trying to distill out the functionality in the X
|
|
server that is still used by the modern Linux desktop. This turns out
|
|
to be not a whole lot. Applications can allocate their own off-screen
|
|
buffers and render their window contents by themselves. In the end,
|
|
what’s needed is a way to present the resulting window surface to a
|
|
compositor and a way to receive input. This is what Wayland provides,
|
|
by piecing together the components already in the eco-system in a
|
|
slightly different way.
|
|
|
|
X will always be relevant, in the same way Fortran compilers and VRML
|
|
browsers are, but it’s time that we think about moving it out of the
|
|
critical path and provide it as an optional component for legacy
|
|
applications.
|
|
|
|
|
|
\section{Wayland protocol}
|
|
|
|
\subsection{Basic Principles}
|
|
|
|
The wayland protocol is an asynchronous object oriented protocol. All
|
|
requests are method invocations on some object. The request include
|
|
an object id that uniquely identifies an object on the server. Each
|
|
object implements an interface and the requests include an opcode that
|
|
identifies which method in the interface to invoke.
|
|
|
|
The wire protocol is determined from the C prototypes of the requests
|
|
and events. There is a straight forward mapping from the C types to
|
|
packing the bytes in the request written to the socket. It is
|
|
possible to map the events and requests to function calls in other
|
|
languages, but that hasn't been done at this point.
|
|
|
|
The server sends back events to the client, each event is emitted from
|
|
an object. Events can be error conditions. The event includes the
|
|
object id and the event opcode, from which the client can determine
|
|
the type of event. Events are generated both in repsonse to a request
|
|
(in which case the request and the event constitutes a round trip) or
|
|
spontanously when the server state changes.
|
|
|
|
\begin{itemize}
|
|
\item state is broadcast on connect, events sent out when state
|
|
change. client must listen for these changes and cache the state.
|
|
no need (or mechanism) to query server state.
|
|
|
|
\item server will broadcast presence of a number of global objects,
|
|
which in turn will broadcast their current state
|
|
\end{itemize}
|
|
|
|
\subsection{Connect Time}
|
|
|
|
\begin{itemize}
|
|
\item no fixed format connect block, the server emits a bunch of
|
|
events at connect time
|
|
\item presence events for global objects: output, compositor, input
|
|
devices
|
|
\end{itemize}
|
|
\subsection{Security and Authentication}
|
|
|
|
\begin{itemize}
|
|
\item mostly about access to underlying buffers, need new drm auth
|
|
mechanism (the grant-to ioctl idea), need to check the cmd stream?
|
|
|
|
\item getting the server socket depends on the compositor type, could
|
|
be a system wide name, through fd passing on the session dbus. or
|
|
the client is forked by the compositor and the fd is already opened.
|
|
\end{itemize}
|
|
|
|
\subsection{Creating Objects}
|
|
|
|
\begin{itemize}
|
|
\item client allocates object ID, uses range protocol
|
|
\item server tracks how many IDs are left in current range, sends new
|
|
range when client is about to run out.
|
|
\end{itemize}
|
|
|
|
\subsection{Compositor}
|
|
|
|
The compositor is a global object, advertised at connect time.
|
|
|
|
\begin{tabular}{l}
|
|
\hline
|
|
Interface \texttt{compositor} \\ \hline
|
|
Requests \\ \hline
|
|
\texttt{create\_surface(id)} \\
|
|
\texttt{commit()} \\ \hline
|
|
Events \\ \hline
|
|
\texttt{device(device)} \\
|
|
\texttt{acknowledge(key, frame)} \\
|
|
\texttt{frame(frame, time)} \\ \hline
|
|
\end{tabular}
|
|
|
|
|
|
\begin{itemize}
|
|
\item a global object
|
|
\item broadcasts drm file name, or at least a string like drm:/dev/card0
|
|
\item commit/ack/frame protocol
|
|
\end{itemize}
|
|
|
|
\subsection{Surface}
|
|
|
|
Created by the client.
|
|
|
|
\begin{tabular}{l}
|
|
\hline
|
|
Interface \texttt{surface} \\ \hline
|
|
Requests \\ \hline
|
|
\texttt{destroy()} \\
|
|
\texttt{attach()} \\
|
|
\texttt{map()} \\
|
|
\texttt{damage()} \\ \hline
|
|
Events \\ \hline
|
|
no events \\ \hline
|
|
\end{tabular}
|
|
|
|
Needs a way to set input region, opaque region.
|
|
|
|
\subsection{Input}
|
|
|
|
Represents a group of input devices, including mice, keyboards. Has a
|
|
keyboard and pointer focus. Global object. Pointer events are
|
|
delivered in both screen coordinates and surface local coordinates.
|
|
|
|
\begin{tabular}{l}
|
|
\hline
|
|
Interface \texttt{cache} \\ \hline
|
|
Requests \\ \hline
|
|
no requests \\ \hline
|
|
Events \\ \hline
|
|
\texttt{motion(x, y, sx, sy)} \\
|
|
\texttt{button(button, state, x, y, sx, sy)} \\
|
|
\texttt{key(key, state)} \\
|
|
\texttt{pointer\_focus(surface)} \\
|
|
\texttt{keyboard\_focus(surface, keys)} \\ \hline
|
|
\end{tabular}
|
|
|
|
Talk about:
|
|
|
|
\begin{itemize}
|
|
\item keyboard map, change events
|
|
\item xkb on wayland
|
|
\item multi pointer wayland
|
|
\end{itemize}
|
|
|
|
A surface can change the pointer image when the surface is the pointer
|
|
focus of the input device. Wayland doesn't automatically change the
|
|
pointer image when a pointer enters a surface, but expects the
|
|
application to set the cursor it wants in response the the motion
|
|
event. The rationale is that a client has to manage changing pointer
|
|
images for UI elements within the surface in response to motion events
|
|
anyway, so we'll make that the only mechanism for setting changing the
|
|
pointer image. If the server receives a request to set the pointer
|
|
image after the surface loses pointer focus, the request is ignored.
|
|
To the client this will look like it successfully set the pointer
|
|
image.
|
|
|
|
The compositor will revert the pointer image back to a default image
|
|
when no surface has the pointer focus for that device. Clients can
|
|
revert the pointer image back to the default image by setting a NULL
|
|
image.
|
|
|
|
What if the pointer moves from one window which has set a special
|
|
pointer image to a surface that doesn't set an image in response to
|
|
the motion event? The new surface will be stuck with the special
|
|
pointer image. We can't just revert the pointer image on leaving a
|
|
surface, since if we immediately enter a surface that sets a different
|
|
image, the image will flicker. Broken app, I suppose.
|
|
|
|
\subsection{Output}
|
|
|
|
A output is a global object, advertised at connect time or as they
|
|
come and go.
|
|
|
|
\begin{tabular}{l}
|
|
\hline
|
|
Interface \texttt{output} \\ \hline
|
|
Requests \\ \hline
|
|
no requests \\ \hline
|
|
Events \\ \hline
|
|
\texttt{geometry(width, height)} \\ \hline
|
|
\end{tabular}
|
|
|
|
\begin{itemize}
|
|
\item laid out in a big (compositor) coordinate system
|
|
\item basically xrandr over wayland
|
|
\item geometry needs position in compositor coordinate system\
|
|
\item events to advertise available modes, requests to move and change
|
|
modes
|
|
\end{itemize}
|
|
|
|
\subsection{Shared object cache}
|
|
|
|
Cache for sharing glyphs, icons, cursors across clients. Lets clients
|
|
share identical objects. The cache is a global object, advertised at
|
|
connect time.
|
|
|
|
\begin{tabular}{l}
|
|
\hline
|
|
Interface \texttt{cache} \\ \hline
|
|
Requests \\ \hline
|
|
\texttt{upload(key, visual, bo, stride, width, height)} \\ \hline
|
|
Events \\ \hline
|
|
\texttt{item(key, bo, x, y, stride)} \\
|
|
\texttt{retire(bo)} \\ \hline
|
|
\end{tabular}
|
|
|
|
\begin{itemize}
|
|
|
|
\item Upload by passing a visual, bo, stride, width, height to the
|
|
cache.
|
|
|
|
\item Upload returns a bo name, stride, and x, y location of object in
|
|
the buffer. Clients take a reference on the atlas bo.
|
|
|
|
\item Shared objects are refcounted, freed by client (when purging
|
|
glyphs from the local cache) or when a client exits.
|
|
|
|
\item Server can't delete individual items from an atlas, but it can
|
|
throw out an entire atlas bo if it becomes too sparse. The server
|
|
sends out an \texttt{retire} event when this happens, and clients
|
|
must throw away any objects from that bo and reupload. Between the
|
|
server dropping the atlas and the client receiving the retire event,
|
|
clients can still legally use the old atlas since they have a ref on
|
|
the bo.
|
|
|
|
\item cairo needs to hook into the glyph cache, and maybe also a way
|
|
to create a read-only surface based on an object form the cache
|
|
(icons).
|
|
|
|
\texttt{cairo\_wayland\_create\_cached\_surface(surface-data)}.
|
|
|
|
\end{itemize}
|
|
|
|
|
|
\subsection{Drag and Drop}
|
|
|
|
Multi-device aware. Orthogonal to rest of wayland, as it is its own
|
|
toplevel object. Since the compositor determines the drag target, it
|
|
works with transformed surfaces (dragging to a scaled down window in
|
|
expose mode, for example).
|
|
|
|
Issues:
|
|
|
|
\begin{itemize}
|
|
\item we can set the cursor image to the current cursor + dragged
|
|
object, which will last as long as the drag, but maybe an request to
|
|
attach an image to the cursor will be more convenient?
|
|
|
|
\item Should drag.send() destroy the object? There's nothing to do
|
|
after the data has been transferred.
|
|
|
|
\item How do we marshall several mime-types? We could make the drag
|
|
setup a multi-step operation: dnd.create, drag.offer(mime-type1,
|
|
drag.offer(mime-type2), drag.activate(). The drag object could send
|
|
multiple offer events on each motion event. Or we could just
|
|
implement an array type, but that's a pain to work with.
|
|
|
|
\item Middle-click drag to pop up menu? Ctrl/Shift/Alt drag?
|
|
|
|
\item Send a file descriptor over the protocol to let initiator and
|
|
source exchange data out of band?
|
|
|
|
\item Action? Specify action when creating the drag object? Ask
|
|
action?
|
|
\end{itemize}
|
|
|
|
New objects, requests and events:
|
|
|
|
\begin{itemize}
|
|
\item New toplevel dnd global. One method, creates a drag object:
|
|
\texttt{dnd.start(new object id, surface, input device, mime
|
|
types)}. Starts drag for the device, if it's grabbed by the
|
|
surface. drag ends when button is released. Caller is responsible
|
|
for destroying the drag object.
|
|
|
|
\item Drag object methods:
|
|
|
|
\texttt{drag.destroy(id)}, destroy drag object.
|
|
|
|
\texttt{drag.send(id, data)}, send drag data.
|
|
|
|
\texttt{drag.accept(id, mime type)}, accept drag offer, called by
|
|
target surface.
|
|
|
|
\item Drag object events:
|
|
|
|
\texttt{drag.offer(id, mime-types)}, sent to potential destination
|
|
surfaces to offer drag data. If the device leaves the window or the
|
|
originator cancels the drag, this event is sent with mime-types =
|
|
NULL.
|
|
|
|
\texttt{drag.target(id, mime-type)}, sent to drag originator when a
|
|
target surface has accepted the offer. if a previous target goes
|
|
away, this event is sent with mime-type = NULL.
|
|
|
|
\texttt{drag.data(id, data)}, sent to target, contains dragged data.
|
|
ends transaction on the target side.
|
|
\end{itemize}
|
|
|
|
Sequence of events:
|
|
|
|
\begin{itemize}
|
|
\item The initiator surface receives a click (which grabs the input
|
|
device to that surface) and then enough motion to decide that a drag
|
|
is starting. Wayland has no subwindows, so it's entirely up to the
|
|
application to decide whether or not a draggable object within the
|
|
surface was clicked.
|
|
|
|
\item The initiator creates a drag object by calling the
|
|
\texttt{create\_drag} method on the dnd global object. As for any
|
|
client created object, the client allocates the id. The
|
|
\texttt{create\_drag} method also takes the originating surface, the
|
|
device that's dragging and the mime-types supported. If the surface
|
|
has indeed grabbed the device passed in, the server will create an
|
|
active drag object for the device. If the grab was released in the
|
|
meantime, the drag object will be in-active, that is, the same state
|
|
as when the grab is released. In that case, the client will receive
|
|
a button up event, which will let it know that the drag finished.
|
|
To the client it will look like the drag was immediately cancelled
|
|
by the grab ending.
|
|
|
|
The special mime-type application/x-root-target indicates that the
|
|
initiator is looking for drag events to the root window as well.
|
|
|
|
\item To indicate the object being dragged, the initiator can replace
|
|
the pointer image with an larger image representing the data being
|
|
dragged with the cursor image overlaid. The pointer image will
|
|
remain in place as long as the grab is in effect, since the
|
|
initiating surface keeps pointer focus, and no other surface
|
|
receives enter events.
|
|
|
|
\item As long as the grab is active (or until the initiator cancels
|
|
the drag by destroying the drag object), the drag object will send
|
|
\texttt{offer} events to surfaces it moves across. As for motion
|
|
events, these events contain the surface local coordinates of the
|
|
device as well as the list of mime-types offered. When a device
|
|
leaves a surface, it will send an \texttt{offer} event with an empty
|
|
list of mime-types to indicate that the device left the surface.
|
|
|
|
\item If a surface receives an offer event and decides that it's in an
|
|
area that can accept a drag event, it should call the
|
|
\texttt{accept} method on the drag object in the event. The surface
|
|
passes a mime-type in the request, picked from the list in the offer
|
|
event, to indicate which of the types it wants. At this point, the
|
|
surface can update the appearance of the drop target to give
|
|
feedback to the user that the drag has a valid target. If the
|
|
\texttt{offer} event moves to a different drop target (the surface
|
|
decides the offer coordinates is outside the drop target) or leaves
|
|
the surface (the offer event has an empty list of mime-types) it
|
|
should revert the appearance of the drop target to the inactive
|
|
state. A surface can also decide to retract its drop target (if the
|
|
drop target disappears or moves, for example), by calling the accept
|
|
method with a NULL mime-type.
|
|
|
|
\item When a target surface sends an \texttt{accept} request, the drag
|
|
object will send a \texttt{target} event to the initiator surface.
|
|
This tells the initiator that the drag currently has a potential
|
|
target and which of the offered mime-types the target wants. The
|
|
initiator can change the pointer image or drag source appearance to
|
|
reflect this new state. If the target surface retracts its drop
|
|
target of if the surface disappears, a \texttt{target} event with a
|
|
NULL mime-type will be sent.
|
|
|
|
If the initiator listed application/x-root-target as a valid
|
|
mime-type, dragging into the root window will make the drag object
|
|
send a \texttt{target} event with the application/x-root-target
|
|
mime-type.
|
|
|
|
\item When the grab is released (indicated by the button release
|
|
event), if the drag has an active target, the initiator calls the
|
|
\texttt{send} method on the drag object to send the data to be
|
|
transferred by the drag operation, in the format requested by the
|
|
target. The initiator can then destroy the drag object by calling
|
|
the \texttt{destroy} method.
|
|
|
|
\item The drop target receives a \texttt{data} event from the drag
|
|
object with the requested data.
|
|
\end{itemize}
|
|
|
|
MIME is defined in RFC's 2045-2049. A registry of MIME types is
|
|
maintained by the Internet Assigned Numbers Authority (IANA).
|
|
|
|
ftp://ftp.isi.edu/in-notes/iana/assignments/media-types/
|
|
|
|
|
|
\section{Types of compositors}
|
|
|
|
\subsection{System Compositor}
|
|
|
|
\begin{itemize}
|
|
\item ties in with graphical boot
|
|
\item hosts different types of session compositors
|
|
\item lets us switch between multiple sessions (fast user switching,
|
|
secure/personal desktop switching)
|
|
\item multiseat
|
|
\item linux implementation using libudev, egl, kms, evdev, cairo
|
|
\item for fullscreen clients, the system compositor can reprogram the
|
|
video scanout address to source fromt the client provided buffer.
|
|
\end{itemize}
|
|
|
|
\subsection{Session Compositor}
|
|
|
|
\begin{itemize}
|
|
\item nested under the system compositor. nesting is feasible because
|
|
protocol is async, roundtrip would break nesting
|
|
\item gnome-shell
|
|
\item moblin
|
|
\item compiz?
|
|
\item kde compositor?
|
|
\item text mode using vte
|
|
\item rdp session
|
|
\item fullscreen X session under wayland
|
|
\item can run without system compositor, on the hw where it makes
|
|
sense
|
|
\item root window less X server, bridging X windows into a wayland
|
|
session compositor
|
|
\end{itemize}
|
|
|
|
\subsection{Embbedding Compositor}
|
|
|
|
X11 lets clients embed windows from other clients, or lets client copy
|
|
pixmap contents rendered by another client into their window. This is
|
|
often used for applets in a panel, browser plugins and similar.
|
|
Wayland doesn't directly allow this, but clients can communicate GEM
|
|
buffer names out-of-band, for example, using d-bus or as command line
|
|
arguments when the panel launches the applet. Another option is to
|
|
use a nested wayland instance. For this, the wayland server will have
|
|
to be a library that the host application links to. The host
|
|
application will then pass the wayland server socket name to the
|
|
embedded application, and will need to implement the wayland
|
|
compositor interface. The host application composites the client
|
|
surfaces as part of it's window, that is, in the web page or in the
|
|
panel. The benefit of nesting the wayland server is that it provides
|
|
the requests the embedded client needs to inform the host about buffer
|
|
updates and a mechanism for forwarding input events from the host
|
|
application.
|
|
|
|
\begin{itemize}
|
|
\item firefox embedding flash by being a special purpose compositor to
|
|
the plugin
|
|
\end{itemize}
|
|
|
|
\section{Implementation}
|
|
|
|
what's currently implemented
|
|
|
|
\subsection{Wayland Server Library}
|
|
|
|
\texttt{libwayland-server.so}
|
|
|
|
\begin{itemize}
|
|
\item implements protocol side of a compositor
|
|
\item minimal, doesn't include any rendering or input device handling
|
|
\item helpers for running on egl and evdev, and for nested wayland
|
|
\end{itemize}
|
|
|
|
\subsection{Wayland Client Library}
|
|
|
|
\texttt{libwayland.so}
|
|
|
|
\begin{itemize}
|
|
\item minimal, designed to support integration with real toolkits such as
|
|
Qt, GTK+ or Clutter.
|
|
|
|
\item doesn't cache state, but lets the toolkits cache server state in
|
|
native objects (GObject or QObject or whatever).
|
|
\end{itemize}
|
|
|
|
\subsection{Wayland System Compositor}
|
|
|
|
\begin{itemize}
|
|
\item implementation of the system compositor
|
|
|
|
\item uses libudev, eagle (egl), evdev and drm
|
|
|
|
\item integrates with ConsoleKit, can create new sessions
|
|
|
|
\item allows multi seat setups
|
|
|
|
\item configurable through udev rules and maybe /etc/wayland.d type thing
|
|
\end{itemize}
|
|
|
|
\subsection{X Server Session}
|
|
|
|
\begin{itemize}
|
|
\item xserver module and driver support
|
|
|
|
\item uses wayland client library
|
|
|
|
\item same X.org server as we normally run, the front buffer is a wayland
|
|
surface but all accel code, 3d and extensions are there
|
|
|
|
\item when full screen the session compositor will scan out from the X
|
|
server wayland surface, at which point X is running pretty much as it
|
|
does natively.
|
|
\end{itemize}
|
|
|
|
\end{document}
|
|
|