XDC2014 Program

Topics

As usual, we are open to talks across the layers of the graphics stack, from the kernel to desktop environments, and about how we make things better for the developers who build them. if you're not sure if something might fit, mail or add it anyway, and find out if there's room in the schedule.

Ideas

Talk proposals

Teaching Waffle how to wiggle

The Waffle project has been designed to abstract the differences between the various windowing systems and APIs used in order to create GL contexts. Additionally it is designed to make the selection of the above a straightforward runtime decision, thus preserving the original GL code unchanged regardless of the actual target OS/windowing system. In the presentation we shall cover the process of retrofitting Windows' WGL with its unique design into the Waffle API.

Author: Emil Velikov

GLX_OML_sync_control

We'll talk about the OpenGL extension called GLX_OML_sync_control, used for synchronizing SwapBuffers requests to vertical retrace:

  • What is it? (Overview, quirks, and subtleties of the spec)
  • How can we test it? (An overview of our proposed Piglit patches)
  • Which drivers implement it correctly? (Spoiler alert: none)
  • What can we do about that? (Ideas ranging from individual driver patches to a Glamor-EGL implementation)

While we'll be sharing our experience and efforts to-date, we also hope for audience discussion contributing to further improvements.

Authors: Theo Hill & Jamey Sharp

Project REclock: Extending Nouveau with Voltage and Frequency Scaling for NVA3/5/8

Over the years Nouveau has grown to become a serious open-source alternative for the closed-source binary drivers provided by NVIDIA, offering support for most graphics cards up to the current generation, nearing OpenGL 4.0 compatibility, supporting VDPAU accelerated video and even some form of Linux support for DirectX 9. Despite its broad API support, the most heard complaint about Nouveau is poor performance. In an attempt to bridge the performance gap between nouveau and NVIDIA's binary driver, I have worked on an EVoC project in an attempt to deliver clock and voltage adjustment for the GeForce Gx2x0 and Gx3x0, hopefully paving the way for future generations of GPUs. The process of adjusting clocks touches many areas of the GPU: subcomponents must be paused correctly, a vast amount of parameters for the memory subsystem need to be determined and correct synchronisation with the display subsystem is required to prevent buffer underruns during scanout. On some GPUs, this work has already resulted in a 50% increase in framerate for serious OpenGL workloads. In this presentation I attempt to give a high level insight in the full process of reclocking at the best of my understanding, give a status update on what does and does not work with the kernel patches I have prepared and I hope to conclude with some benchmark results while I push the boundaries of my braveness with a demo of reclocking on the GeForce G210m.

Author: Roy Spliet

Graphics beyond the main compositor

Most of the graphics development is targeted at the main session compositor. This is reasonable, as it is the interface that is used by far the greatest part of the time. However, there is a lot of interesting development going on beyond the main desktop environments, including boot-loader graphics, initrd UIs, emergency terminals and late-shutdown screens. While the graphical demands in those situations are rather low, we have to deal with a different, unique set of problems: accessing graphics via generic drivers, smooth transition to a full-featured driver, reducing memory footprint and loading-time to a minimum, claiming resources in emergency-situations and more.

A lot of work has already been done to solve these problems, but challenges keep coming it. As only very few people actively work on those problems, this talk will walk through the different projects, describe the challenges and what future work is being planned. Eventually, our goal is to reduce moments off the main session to a minimum, make them as unobtrusive as possible and smoothly integrate into the different desktop environments.

Author: David Herrmann

Adding tablet devices support to the Wayland protocol

  • A brief explanation of the parts that needed to be worked on in order to get tablet support in Weston (libinput, the wayland protocol, and weston itself)
  • The changes that were made to libinput, and the thinking behind the design choices that were made.
  • The changes we've proposed for the wayland protocol in order to allow for tablet support, and the thinking behind the design choices that were made with the protocol.
  • The changes that were made to weston along with the design choices that were made. This part's going to be a lot more brief then the last two since it's not in regards to an API or a protocol.
  • A brief summary of the work that still needs to be done
  • Questions, suggestions, etc.

Author: Stephen Chandler Paul

The status of the graphics stack on FreeBSD

Today, FreeBSD is well behind Linux in the graphics stack area, but the community regains interest in the topic. Users want a beautiful and friendly desktop environment on their shiny new laptop, they want to try this Wayland thing all their Linux buddies talk about, and above all, they want to mine virtual currency and be rich!

But all of this comes with a price: contributors and users hit critical problems due to the gap with Linux. Some of them are technical, some are human. Right now, the Linux community leads developments and we need to learn again how the graphics stack work from top to bottom.

After a short history of major changes, the presentation goes through the current situation in the kernel, the packages and with our users. It then concludes with the actions taken and planned to restore a nice desktop experience on FreeBSD and, in the long term, be part of the developments, side by side with not only the Linux community but other platforms too.

Author: Jean-Sébastien Pédron

Scheduling hint: Friday

GLSL compiler: where we've been and where we're going

The generated code quality of the GLSL compiler (i965 backend included) has improved significantly in the last year without significant infrastructure additions. How do we measure our progress and how do we tell what we still need to improve?

Can we continue making significant improvements without adding significantly to our compiler infrastructure? (Probably not)

We know that we want an SSA-based compiler, but we have a number of options to get us there. How do we properly evaluate our options and continue making significant improvements to our generated code?

Author: Matt Turner

Wayland fullscreen shell and some ideas about compositor nesting

While the Wayland protocol allows developers to write compositors that run much closer to the hardware. However, running directly on KMS means that somethings that were previously done by X or auxiliary clients now have to be implemented in the compositor.

One such capability is that of screen sharing/recording. On X, this is done by the client grabbing the screen contents and (in the case of screen sharing) faking input events. Since neither of these is possible in the Wayland world, we need another solution. One solution would be to implement these capabilities in the compositor. However, that would require Weston, mutter, and KWin to each implement VNC, RDP, and whatever other protocols their user wanted. The solution I will present makes use of compositor nesting to allow compositors to punt the problem of talking protocols and doing video compression to an external process using the Wayland protocol.

Author: Jason Ekstrand

Consolidating the input stacks with libinput

In the X world we have the server providing us with much of the higher-level concepts of the input stack and a number of input modules for some device-specific functionality (wheel emulation, tapping, etc.). Communication between those parts is difficult and almost nonexistent.

Wayland requires the input stack to be re-written for every compositor. To avoid that, we have created libinput, a library that provides almost the whole stack of input processing, across all devices. The compositor's input stack is a thin wrapper around libinput, limited to the actual semantic handling of the events.

This talk outlines libinput, talks about what we can already do, what we can't do yet and why not, and what we may be able to do once someone tells us what to do in the first place.

Author: Peter Hutterer

Scheduling hint: Please schedule this talk right before Stephen Chandler Paul's

Porting drm/kms drivers to DragonFlyBSD

Starting from 2010, the Xorg graphic stack has begun to require the use of kernel components to operate.

This is the story of how I ported some of the drm/kms kernel drivers and their associated infrastructure to the DragonFly BSD operating system, some choices I made and plans about future actions.

Author: Francois Tigeot

Modern and Interactive Scientific Visualization

Whereas the availability of data increases exponentially fast, the current visualization tools available today in Python do not scale gracefully to big data. The major plotting library in Python is Matplotlib and is mostly focused on the generation of static publication-ready figures rather than interactive visualization. These are really two different, and nearly orthogonal goals. For the former, high display quality is the major objective, whereas speed and reactivity is much more important for the latter. Matplotlib can be used for interactive visualization, but it has not been primarily designed for this. Consequently, the frame rate tends to be low on medium-size data sets, and million-points data sets can not be decently visualized in this way.

Vispy is a new high-performance interactive 2D/3D data visualization library that leverages the computational power of modern Graphics Processing Units (GPUs) in order to offer both fast and high quality scientific visualization using the Python language and modern OpenGL (shaders). Our primary goal is not to make publication quality plots -- even if we expect high quality using modern and dedicated GPU techniques such as dashed lines, curves, markers, arrows, interpolations -- but rather to get a sense of the data by visualizing it interactively. The nature of the data can be anything: real-time signals, maps, high-dimensional points, surfaces, volumes, images, textures, etc.

Main sites: vispy & glumpy.

Author: Nicolas P. Rougier

FreeRDS and its X friends

The FreeRDS project aims to provide an open-source RDP server based on FreeRDP. This talk will concentrate on the interaction between FreeRDS and X11 / Wayland.

First I will introduce the FreeRDS architecture. With FreeRDS, the content to "display" is created by content provider programs, then FreeRDS encodes and sends created frames packed into RDP over the network. So I will present Xrds that is an X11 content provider for FreeRDS: how we have implemented it and which problems we have faced.

I will close the presentation by talking about the weston FreeRDS backend - the FreeRDS compositor.

Author: David FORT

HPC: An OpenCL simulation of molecular dynamics on heterogeneous architectures

A molecular dynamics (MD) simulation involves computing the evolution of a system of particles in time. It is mainly used in physics where applications need to track several millions of particles. Due to the compute-intensive nature of MD simulations, researchers tend to use highly parallel computing architectures and accelerators (e.g. NVIDIA GPU & Intel Xeon Phi). In this presentation, I attempt to showcase an efficient OpenCL simulation of short-range particle interactions on heterogeneous architectures. This multi-accelerators approach aims to distribute the workload among the devices and to overlap data transfer and code execution in order to reduce the latency.

Author: Samuel Pitoiset

Randomized Testing

Testing all of the corner cases of a complex piece of software is hard. It's really, really hard. The problem can be thought of from a search perspective: somewhere in the space of possible inputs, the software under test may produce incorrect results. As a tester, the goal is to find those inputs. For many pieces of software the space of possible inputs is uncountably infinite. Traditional search techniques fail in these scenarios.

All is not lost, however. There are many examples in algorithm research where judicious application of randomization to exceptionally difficult problems can either solve the problem correctly in probabilistic finite time or solve the problem in finite time with probabilistic correctness.

Numerous aspects of an OpenGL shading language compiler have uncountably infinite possible inputs. Specifically, the possible set of uniform blocks is uncountably infinite. Each possible uniform block has a single, valid memory layout. Many GLSL compiler implementations produce incorrect layouts for some uniform blocks. These factors make this an ideal "search" problem for a randomized technique.

I would like to present:

  • Background of randomized techniques.
  • The problem of uniform blocks, and why it's so hard to get right.
  • The randomized test generator that has been implemented.
  • Sampling of the bugs that it has already found.
  • Ways to extend the current test generator.
  • Additional areas that would be good candidates for a similar testing method.

Author: Ian Romanick

Tizen, a Wayland-powered operating system

Tizen is a fully open-source and embedded-oriented operating system. Having a tradition of using the latest technology available, it integrated Wayland since its first stable release in 2013. Pushing this move forward, the first images using only Wayland/Weston, with no X11 server installed, were released as soon as February of 2014.

There are still a lot of Wayland-related challenges to overcome, though. There are different profiles (Common, IVI...) with different needs, application frameworks to integrate, shortcomings in the Wayland protocol or in its shells... we will do an overview of the work done in this area.

1) A display-system-independent build system

The Tizen build system provides a way to build packages and create a bootable image independently of the display system used. As a result, most of the core packages are built with abstracted GUI libraries or with conditional flags, allowing supporting multiple display backends. We will have a quick look about how this works.

(sample packages : Mesa, weston-common, x11-common, EFL...)

2) Wayland shells, Tizen API integration, toolkits integration

The Tizen framework provides an API for end-user applications, so they can for instance hide and show themselves. As the core Wayland protocol did not provide these facilities, we made adaptations to the work-in-progress XDG-Shell protocol, which are now required by the framework and were first tested on Tizen before their upstream integration. There are still adaptations in progress, for which we will show demonstrations and a quick overview.

(new requests : xdg_surface_minimize(), xdg_surface_present() )

Tizen IVI is an in-vehicle-oriented flavor of Tizen. A prerequisite for IVI releases is the IVI-Shell protocol, an alternative shell interface for Weston based on the GENIVI group specifications. We participate in the upstream process by providing a Weston repository with IVI-Shell patches integrated, and integration in the framework and various toolkits (see below).

(ivi-shell : upstream Weston with patches, weston-ivi-shell plugin)

With regards to all the work done with XDG-Shell and IVI-Shell, it is important to provide integration in the available toolkits. We actively push patches to their upstream communities, and backport them into Tizen when necessary.

(xdg-shell & ivi-shell : EFL, Qt, Ozone-Wayland for Crosswalk… )

3) Wayland/Weston and the Tizen multiuser challenge

Tizen is currently switching to a fully multiuser model, which implies having several users connected to a same box and sharing input devices and displays. Making this work smoothly requires modifications in both the application framework and the compositor plugins. We will here sum up what is planned at this date.

4) Wayland/Weston and the enforced security model

Tizen is progressively switching from a security framework named privilege-manager to another one named Cynara. This technology upgrade is the perfect frame to allow to set up a communication channel between privileged applications, the compositor, and a security backend. This will allow applications requiring privileged access to the compositor (screenshooting, compositor plugins overloading...) to do so in a trusted way.

NOTES & REFERENCES

  • https://www.mail-archive.com/general@lists.tizen.org/msg02056.html
  • http://lists.freedesktop.org/archives/wayland-devel/2014-July/016224.html
  • https://github.com/Tarnyko/weston-ivi-shell - http://www.phoronix.com/scan.php?page=news_item&px=MTc3MjM - https://github.com/01org/ozone-wayland/commit/a034a018b6ec317ec5559dcce6efec916ec40512
  • https://wiki.tizen.org/wiki/Multi-user_DisplayManagement

Author: Manuel BACHMANN

Scheduling hint: An hour talk

Status Report on A New Linux OpenGL ABI

At XDC 2013, I gave a proposal for a new Linux OpenGL ABI that would allow multiple OpenGL implementations to coexist cleanly on the filesystem:

And the implementation so far is here: libglvnd

In this presentation, I will give a status update on the work that has been done thus far and describe what work remains.

Author: Andy Ritger

Enabling Alternative Window Systems with a non-Mesa Graphics Driver Implementation

Alternates to the X Window System, such as Wayland and Mir, have evolved through novel use of the building blocks of the Mesa-, DRI-, and DRM-based Linux graphics driver stack.

We have been working to support alternative window systems in the NVIDIA Linux Graphics Driver through:

  • Refactoring our driver stack to more closely align to the organization assumed by these alternative window systems (e.g., by working to plug into DRM-KMS).
  • Proposing new EGL extensions that alternative window system implementations can use to enumerate GPUs, correlate objects between EGL and DRM-KMS, and present frames through EGL + DRM-KMS.

In this talk, I will describe the work NVIDIA has been doing to prepare to plug into DRM-KMS, present the EGL_EXT_device_* and EGL_EXT_output_* family of extensions:

and suggest an EGLStream-based mechanism to present frames to an EGLoutput

Author: Andy Ritger

Nouveau: Expose NVIDIA's performance counters to the userspace for nv50/tesla

A hardware performance counter is a set of special registers which are used to store the counts of hardware-related activities. Hardware counters are often used by developers to identify performance bottlenecks in their applications. There are two types of counters offered by NVIDIA which provide data directly from various points of the GPU. Compute counters are used for OpenCL, while graphics counters give detailed information for OpenGL/Direct3D. It took me two Google Summer of Code (GSoC) to manage to expose them in Nouveau. In this presentation, I will first sum up my GSoC 2013 where I reverse engineered and implemented compute counters for nvc0/fermi in mesa. Then, I will introduce my GSOC2014's work to design a kernel interface to expose compute- and graphics-related counters to the userspace (mesa).

Author: Samuel Pitoiset

Glamor Status Report

Glamor, a 2D acceleration subsystem for the X window system, was started by Eric Anholt in 2008. In the intervening six years, GL support on common Linux desktop hardware has grown dramatically. This has allowed us to re-engineer Glamor from the ground up. Glamor now relies on versions of GL and GLES that permit it to fully accelerate the X graphics API, essentially eliminating the need for efficient software fallbacks.

Glamor development has been accelerated by the integration of Glamor support into the Xepher X-on-X system. This brings the ability to work on X acceleration to developers with only a single machine capable of running X, eliminates the need to run development code as root, and offers faster edit/compile/debug cycles and better debugging tools.

This presentation will describe both the current Glamor architecture, Xephyr integration, how Glamor uses various GL and GLES versions and plans for future Glamor work.

Author: Keith Packard, Intel Open Source Technology Center, keithp@keithp.com

KWin went XCB

Over the last two years KWin and other components of the Plasma workspace and the KDE Frameworks were ported from XLib to XCB. This transition was triggered by the fact that Qt 5 uses XCB for the event handling. Thus all code processing XLib events had to be ported to be usable with Qt 5.

This talk will explain why we did a complete transition to XCB in KWin and the state of the porting. The talk will highlight the issues we hit during the porting and also what we really liked about XCB in comparison to XLib.

As a C++ application using a low level C library is not particularly thrilling to the developers especially as it can be error-prone. The KWin development team went a long way to make using XCB inside the code base as convenient as possible and safe using advanced features of C++ like RAII and variadic templates. The talk will explain why we introduced wrappers for XCB and illustrates the advantages from a coding point of view. As these wrappers do not dependent on other parts of KWin and are mostly template based header only implementations they might be of interest for other C++ projects interacting with XCB.

Author: Martin Gräßlin

Security in Wayland-based desktop environments: Privileged clients, authorization, authentication and sandboxing!

We have been talking about security in the Graphics Stack for some time. Wayland has fixed all the problems we had ... but at the expense of usability and accessibility as applications are now unable to perform any sensitive task such as taking screenshots or injecting input events. Indeed, some applications (especially the ones related to accessibility) really DO require privileged capabilities and should work across the different Wayland compositors. Before designing privileged interfaces, it was thus necessary to think about how to handle privileged clients securely. The biggest issue with handling privileged clients is not how to grant them the rights, it is about making sure that the user understands what is going on on his/her system. This sometimes require to capture the user's intent somewhere in the TCB (Trusted Code Base), often in the compositor.

In February, we proposed a way to handle privileged clients. We first started by listing different ways of capturing the user's intent in the Wayland compositor. We then talked about how to expose the current security threats and vulnerabilities to user by adding an icon to the systray allowing the user both to see the security properties that may become violated and by which programs. The user could then revoke the rights away from an application for the current instance or any further instance.

We also proposed a way to delegate security decisions within the compositor to a centralized security decision engine such as SELinux, AppArmor or polkit. This would allow distro developers to ship applications with their security policy for the system and the graphical environment. We called this proposition Wayland Security Module (WSM) because they resemble the Linux Security Module interface found in the Linux kernel. An android-like system could thus be implemented where the package manager displays the capabilities required by the application and the user would select the permission he/she gives to the application. The same policy could be shared across all the Desktop Environments thanks to the common WSM interface that would need to be used by all the wayland compositors. Work on the WSM is currently on its way.

This leads us to another GUI-related problem of applications, they need to be able to access files only when the user wants it to. Capturing the user's intent when it comes to giving permission to an application to open a file requires to delegate the file chooser to an external process. Doing so has several challenges that we will talk about before showing our proposition and show you our interface (both from the code and the GUI perspective).

Currently, any application can pretend to be your DE's window that asks for credentials. This inspired us to look into the different of authorization and authentication UIs on major OSes and their problems. We finally proposed several ways of making your DE's authentication window be unspoofable by making it trivial for a user to check its legitimacy.

Authors: Martin Peres & Steve Dodier

AMD's New Unified Open Source Driver

AMD is moving to unify it's current open and closed source Linux drivers over a common, shared, open source kernel driver.

This talk will provide an overview of our plans for the future and the challenges we face as we move forward.

Author: Alex Deucher

StarPU: seamless computations among CPUs and GPUs

Heterogeneous accelerator-based parallel machines, featuring manycore CPUs and with GPU accelerators, provide an unprecedented amount of processing power per node, shipped in a very complex machinery. To fully tap into the potential of such machines, one has to deal with optimized task scheduling, multiple accelerators, overlapped data transfers, unused data eviction,... Achieving all that by hand becomes harder and harder, and directive-based languages are often too explicit to really achieve optimized execution. This talk will show how introducing a dynamic run-time systems allows to achieve such optimized execution, under the guidance of the application, through a task-based programming model. It will also discuss what kind of low-level support is useful to be able to optimize execution, and how such a run-time can be integrated as a backend for high-level languages which thus do not have to care about low-level execution.

Author: Samuel Thibault

Where does accessibility plug into the graphical desktop stack?

Yes, it does make sense for a blind user to use a graphical desktop. More generally, it has to be accessible to a very wide range of handicaps, so we need to make sure that accessibility tools can plug into the stack to get, modify or inject information, so as to compensate the user's disabilities.

I will briefly remind the path of an 'a', from pressing the keyboard key to the application, i.e. the input part, and then from the application up to its rendering on the screen, i.e. the output part. I will then explain how various tools plug into various places. On the input side, there are virtual keyboards, braille keyboards, AccessX keyboards,... and infamous issues between keycodes and keysyms, which we could perhaps take the opportunity to fix with Wayland. On the output side, there are magnification tools, color modifiers,... and last but not least, screen readers, for which I will explain the accessibility bus (at-spi) basics.

Author: Samuel Thibault

Backlight kernel ABI discussion

Recently 3 issues with the kernel ABI for backlights have come up:

1) It would be nice to have the backlight level as a property of the connector. Problem is mapping a backlight interface to a connector. There are some ideas to let userspace tell the kernel which backlight interface to use (through e.g. udev rules), but otherwise handle this in the kernel

2) What is the meaning of a brightness level of "0" (or min, but troublesome cases all seem to actually have 0 as min), does this mean backlight at minimum or backlight off ? Currently backlight interfaces are not consistent in this. We need to clarify the ABI here (and then fix drivers which don't match the clarified API)

3) Related to 2, for some backlight interfaces the value has a scale of perceived brightness, where for others it controls the raw pwm value (which does not map linearly to perceived brightness)

Author: Hans de Goede