Slides available on http://brl.thefreecat.org/2014-10-xorg.odp http://brl.thefreecat.org/2014-10-xorg.pdf http://brl.thefreecat.org/2014-10-xorg.html [...] now, with a presentation from Samuel... on accessibility. He's been working on this topic for... many years. On a side note, we really need someone to help with the video recording, we can't have just one person doing that. So, if you are interested in learning how to do it, it's very simple. Just come, and just go to Julien, and yeah, it's gonna work out, thank you very much. Samuel... [Slide 1] Can you switch me on? Yes, thanks! So we'll talk about accessibility on a very technical side, so you can understand how it plugs into the graphical stack, that you know. So I have put the slides on brl.thefreecat.org, if you want to get back to the previous slides. [Slide 4] So I will first introduce a bit about accessibility, to explain the basic needs, the basic principles of accessibility. Then I will talk a bit about what happens when you press on [key a] and it show upon the screen, because not everybody knows everything along the path, so I will just bring back your memory and things. And then I will discuss about accessibility, on the input side first, and then on the output side. And especially on the input side, there are a lot of things there, particularly concerning Xorg. [Slide 5] So, first... Just a quick question: what do you think can pose accessibility issues here? What is the kind of problems we have here. So it's graphical, so OK, there is a text version of gnuplot, but what else? Colors, yes, exactly. I would like to know how many of you are colorblind. Please raise your hand if you don't distinguish at least some colors. So there are like three people out of, I don't know, fifty maybe. So that's a lot actually, there are 8 percent of the male people who are colorblind, so that's huge, actually. [Slide 6] So what is accessibility? So it's contracted A11Y. It means being usable by people with specific needs. It doesn't necessarily mean handicap or whatever, it's just specific needs. So of course you would think about blind people, but there are also some people with just low vision: they can see, but not so well. Of course there are deaf people, for Xorg it doesn't matter so much, but that's important. Colorblind, as I said. People with just one hand; try to press control-alt-backspace with just one hand... Cognition issues, so you have difficulties with understanding complex interfaces, and things like this. Motor disability, so using a keyboard with Parkinson is quite difficult if you don't have something that helps you. And elderly people, who basically have everything at the same time [giggles]. So, if you are interested, there are some HOWTOs, which bring some information about all of these, in more details. And the important thing to understand is that maybe it's you. Maybe not now, maybe in twenty years, maybe next week. You may break your arm, and then you have only one hand, and you can not press control-alt-backspace any more. So accessibility is not just handicap, because handicap depends on the situation, and on the time, maybe you will have just one hand for one week, and then you will have your two hands again, etc. [Slide 13] And then the question quickly comes: but why the hell would you want to make a graphical interface accessible to blind people, while there is text mode which is so much easier to make accessible. The problem is simply software: there is a lot of stuff which is not available in textmode; if you want to have Javascript support, you have to have a graphical navigator, because otherwise it doesn't really work properly. And perhaps you also have business applications, and if you don't want to lose your job, you have to use the software that you have to use, and then you have no choice, you have to use a graphical desktop. And a more social thing is that when you are non-technical, you usually need some help from people around, and they are not technical either, and if you have your own software, then they will not be at ease with helping you, because they will say "OK, I don't know your software, I don't know how to help you." While if everybody has the same software, then people can get help from others, and it works fine. [Slide 14] Another common thing that people would think about is writing dedicated software, so I've mentioned some things... Generally it's a bad idea. First, just because when somebody writes a software for disabled people, they will think about their disability, and that will make a software which is suited to that person, but maybe not other people, so it's better to try to make accessibility in general. And then there is the lack of power, of manpower. For instance edbrowse has some javascript support, but it can not work on the long term. You do not want to implement a webbrowser nowadays, you just use the existing ones, because you don't want to implement javascript, flash, etc. etc. And the same for an office suite, the compatibilty with Word etc. is a pain to maintain. And as I said, it's better to use the same software when you work with somebody, just to understand each other, exchange files, etc. it works much better. So that's why it's better to make the existing applications accessible. [Slide 15] So, there are three main principles that you will have is: just the same software, made accessible, for a lot of reasons, and another thing, which is not obvious, is you want to have synchronization between both people using the software, both the disabled and the non-disabled people. That is, you just have alternate input and output of the same thing, so that the two people working together can understand each other. If somebody sees something that the other one doesn't see, then you have problems with working together. And you want this to be pervasive, that is, when you go to a library or whatever, you have a computer for, I don't know, finding books in the library, and things like this, you do not want to have to ask for the system administrator, for installing some software, configuring it... anyway he is not here, so you can not contact him. So ideally it should be available on all platforms, all the time, just ready to be enabled. [Slide 16] Just to give a few ideas about the status we have on the free software side. So text mode is quite well accessible, but for beginners, it's not really an option, quite often. Gnome is quite fine, we have still had quite a regression with gnome3, because they have reimplemented a lot of things. And in the end we are quite late compare to the Windows world, because we have started just some time ago and not so much. And compared to Apple, we are really at Stone Age. Apple made a lot of efforts into accessibility, they even have practice lessons at their AppleStores, for people to understand how to use the accessibility software etc. And it's integrated into the computer, you just have to enable it, and we don't really have this in the free software side, that's quite sad. [Slide 17] So... That was just an introduction about accessibility. I will talk about the 'a' press. [Slide 18] So, starting from the beginning... [sorry] So we have the keyboard [never mind]. So we have the keyboard, so you press the 'a' key on the keyboard. I mean on a qwerty keyboard, so actually the third row... the first key of the third row. And for the keyboard, he doesn't care about what is printed on the key. So '1e' just means the physical position of the key. And so we have the driver in the Linux kernel which takes this, and translate that into an internal kernel value, which is actually the same, but that's another space. And so the input layer pushes that the evdev driver, which pushes that to the input-evdev driver of the Xorg server, which adds 8, just because it prefers that value. And pushes that over the wire to the X client. And so, what is pushed to the client is a keycode, that is a physical position, and not 'a'. It's just the third row, first key. [Slide 19] And then, here, this is just an event, among all the other events of the client, so there is the toolkit, which will get this event, and see "Ok, it is a keypress, I will push that to Xkb", which translates that into a keysym. So now it is really 'a'. And then the input engine turns that into a signal which is sent to the widget... which decides to do something. For instance append 'a' to the text. [Slide 20] And then, we have the output. So the widget has some text, which it wants to output. It will use actually, nowadays, a text rendering engine like pango. And, so, what gets out of the client is a pixmap, not text, and which is pushed to the driver and to the video card. So we have a pixmap really early, right in the client. And one thing is, there is perhaps no screen. Please do not make drivers which assume that there is a screen which is plugged. Blind people don't have screens, they don't care. So if the driver doesn't want to start, that's really a problem for them, because they have to buy a screen which they won't use. [Slide 21] OK, so that was the story. And then I'll talk about input, which is a long thing actually, there are a lot of things. [Slide 22] So. first, with motor disability, some people may be able to use only some things, like just the keyboard. And then, they can not use the mouse, so you can use keyboard shortcuts, or you can use something I will explain later, to actually move the mouse with the keyboard. Maybe they can only use a joystick, they can not even move a mouse, just a joystick, and then you make a mouse out of this. And if you only have a joystick, or a mouse, or just one button, and you want to type text, then you can use a virtual keyboard, for instance. And there are lot of things I can even imagine about, because you never know what people may need. [Slide 23] So if you only have one hand, and it is permanent, then you would like to have a fast way of typing things. And if you had to use a standard keyboard, you would have to move the hand quite a lot to access all the keys. So one way to do this is to use a toggle that "mirrors" the keyboard like this, so that you can keep for instance your left hand on the left, and you can switch between the normal layout, and revert layout. So, this is not implement in Xorg, I'm not sure exactly where to implement it, and the actual layout details, I don't know, but that's something that we could implement. [...] Oh, Xkb does have this? [It could do] It could do, yes, yeah. OK, well, it has to be done some way. [Slide 24] AccessX is this kind of thing implemented in Xkb. So there are a lot of things... Like StickyKeys, so to press control-alt-backspace, you actually press control several times, to lock it, and then alt several times, and then you press backspace. So it "sticks" the modifier. Mouse keys is just turning the keyboard into a mouse. SlowKeys means, you have to keep the key pressed a long time before it actually shows up, just because people are really slow, have problems with moving their hands. Or conversely, if you want to slow down the repeat, because then, they will keep the key pressed for a long time, and then it will repeat a lot, so you slow down this. ToggleKeys would alert when you press caps locks and things like this. BounceKeys is particularly for Parkinson, so it will, if it gets a lot of key strokes in a short period of time, it will only take one of them, so that if you press, because you have Parkinson, you only have one key press. So this is all implemented in Xkb, both in the X server and the X client, depending on the precise details. [Slide 25] So, a virtual keyboard is mostly like this, so you have a keyboard, and you can click with the mouse, so there is some details about the focus, but basically that's this. [Slide 26] And it works by just injecting key presses right here, between the X server and the X client, so it injects keycodes, so physical positions. So it works well with an actual PC keyboard show on the screen. [Slide 27] And for braille device, that's a bit the same, at least for some of them. Some braille devices have a classical PC keyboard, so you can type, and what we have nowadays is brltty which discusses with the device, and injects these physical positions right into the kernel, so it works both in text mode and Xorg. It's just a new keyboard, actually, for the kernel, which is fed from userspace. So there is no problem here. [Slide 28] The problem comes when you have a braille keyboard. A braille keyboard is a keyboard with 8 keys which correspond to the 8 dots of braille. So you have to 256 different possibilities. So, just a couple of details. Only 'a' to 'z' is standard, the rest is completely dependent on both the language: it's not the same in English and in French; on the country: Belgium, Canada, and France doesn't have the same; and even on the usage: French braille has moved quite a bit, and some people use something else, etc. So the translation is far from simple. [Slide 30] And the problem is also that we have a keysym here, because I press 'b' for instance, on my braille keyboard, it's 'b', it's not a physical position on a PC keyboard. So what I would like to do is give the X server a keysym, but there is nothing like this, just because it's a keycode which gets out of the server. So I could try to inject that into the client, there is nothing like this at the moment. So what we do at the moment is backtranslate thanks to Xkb into a keycode, which is horrible. [You have input methods...] I will talk a bit, maybe, later, about some things. But that's one of the solutions that we have, and of course it poses a lot of problems, because if you want to type capital 'A', then you have to find the modifier to make the capital. If you want to type 'o' with a circumflex, you have to find the combining 'o', or the dead 'o', err the combining circumflex, or dead circumflex. Some people, what they do, is they remap one keycode into what they need, then simulate it, and then remap back to what it was. Which is horrible, but it works... [Slide 31] Some people would like to use the main PC keyboard as a braille keyboard. So you will have two steps: you take these keys, 'a', 's', 'd', 'f', turn that into dots, and then turn that into text. [Slide 32] So what I did was to implement it in Xkb, so we have these two keypresses, 'f' and 'd', which the layout turns into braille_dot_1 and 2, so there are several layouts depending on people. So we have an actual braille keyboard. And there is some filter which combines them into 1-and-2 at the same time, and then we can use XCompose, to turn that into some keysym. It's a bit of a hack, but it works quite well. [Slide 33] Another way to do all of this is due actually to another thing, which is horrible with braille, which is that I talked only about grade 0, that is we have a bijection between dots and a letter. But actually people would like to use abbreviations, just to type faster, so for instance, the -ation suffix coded like this in braille. There is ambiguity that is, it's the same as capital 'N', so there are some rules to rule out the ambiguity. But it's really complex, so in the end, we have to use an ibus module, for instance. [Slide 34] So that, you have all the keypresses, which get into the client, and then ibus gives that to an external server, which does everything that is needed, configurable, and everything, and then turns that into 'b', so it works quite well. So this is really recent, we have done this this summer, with the Google Summer of Code program. [Slide 35] So, I would like to ask: what about Wayland? Is it passing keycodes, keysyms? I've dug a bit, I haven't really found this kind of details. So ideally it should be able to do both, so we could inject both keycodes and keysyms, depending on what we have. So perhaps it's the opportunity to fix these things before we have this installed on all desktops. [At the moment we're sending keycodes, so I don't know, but I will admit that I don't really know the Wayland client stack that well] I'm sorry I couldn't hear... [At the moment we're passing keycodes, so I don't know, but I'll admit that we're only at the lower levels so far, so I haven't really looked at the client, so if anyone else can join me in the client, it's [...]]. OK. So this was about input, maybe there are some questions already? [... Wayland Text protocol... ] I couldn't hear [... The Wayland Text protocol... ]. The textmode, or Text? [Text protocol] Well, I will talk about output later, but [...] It's for what? [for the input] For the input... I don't about [it used by the input methods ... maybe ...] Well, for me it's a solution like this... Never mind [With Wayland, there is a protocol extension about the default input events, and that's the Text protocol, and that is specifically designed to do most of that stuff] OK. Well, we can discuss about it later maybe. Yes? [...] Yes, you have to have a good keyboard, so you can actually press independently all the keys. At least some of you keyboard would be independent. OK? [Slide 37] So now, let's talk about output. [Slide 38] It's a bit more complex, for some things. At least we already have a lot things which are implemented. So for people with low vision, they can for instance just tweak the DPI, so that we express that for us the screen is small, and then the desktop will use big icons and fonts, and everything. We can use XRandR panning support which provides actually some zoom, some basic zoom. And with gamma tuning and color inversion, you can improve, depending on the people, how they see colors. Just reverting the screen, I don't know why, but some people like it, don't ask, I don't know. GTK3 has a way to get perfect magnification. Unfortunately I have never had the time to work on this. The idea would be to have the application to render its window in a big pixmap, and then you can show that. The good thing is that it's perfect, because it's perfectly drawn, instead of taking a pixmap and then zoom it in. That's much better doing it this way. [Slide 39] So, but the problem is that, for blind people it's not enough, because they just don't see anything. And also a lot of things, like a virtual keyboard, you want to have just the buttons show on the screen, available and directly, and things like this. And so you wouldn't want to patch the rendering, but rather get the semantics from the application. [Slide 40] And... quite a long time ago, that was done by just snooping the X protocol, so you would see the text going from the client to the server [Slide 41] But nowadays, it's impossible, because you have pixmap over the wire. [Slide 42] And so the idea is that we have this application which is rendering something, but somewhere there is some abstract representation, and this is what is being exposed through a bus to the screen reader, to be then rendered on some devices. [Slide 48] So the idea is that, so we have our widget which is getting 'a', it will both push the text for rendering, and also send a signal to ATK, which is just a library which turns that call into a message which is sent over the bus to the screen reader, which can then write this through braille or speech or whatever. [Slide 49] And conversely, the screen reader wants to inspect what the application is doing. And so it will send a gettext message, which ATK receives, and peeks the text from the widget, and sends it back to the screen reader. So that allows to get the text, but also the parent, the children, and everything. [Slide 50] So that when you have an application which is like this, you have a window with some container, a menu, some buttons etc. the screen reader can actually browse through all this, and then get the information and expose that to the user the way he likes, so either a virtual keyboard, or braille device, or speech, or whatever. [Slide 51] So technically speaking, this works well with for instance GTK, and soon enough, KDE applications. Adobe did this for Acrobat Reader, they spent the effort for this. And of course all the old applications, which do not use ATK, are not accessible. [Slide 52] But still, in practice, even if technically they are accessible, that's not enough. Because for instance if you have this kind of dialog box, [Slide 53] and you have coded it like this, so you have first a column with "first name", "last name", "password", and then a second column with "foo", "bar", "baz", when you are on "foo" you don't know whether it's the first name or the last name. [Slide 54] So what you would have to do in your application is rather to define a "label for" some text, and then the screen reader knows the semantics between the label and the text. [Slide 56] When that's not the case, what we do is actually use some scripts in the screen reader to match all of these by hand, actually, so it's a bit patching, it's sad, but when you don't really control the source code, you have to do this. [Slide 57] So in the end, making an application accessible is not so difficult, it's just making accessible from the start, by just making it logical, and just writing... actually it's common sense to write things logically and not visually. [Slide 59] So design your application *without* the graphical output in mind, so that you only have some logic in there, and then the screen reader will be happy. It's a bit like CSS actually. And if possible, just use standard widgets because *they* have been made accessible. If you write your own widget, then you will have to implement ATK, to expose the information for the screen reader. And if you have some image, you have to provide a text, an alternative text for this, so that the screen reader can read it. And keep things simple. That will help everybody. That will particularly help people with cognition issues, but... everybody actually, so keep it simple. [Slide 60] So there are some pitfalls, I won't detail. [Slide 63] You have a tool which allows you to check that you actually expose the information. So it's called accerciser, and you see all the applications and you can see the tree of widgets in there, and check that you have the actual text from the widget. [Slide 64] There is some documentation, so the accessibility HOWTOs are quite old, but they are still really interesting to read. Gnome has written a development guide, for GTK applications, but it's quite general actually. [Slide 92] So, to conclude... So accessibility has a lot of various needs that you wouldn't even think about, so it plugs at various level in the stack, and it needs various tweaks, we have seen some of them for the keyboard. And the thing is: with Wayland, we need no regression there, because people will not be able to use their computer any more at all, because they can not press keys, etc. So we have to pay a lot of attention there. And, as I said, accessibility does not necessarily needs the rendering, but also the semantics, so the idea is just to separate the form from the content, so the screen reader can access the content, and have the logic of the content, and then there is some form, for the screen, but also for the screen reader. Thanks. [claps] So are there questions? [Have you any advice on having a kind of testing environment, where you can actually hear this stuff. Well I know [...] Is there any trick so I can do to an application or so, to help me spot errors I'm making? ] So accerciser let you check that it works. Actually testing would usually mean trying to use the screen reader, and it's not easy to learn how to use the screen reader. Mostly, what you can do -it's explained in the HOWTOs-, you can just try to use you application with just the keyboard, and quite often you will notice that your tab order is not interesting, things like this. We just show "OK, your logic is not so good". And quite often, that's actually enough, in that maybe your software will not be so much usable, I mean from an ergonomic point of view, but at least it will be accessible, people can use it, and that's already a huge thing, compared to not being able to use it at all. [OK, is there a tool to visualize what ATK it peeking out and sending out on its bus?] On the bus, I don't know, you can actually read from dbus and see what is showing up, but usually it's too complex, because you have all the insertions and etc. What you can do, however, is just taking the Orca screen reader, and enable the braille monitor, which will show you what would be rendered on a braille device, so you have an idea of what is being seen by the people. [OK, thank you]. [Anyone else?] [Many applications now are web applications] Yeah. [How realistic is it to expect a blind user will be able to use a modern web application with javascript and a bunch of similar things, which annoy *us* already, and are probably much more difficult for the blind people?] So, I haven't talked about web accessibility at all, because that's not the kind of thing I'm working on, and there is huge work on this indeed... which is being done, at least. Javascript is not too much a concern. Flash was really horrible for this. The tendency of the W3C is in good way, in that you have a browser which actually has the representation of the application which is being run, and then it can expose that to the screen reader. And then of course it's up to the application to have some thing which makes sense, etc., and not just a column and a column, and then that's just the same thing, applied to the people writing the website. And then the same, you would have scripts in the screen reader for a particular website, to put back some semantics in there. [Well, I have one question. In the Wayland model where every application is just outputting pixmaps, then maybe the place for accessibility is more in the... well GTK or QT. This is something that... you could use before the X server because all the rendering was done in the X server, but it's not done in this way now, so...] And anyway it was really not convenient to see the text here, text there, but you don't have the semantics between both, so it... it's actually better to get the abstract representation [Yeah, from QT or GTK] [Are they doing a good job?] Say it again? [Are they doing a good job so far?] They, you mean the application writers [no, no, the toolkit developers]. Yeah, yeah, it's quite fine. GTK only for now. KDE really lacks manpower to actually implement it. There is some basic thing which is working, but just for, like the label and that's all, so it's basically not usable, but at least technically there is something which is pushed on the wire. [Very well, thank you, anyone else? Perfect, thank you again!] Thanks! [Claps]