User documentation contains documentation for running the X server and basic X clients, as well as protocol specifications and basic client programming documentation.
Developing X applications
For low-level X development, XCB, the X C Bindings provide a clean low-level protocol binding. Its older cousin Xlib (or libX11), is not recommended for new development, but is still very widely used. Xt is a similarly deprecated library for building toolkits. Documentation for Xlib and Xt is available.
Downloading, running, and developing X.Org code
The development documentation describes how to download and build the X.Org code (including the X server and drivers), and rough notes on their internals.
The X New Developer’s Guide attempts to be a gentle introduction to how the entire stack is put together and how some of the pieces work.