User documentation contains documentation for running the X server and basic X clients, as well as protocol specifications and basic client programming documentation.
Developing X applications
For low-level X development, XCB, the X C Bindings provide a clean low-level protocol binding. Its older cousin Xlib (or libX11), is not recommended for new development, but is still very widely used. Xt is a similarly deprecated library for building toolkits. Documentation for Xlib and Xt is available.
Downloading, running, and developing X.Org code
The development documentation describes how to download and build the X.Org code (including the X server and drivers), and rough notes on their internals.