When using the X Window System, the X Server is the actual display which programs are drawn on. This is confusing to many people, as the display is right in front of them, and typically servers are far away, on the other side of the network.
In the case of X, though, the thing that is being served out is the ability to display on the screen. X Clients are programs (that can be run locally or remotely) that draw on the screen provided by the X Server.
A X server to connect to is identified in a display string, usually stored in the DISPLAY environment variable. The display string takes the form: hostname:display.screen. Most users only have a single display (:0) consisting of a single screen (.0, which can be left off). When the server is local, hostname is left off as well, leaving only :display.
X server implementations
The X.Org Foundation develops an X server (usually refered to as Xorg) as a reference implementation of X (they control the X protocol specification). Development is opened up and put on the freedeskop site. It can be downloaded at the x.org website. Xorg is a fork of the XFree86 project, and is currently (2007) the most popular open source X server.
XFree86 used to be the most popular X server implementation, however, the popularity of XFree86 has suffered recently as the terms of the XFree86 license contain conditions thought to be too restrictive for some distributions, who have since switched to Xorg. XFree96 is actually a suite of programs including a X Server, drivers for a vast array of graphics cards, support files such as fonts etc., and a number of standard desktop utilities such as xterm and twm.
TinyX (also known as XKdrive) is an X server designed for special environments with limited memory or disk space. Each TinyX server is compiled for a specific driver (for a graphics chipset) and with a selected number of extensions. It doesn't use XFree86's more modular approach to reduce it's size. It is included in the source of XFree86.
Athene is a commercial windowing system with support for X11 . The free (as in beer, for non-commercial use only) version of athene includes vesa support only, but the full (paid) version has accelerated display support. Also includes pandora, zterm, a login manager, and three desktop environments.
For other OS