Xlib is the shared library provided by X to interface with the X server, allowing developers to use X resources. An X11 client (application) will be linked (dynamically, probably not statically) to Xlib. Xlib is famous for being hard to use, especially since it is not a GUI toolkit, but more a graphics library with a annoyingly hard API.
The average developer likely does not have to worry about Xlib though, as he is probably using a real GUI toolkit (such as Qt, GTK+,...), or a more friendly graphics library, which takes care of the crazy Xlib magic for them. Success.