As you travel around the internet, you leave a trail to follow. When Hansel and Gretel walked through the forest, they left a trail of crumbs. Your four-year old also leaves a trail of crumbs - especially after eating crackers.
Thus the term cookie - refering to a group of crumbs - came into common usage.
A cookie is used by website author's to remember you when you come back. This is so that you can get the greeting: hello spaz. You last visited our page 01/09/04. Since then there has been 4 new articles posted. etc etc.
Sometimes a cookie can be malicious, and sometimes they are good intentioned. If you get a creepy big brother feeling that somebody is watching you, your browser will have an option about accepting cookies - you also will be able to delete all past cookies from there. Or else there will be one or more plugins (a.k.a. extensions) that you can download and add to your browser to improve your control over the cookies.
Cookies are simple text files on your hard drive with key words that the website will read to get information. They sometimes have an expiry date, meaning that after a certain time they will erase themselves.