Well it depends on the day and location (urban, country, or more remote) and I break it down into essentials, extras and the bonus items. I've only been caching a few months but every new experience leads me to go out with more items the next time. When you encounter a wet log book for the 2nd or 3rd time you realise it pays to bring a spare with you.
The bare minimum for me is (I don't go caching without these):
GPS Device
Pen
Pencil (or spare pen)
Notepad (for multi cache's writing up notes if the phone is going to die shortly etc.)
Items for swapping
Tweezers for helping get logbooks out of nano caches and micro caches (also handy for splinters)
Bottle of water
The added extras I find are invaluable are (i rarely go without these):
Camera for logging unloggable finds (muggled caches, full or overly moist logbooks, etc.) - also great for recording the days adventures or interesting things you encounter near a cache. Most importantly if you carry an SLR or similar proper camera it's a great way to linger without looking suspicious. Or investigate a location in detail pretending to be doing some macro photography, etc.
Geocoins/Geotags - for nice swaps
Plasters and some antiseptic or wet-wipes
Human Backup - geocaching is far more enjoyable with 2 or more people, far easier with extra eyes and hands to search, and far stealthier when you have an extra body or two to keep eyes peeled and shield your actions
Spare batteries
Monopod (one legged camera support that doubles as a walking stick and useful probing/poking device) http://www.amazon.co.uk/Manfrotto-680B-Section-Monopod-Black/dp/B000A2JU7W/ref=sr_1_2?s=electronics&ie=UTF8&qid=1319646610&sr=1-2
Gloves - I use these for photography as well (lightweight, hard wearing with gripping fingers and palms) http://www.amazon.co.uk/Matin-M-7861-Photo-Gloves-Large/dp/B004EBUUMO/ref=sr_1_24?s=electronics&ie=UTF8&qid=1319646652&sr=1-24
Multitool
Torch
MP3 player (if solo caching I like to be in my own little world)
The things that can really help are (mostly the extras are for cache maintenance):
Spare film cannister
Spare logbooks
Tape
rubber bands
Clear plastic bags
A smallish rubbish bag for some tidying up if the area is heavily littered
Nano- or micro-cache ready to be deployed if I find a good spot