I have had several encounters with police while caching, however, I too ALWAYS carry a couple of brochures and cards with me. The times that I have been "stopped" I immediately inform them that I am geocaching and that I have a brochure that explains it fully (NEVER reach in your pocket for one until you say that you are reaching for it!) I also have a geocaching.com 3"x3" sticker on my windshield and a geocaching.com licence plate frame on the back that has helped.
Due to this foresight, the worst I have been through is bad jokes, "geocaching? Well if there's cash, I can have 20 officers here in less than 5 minutes to look for it." (Tip: smile and laugh). I have even had one officer stop me that was a geocacher and gave me a tip on how to find it.
My biggest tips would be: Don't have attitude, say you are geocaching and offer a card or brochure to explain. There are quite a few websites (ie: www.geocacher-u.com) where you can print cards & brochures.