I have yet to come across a muggle while caching. Normally, to aviod them coming to me, I just point at some random object (if I'm with another person) and act as if I'm explaining something. It works.
If one would come up to me, I'd say, I'm treasure hunting.