I'm still new and haven't been confronted yet, but the other night I was searching for a cache and someone walked by (it was in a wooded area, and strange to see anyone around). I gave him a questioning look instead, and he explained to me what he was doing instead (just a late-night golfer walking to my bike)