Hey, Muse, I know how you feel. I'm a lone 'cacher as I don't have any family or friends nearby. I also live in an area that is saturated with drugs, mainly marijuana. Some work buddies have warned me about some areas to avoid, but it makes me nervous going after some of the caches.
I do not like having to guess where to start at, especially if looks like I'll have to cross private property to get to the cache. I'm not really in fear of my life or anything, but I do get kind of nervous. These are weird times and you can never really be sure about strangers.
I totally agree with you on this issue, and would like to see more detailed information about the areas around the caches.