I do the vast majority of my caching alone. I absolutely love to run out the door for a FTF at 1 am but I have left GZ several times when I've heard questionable noises in the woods. Most of the time those noises are accompanied by flashes of light with the woods "30 more feet!" but when there is no light, and no voices and lots of noise I BAIL!
In fact one time my best friend and I were out doing a 2 am FTF run in the woods, and he has a 20lb dog he brought with. We all heard a noise and the normally fearless dog whimpered and ran under his legs. We got out of there REAL quick.
I think the best thing to do is be aware of your surroundings. I recently purchased handgun and will be getting a concealed permit soon. At which point it will always be with me on nighttime FTF runs. And like everyone else said; the last thing I would ever want to do is have to draw it. I would much rather be aware, and leave before that need arises.
Happy hunting everyone!