...But I take my kids anyway. Out there in the woods we've come into conact with dangerous plants (poison ivy, thorns), dangerous animals (yellow jackets, skeeters), dangerous situations (back roads, cold weather boating), and well, you get the picture.
The kids love it. My wife even enjoys it, too. Caching gives us a reason to go somewhere, it makes the trip interesting all the while teaching the kids (and the wife) about navigation, planning, the outdoors, and ethics (e.g. trade-up!), and safety. Because we try to be good parents we keep it safe as possible by carrying first aid kits in our packs, first aid training in our heads, wear our seatbelts, our PFDs, identify and avoid dangerous plants and animals, and generally being watchful for dangerous situations.
I can't think of a better way for you to spend an afternoon with your kids. My wife won't *let* me go caching *without* my kids in tow. She recognizes how good it is for them and me.