We look at the cache descriptions, terrain, logs, and maps. The logs sometimes have clues in them to the difficulty. I wish CO's would put attributes in, but not everyone does. Any lengthy hikes are out of the question generally, and we'll probably never do a 5/5. No kayaking either. Oh, and we can't climb trees. We keep teasing that we are going to start packing a grappling hook and rope to help us get back out of places. We are a married couple, and sometimes I (the wife) wait while my husband goes down slopes, places with not so secure footing, etc. At present I have arthritis, some slipped discs, sciatica and bursitis in one hip. Some days I wait in the car and watch my delightful spouse do the drunken bee dance. But I am an awesome navigator and map reader, a pretty good puzzle solver and I have some geosense and common sense about where things might be.
We've gotten pretty good at weeding out caches that are beyond my capabilities. Sometimes we fail, but that's part of the game.