I use a Dell Axim x30 with a holux gpslim 236. They work great. I bought a Rhinoskin a long time ago to protect it from drops. (It does protect it from drops, I've tested it inadvertantly many times) I would have to say that some type of protective case is almost a necessity. It doesn't have to be an expensive otter box though. The case I have isn't waterproof. But so far I haven't ran into many geocaching trips through water.
Features for a PDA should include Bluetooth, to keep the option open for a Bluetooth reciever. Windows Mobile 2003 se. That will give you the option for landscape mode if so desired. If you can find one with a 640x480 screen that would be nice, but not a necessity. The higher res screen will enable you to see more of the map features clearly while still zoomed out. I use a program called magic button to switch between programs easily. Processor speed of course makes a difference. I would recommend at least 400 Mhz.
I use BeelingGPS. It can handle the gpx files completely. I tried gpstuner but everytime I tried to access the gpx info it tried to connect to the web for it. While I'm out in the field that's not possible for me. The only thing that is lacking in BeelineGPS is mapping. I use Mapopolis for that. There is a program called gpxtomlt. It takes the gpx file and makes waypoints for mapopolis. Then I use Mapopolis for navigating to the waypoint areas and then switch to Beeline to pinpoint it. There are plenty of programs to choose from. Someone should be able to find one to fit their needs. There are so many option that open up to you with a PDA that you don't get with a dedicated unit unless you are ready to pay a lot for it.