I used a PDA for a short period, but due to the listed con's and due to the bad choice of PDA I don't use it any more. The main reson for me to start with a PDA with build in GPS was the use of other applications outsite geocaching. The advantage is you can do a lot more with it, like use it as a navigation system (in those days an handheld GPS was not a good navigation system).
A handheld GPS is made for geocaching and has a lot of advantages. I use an old PDA to take along for tools and listings, which provides for me a perfect combination.