I've had my Visor for about a year and have no complaints. I got a GPS Companion for it just days ago and, again, have no complaints.
I am, however, a newbie to GPS and geocaching (although on my first and second journeys out my girlfriend and I had success).
I've never seen GPS-only units so I can't comment on them, but I do love having the ability to download stuff to the Handspring Visor. I just downloaded a Hint Decoder (link at geocaching.com) that'll save me a few minutes. I've downloaded some other software that I'm just now starting to look at.
Yeah, the Visor/Companion combo looks a little geeky (not as rugged as the GPS-only units I've seen photos of), but it works!