Here is my 2 cent's. I've had my gps for about 6 weeks.
A cover is a good idea. I'm using an old sock for stuffing mine in the glovebox or my bag to keep the screen from getting scratched up. Maybe they have Cellphone type covers out there somewhere.
A PC cable. If your's comew with it, great. If not, it's a must have.
Toposoftware. City maps are nice but getting lost in the city is harder to do. Good topo software to set up waypoints etc. and to help you with trails is next on my list.
City software. My GPS has mapping and came with it, it's nice. Can't say how nice if your's doesn't do mapping.
NiMH Batters. I got kids, we have rechargable batteries. After we got the GPS, the kids god the NiCAD's and we got the NiMH batteries. These things eat batteries faster than Gameboys and flashlights. (ok maybe not Gameboys)
Anyway, that's my list.
In order.
1) GPS, hey it all starts here. Make sure it has a wrist strap.
2) PC Cable.
3) Rechargable Batteries. (Alkaline seem to be harder and harder to find.)
4) Mapping software of whatever type you need.
5) A cover. Why no. 5? Cause an old sock works until you get the 'more important stuff'.
By the time you get all this stuff, you will know what you want in accessories.