A good source for small trinkets are the quarter machines at the supermarket. Tiny ninjas, fake rings, plastic aliens and lizards are fun.
I prefer a pen over a pencil as some logs are wet or damp.
Mini flashlight is useful
GPS: Stick with Garmin or Magellan. Etrex or Explorist seem to be in the price range you are looking for.
Make sure you can plug in via USB for easy downloading of caches (paperless caching)
A good background map on the unit is also a plus. One with good road map while driving.
Extra accessory such as a dash or windshield mount for the car would also be nice.
Hope this helps!
-Bunny Gunner