I started a few weeks ago. I got a Garmin etrex (the yellow one) for about $60.00 US on ebay and so far it's worked great for me. I suspect I will need a more advanced model in the future once I start getting into some more difficult caching.
Good luck!