If you are just getting started, try E-bay. I have purchased three GPSr's, and none of them cost more than $45US even with the shipping. They all work great, and get me where I need to be every time. I would just recommend sticking to a name brand like Garmin or Magellan. Before you commit to buy just check the GPS reviews at geocaching.com. They are reviewed by folks using them for geocaching, so you get an idea of how they perform.
Why spend a lot of money until you know what features you really want or need. Once you get into the swing of things you will find out the features you will want on your next unit. Then, you can spend the big bucks.