I remember when I started Geocaching and started using a cheap GPS... If I knew then what I know now, I would have footed the extra $$ for something a little nicer and FASTER.
I have an Oregon 450 for sale, which has a much larger color screen, faster processor, PAPERLESS GEOCACHING! It has the cables, carrying case, and even the original box for $175. It has some trail use, with minor blemishes, but its definitely a few steps ahead of what you are going to find under $100.
In the end.. whatever you decide to do, ENJOY THE RIDE and just foot the extra $$ for something a little beefier, with more options, and you WONT regret it!! Most of those cheaper GPS units come with very little internal memory and wont even accept an SD card for expansion (something I wasn't aware of when I purchased my first GPS) Not to mention they are usually painfully slow!
If you are interested in the one I have for sale, just shoot me an E-mail.