I have Palm TX that I use for paperless geocaching. Though for just storing cache descriptions, hints and logs it is an overkill. You can get any Palm (even the cheap ones or used ones from Ebay). Just make sure that it has USB port and data cable so you can load the information easily. Older units like Palm 505 or 515 are good too but if they come with serial port cable, you have to make sure they will be compatible with your computer.
You can get CacheMate for like $10. It is one of the best programs for Palm. It allows you to setup multiple databases of caches, view descriptions, logs, hits, etc. You can also log your find with date/time and notes. It also has nice feature that finds nearest caches (you just have to enter coordinates).
I also use CMConvert to transfer GPX files from geocaching.com site to Palm as PDB files. Fairly painless process.
If you have any questions let me know.