I agree with those who suggest that you simply get yourself a compass and not worry about replacing your GPSr. While I'm not a veteran geocacher (just under 100 caches found), I think the problem is in the inherent slowing down as one approaches a cache. We slow down, look around, the GPSr needle begins to spin as we slow down, and the next thing you know we're going in circles.
As you get to within 15 to 30 meters of the cache, take note of the bearing and distance. Put your GPSr away, set the bearing on your compass, and approximate distance. When you get to where your compass and distance estimate tell you the cache should be, put your personal search engine to work: start thinking like a cacher, and find the cache.
If you need to, back away from your suspected location by at least 15-30 meters, get the GPSr out again, and get a new bearing and distance. Again, use your compass and distance estimates as soon as the GPSr gives you the info you need.
My experience is that I over rely on the GPS receiver when I'm close to the cache, when I should be using my eyes and my brain. Your eyes and brain are more useful than your GPSr, and way less expensive!