This type of behavior is normal. The GPSr has a limited ability to figure precisely where it is. Sometimes 20 feet or so if there are less then perfect conditions, even less when there are greater obstructions. When you get close to the final spot, the pointer will jump around because the calculated location can change by a few feet due to the location uncertainty, even when you are not moving.
If you have a good signal and can get close to the location before this happens, just find the cache. If the cache is in a place where there is heavy local obstructions, like in a small grove of trees, the GPSr error will get quite large. Get a visual bearing from two different locations outside of the obstruction and mentally triangulate them to the spot where the cache is.