I use a Sony-Ericsson Xperia (X10) android phone and I often find the same thing however, I agree with what someone above was saying about the "error" level.
I mainly use two applications for paperless: one talks with the geocaching.com website to get information, hints, descriptions, post logs, etc... and the other one, like you, I use for so called "off-line" paperless geocaching.
With both apps (or the other ones i've installed), the error level (the plus or minus accuracy) of the GPS/GPS+Network will go up and down. I suspect this is the case with most non-dedicated gps devices or even real gps devices. (There are a lot of cache descriptions saying things like "the tall buildings can throw off your GPSr", etc..)
However, keeping an eye on the error/accuracy indicators really helps a lot.
If the GPSr told me exactly where to look in only a 1 meter (3ft) radius, there wouldn't much actual "searching" involved. Switching between compasses using GPS bearings and the magnetic compass bearing in my device helps a lot along with making use of the mapping features.
In the worst case, they get me within 15-20 meters of the cache, where the search fun can begin.