One thing I have notice using the iPhone for finding urban caches is that no one gives you a second glance when you are wandering the city with an iPhone in your hand. You're just one of hundreds of others doing the same thing.
If you have a device that isn't common (or is bright yellow) in your hand, you might draw a bit more curiosity.
Of course, if you are walking along holding the iPhone horizontally in front of you, and staring at it as you walk, you might be thought of as a bit odd, but that's not the device's fault.
My $.02 as a newbie to geocaching.