I've been yelled at once, it was about two years ago I think. Me, my sister and two friends had logged a cache in the woods near a public road. We were coming down a quite steep hill when a middle aged man dressed only in his underwear rushed from a bush and started yelling "get out of my yard". There was no house to be seen anywhere, just woods and the cliff. I double checked the map afterwards and there was a house, but really far away from the path we took. I think that man has heard us going up the hill and decided he didn't want us anywhere near where he can hear us.
In Finland, we have this law about woods and bushes, it's called jokamiehenoikeus in Finnish. It gives everybody a right to hike anywhere, collect berries, mushrooms etc, just as long you keep away from the yards. I think it's polite to keep the distance so you cannot be seen from the yard. But in this case, we were really far away from the house. It's also typical here that people think everything they can see is their territory and in some places there are (illegal) private property signs in the middle of the woods.