Private property on public land is STILL private property... because WE ALL own the public land.
If your Church erected a manger in a park for Christmas, could I claim it was "abandoned" and move it to my house? Of course not. Theft is theft, and being a jerk is being a jerk.
While I think, frankly, that trying the guy in court is extreme and silly, and a huge waste of time and money, humiliating a guy who derived pleasure from deliberately finding and stealing something that brought OTHERS happiness... well, I'm good with that.