Oldest cache I've found: GCE4 Another Brick in the Wall, placed Dec-2000 in New Canaan, CT. I believe it's in its original container.
Also an oldie, having just had its 10th "birthday" and special in that it's a traveling cache: GCC8CB Town & Country. The mention is a little off-topic, since this one isn't in the same spot (by definition), and recently got a new container.