The oldest cache in Australia is in Sydney and is called Lane Cove - May 18, 2000. See Cache No. GC3E.
It is now a virtual because New South Wales National Parks and Wildlife Service no longer allow physical caches in NSW National Parks.
Cheers,
Rob.