The first geocache in New Zealand was placed by Peter McKellar near Rotorua on the 12th May 2000 - this was in fact the first geocache placed outside of the United States! It was unfortunately trashed before anyone could find it.

Geocache by Peter McKellar (geocaching.com)

The first geocache in the South Island was place by Bob C on the 5th June 2000 southwest of Christchurch. It only had one successful log before being washed away in flooding.

Geocache by Bob C (geocaching.com)

The oldest geocache still active in New Zealand was placed near Wainuiomata on the 26th of May, 2000 by Kevin Anderson.

Geocache by Kevin Anderson (geocaching.com)