IMO geocaching is both a sport and a game, much like hockey, basketball, tennis, lawn bowling, etc.
Each hunt is a game where the object is to find the cache, hidden by another player. Geocaching as a whole is a sport consisting of many people participating in many separate games.