Jump to content

Rating a Cache


Recommended Posts

First off sorry if this has been asked, but did a quick search and didn't see anything. 

 

I am curious for those who have made caches how to you decide to rate difficulty. My wife and I have made several caches prior but they are usually rather simple, so I just placed them 1 or 2 stars really. My newest cache however is intentionally tricky, my wife says its level 5. I would love for it to be but feel like that might be a little arrogant or cocky to say that's its worthy of the 5 star rating. So the question is what do you think earns that right? Because in my area they are extremely rare to see. 

 

 

Link to comment
11 minutes ago, Dan2099 said:

First off sorry if this has been asked, but did a quick search and didn't see anything. 

 

I am curious for those who have made caches how to you decide to rate difficulty. My wife and I have made several caches prior but they are usually rather simple, so I just placed them 1 or 2 stars really. My newest cache however is intentionally tricky, my wife says its level 5. I would love for it to be but feel like that might be a little arrogant or cocky to say that's its worthy of the 5 star rating. So the question is what do you think earns that right? Because in my area they are extremely rare to see. 

 

 

I was going to look at your hides online but there aren't any.

  • Upvote 3
  • Helpful 1
Link to comment
31 minutes ago, Dan2099 said:

I am curious for those who have made caches how to you decide to rate difficulty. My wife and I have made several caches prior but they are usually rather simple, so I just placed them 1 or 2 stars really. My newest cache however is intentionally tricky, my wife says its level 5. I would love for it to be but feel like that might be a little arrogant or cocky to say that's its worthy of the 5 star rating. So the question is what do you think earns that right? Because in my area they are extremely rare to see. 

 

Clayjar says difficulty is "How easy it is to find the cache", and that's it.   Getting to it/accessing it is Terrain.

For example, I have a huge camo rural mailbox on a tree limb.  You can see it approaching but need rope (or a ladder) to access. It's 1.5/5. 

I like clayjar because it's always been accurate for us and doesn't change, unlike some things do when the Help Center gets "updated". 

To view, see here.

 

 

Edited by cerberus1
  • Upvote 3
  • Helpful 4
Link to comment

My cash is a Batman themed birdhouse cache. The cache is about three Batman villains joker, Two-face, and the Riddler. Each villain  will represent a lock on the cache so therefore three locks one word lock and two regular keypad locks. The Riddler is the first lock you must face, two riddles appear on the side of the cache. However, they are both misdirect and the real way to solve is a cipher magnetized and hidden on the bottom of the cache. If they figure out how to open the Riddler‘s word lock, they gain access to a small compartment on top where you will find two-face flipping his coin and a tethered magnet. The magnet is used to get a key with a joker key ring attached to it. This key unlocks the second lock leaving no other clues on how to find the third key. But going back and looking at two-faced flipping his corn which is a hint that you need to flip that panel and underneath hidden is the final key. 

 

image.thumb.jpg.dda3e7dd615ea6d369e8a86ecce9875d.jpg

 

Link to comment

From what you've described I'd say it'd be at least D4 and even D5 wouldn't be too unreasonable. What I often do when trying to settle on a rating (usually more for terrain than difficulty as my hides are usually pretty easy to spot once at GZ), is compare mine with others I've found that are half a star either side of what I'm thinking. Those half a star higher ought to be mostly tougher than mine and those half a star lower ought to be mostly easier, if not I'll adjust my rating accordingly.

  • Helpful 2
Link to comment

I guess I should have said, the terrain is definitely a 1 star. Its going in a rural area but its easily found right on a post next to the parking spot. I mean it will be like a dark camo green, just as extra protection against muggles.  I think you are right, might use a half star. 

Edited by Dan2099
Link to comment
12 minutes ago, Dan2099 said:

I guess I should have said, the terrain is definitely a 1 star. Its going in a rural area but its easily found right on a post next to the parking spot.

I mean it will be like a dark camo green, just as extra protection against muggles. 

 

Looks (to me) to be maybe a 4, maybe D5 gadget cache, that might be a 2 after folks notice screws holding it together if they "run outta time".

Curious... Is this on your own property?   A "post"?   Can you reach from a wheelchair?

Seems permission info to the Reviewer for a cache on a post I'd think would be interesting...   :)

  • Helpful 2
Link to comment
1 hour ago, cerberus1 said:

 

Looks (to me) to be maybe a 4, maybe D5 gadget cache, that might be a 2 after folks notice screws holding it together if they "run outta time".

Curious... Is this on your own property?   A "post"?   Can you reach from a wheelchair?

Seems permission info to the Reviewer for a cache on a post I'd think would be interesting...   :)

yeah I own the land and post. It’s part of my family farm. I’ll also probably strip the heads out of the screws, to be safe. Also it will be premium only. That seems to help them last longer I’ve found, but not always. 

Edited by Dan2099
  • Upvote 2
  • Helpful 1
Link to comment
14 hours ago, cerberus1 said:

Clayjar says difficulty is "How easy it is to find the cache", and that's it.   Getting to it/accessing it is Terrain.

For example, I have a huge camo rural mailbox on a tree limb.  You can see it approaching but need rope (or a ladder) to access. It's 1.5/5. 

I like clayjar because it's always been accurate for us and doesn't change, unlike some things do when the Help Center gets "updated". 

To view, see here.

 

Clayjar only has one question for the D rating, with five options corresponding to D1-D5. While it might be okay for traditionals, for multis or puzzles that have multiple legs or clues to the location, the minimum rating is D3 so it compresses the scale a lot on those. When I started in 2013, Clayjar was the official rating tool so pretty much every multi or puzzle was D3 or higher, which wasn't really helpful for separating the straightforward ones from the moderate mind-benders.

 

Likewise for terrain, Clayjar puts a lot of emphasis on bicycles. I can only imagine that where Clayjar lives they don't have steps, but in the coastal hinterland here, which is an ancient sandstone plateau at around 150 metres with many steep-sided gullies cut into it, unless the track is following a ridge it's almost certain to have lots of steps. For example, the cache I currently have in planning is a hike of about a kilometre with a climb from sea level to a vantage point about 100 metres up, and most of it is like this:

 

Steps.jpg.8caaac03d388cb6ace72996fe7dc58c0.jpg

 

So a pretty reasonable hiking trail, probably a terrain 3 given the elevation gain and distance, but not something I can easily relate to riding a bike on (or pushing a stroller for that matter). So no, I haven't found Clayjar to be particularly useful in rating my caches.

Edited by barefootjeff
  • Funny 1
  • Helpful 1
Link to comment

Similar to rock climbing ratings, there is a heavy degree of subjectivity. A terrain 4 in mountainous Pocatello Idaho compared to flat Houston Texas will probably be quite different. The guide mentioned earlier in the thread, found on the 'hide a cache' process is a great starting point but sometimes you just need to use local existing caches to compare and contrast how hard the new one you are putting out really is.

I wanted to make every difficulty and terrain option (a full fizzy grid) available in Bannock county where I live and it means a lot of them have tough puzzles or some sort of climbing to get to. It's a bit backwards to start with the rating instead of the puzzle but it's been fun. Some of my T5 caches get the best stories in the logs.

  • Funny 1
Link to comment

I've been thinking a lot about cache difficulty lately (not terrain which is a different attribute) because the official guidance is too qualitative in my opinion – e.g. 'easy', 'relatively easy', 'mild', etc. – and therefore based on individual experiences. Since I’m an analytical data nerd at heart, I think a better way would be to quantify cache difficultly as a quotient of finds to total play logs (excluding reviewer activity, notes, enables/disables, maintenance, etc.) where total play logs equals Finds plus DNFs. In this way the difficulty rating would be a ratio representing how often the cache is (or isn't) found on average.

For example, there's a D-5 close to our house hidden back in 2015. There are 85 total play logs (39 Finds, 46 DNFs) as of today including our own DNF we logged a few days ago. Some quick division tells us this cache (on average) is found only 46% of the time and DNF'ed the other 54%. If a cache is DNF'ed more often than it's found, I think a D-5 rating is appropriate.
 

By contrast, we found a D-1.5 the other day in less than 5 minutes. To date this cache has a 96% found rate (331 total play logs) which makes it a better candidate for D-1 difficulty in my opinion.
 

To exclude error from new hides, caches would need to have a significant number of play logs (let's say 10, for example) before a difficulty rating would generate, but this might be viewed as unpopular by some cachers. My most recent hide (9 play logs) has a 56% found rate so far which is surprising to me (I only gave it a D-2), but the log sample is likely too small yet to get a reliable rating.
 

Some basic statistics would be necessary to plot the distribution of difficulty ratios across all caches and normalize the universal ratings scale. One might assume caches would have a normal distribution clustered around D-2.5, but the data may not actually support this, so an alternate model might be more appropriate to determine the most representative difficulty scale.
 

For the record, I am not a statistician by trade, but what I’m proposing here any Statistics 101 undergrad would make quick work of. By no means is this a perfect system because logs can be deleted, not recorded, etc., but the point is to make difficulty ratings data-based (within a margin of error of course) and objective. This would certainly be more reliable than asking CO’s to simply guess their hide difficulty which is entirely subjective as is the consensus here.

Edited by savagedavages
formatting
  • Surprised 1
Link to comment

What would the difficulty rating be as of the time the cache was published, before 10 logs accumulate?  Many geocachers filter search results to exclude low difficulty caches, high difficulty caches, etc.

 

What happens if the ratio you described changes over time?  If the difficulty rating changes automatically based on log activity, the idea materially impacts anyone working on Fizzy Grids or any challenge caches based on D/T ratings.

 

Do you count repeat DNF's by a 0/0 account the same as a DNF by someone with 5,000 finds and 50 hides?  What about a string of five DNF's on the same day by a group of new geocachers, with finds the day before and the day after by experienced geocachers?  Because these things happen all the time.

  • Upvote 2
  • Helpful 2
Link to comment
2 hours ago, Keystone said:

Do you count repeat DNF's by a 0/0 account the same as a DNF by someone with 5,000 finds and 50 hides?  What about a string of five DNF's on the same day by a group of new geocachers, with finds the day before and the day after by experienced geocachers?

Does the DNF of someone who searched for less than 5 minutes count the same as someone who searched for more than an hour? What about the DNF of someone who searched for less than 5 minutes vs the Find of someone who took more than an hour to find the cache?

 

I enjoy difficult well-camouflaged hides, and I seek them out. But there have been times where knowing that the cache had a D4 rating allowed me to ignore dozens of hide locations that were too obvious for a D4 rating, and allowed me to find the cache faster.

 

Meanwhile, there are people who avoid caches with high difficulty ratings. Should the statistics include people who skip the cache, so they never log a DNF?

Link to comment
12 hours ago, savagedavages said:

I've been thinking a lot about cache difficulty lately (not terrain which is a different attribute) because the official guidance is too qualitative in my opinion – e.g. 'easy', 'relatively easy', 'mild', etc. – and therefore based on individual experiences. Since I’m an analytical data nerd at heart, I think a better way would be to quantify cache difficultly as a quotient of finds to total play logs (excluding reviewer activity, notes, enables/disables, maintenance, etc.) where total play logs equals Finds plus DNFs. In this way the difficulty rating would be a ratio representing how often the cache is (or isn't) found on average.

 

For cache types other than traditionals, there are other factors besides container camouflage that contribute to the D rating, and for puzzles in particular, that difficulty of solving won't result in DNFs, as people won't attempt the physical cache until they've solved the puzzle. Likewise for challenge caches where the D rating reflects the difficulty in qualifying for the challenge and someone not yet qualifying won't log a DNF as a result.

 

12 hours ago, savagedavages said:

To exclude error from new hides, caches would need to have a significant number of play logs (let's say 10, for example) before a difficulty rating would generate, but this might be viewed as unpopular by some cachers. My most recent hide (9 play logs) has a 56% found rate so far which is surprising to me (I only gave it a D-2), but the log sample is likely too small yet to get a reliable rating.

 

With my trickier hides, I've found the ratio of DNFs to finds tends to increase over time as the earlier finders are usually the more experienced cachers in the local community. The ones that come later on, in the months and years after publication, tend to be mostly visitors to the area (who would be less familiar with my hiding style) or new players with undeveloped geosenses. But even so, it's still a highly individual thing. My most DNFed cache (GC5H5G2 with 75 finds and 13 DNFs and which I've rated D2) is sitting in plain sight with no camo at all if you look in the right place, but its hidey hole is below people's line of sight and easily overlooked. The DNFers have typically thought they'd searched everywhere at GZ then widen their radius until eventually giving up, and those that have come back for a second successful attempt often can't believe they didn't see it the first time.

 

Also, how long do you wait before rating a cache? I have a multi published in January that's still only had three finds (and no DNFs) and one published in 2016 that's only had 9 finds (and one DNF due to muggles in the vicinity). Which raises another point, often the cause of DNFs is transient in nature, such as the presence of muggles or something else that interrupted a search; I had someone DNF one of mine because of a snake blocking the track. For someone else searching at a different time, it might be an easy find.

 

12 hours ago, savagedavages said:

Some basic statistics would be necessary to plot the distribution of difficulty ratios across all caches and normalize the universal ratings scale. One might assume caches would have a normal distribution clustered around D-2.5, but the data may not actually support this, so an alternate model might be more appropriate to determine the most representative difficulty scale.

 

There are far more easy caches out there than there are trickier hides, so globally the average difficulty is probably only a touch above 1.5. That also varies a lot by area, as in many places there's nothing but roadside traditionals whereas others have lots of hiking caches, puzzles or multis. Globally, micros predominate but in my region they're in third place behind smalls and regulars, so the caches here are likely harder to get to (the average terrain rating is higher than 2.5) but easier to spot once there.

 

12 hours ago, savagedavages said:

For the record, I am not a statistician by trade, but what I’m proposing here any Statistics 101 undergrad would make quick work of. By no means is this a perfect system because logs can be deleted, not recorded, etc., but the point is to make difficulty ratings data-based (within a margin of error of course) and objective. This would certainly be more reliable than asking CO’s to simply guess their hide difficulty which is entirely subjective as is the consensus here.

 

I disagree, I think COs, especially the more experienced ones, are much better placed to assess the difficulty of their hide in its environment than any algorithm based on global log statistics. I find the Help Centre table, with its suggested find/solve times, is a fair enough guide and over time I've been pretty happy with the D ratings I've chosen when creating my caches. One technique I often use is to compare the difficulty of my new hide with other caches in the region (is mine trickier or easier than those?) and use their D ratings to get a feel where mine should slot in. It's rare that I've felt the need to change the rating (D or T) post publication and when I have it's only ever been by half a star.

Edited by barefootjeff
  • Helpful 1
Link to comment
13 hours ago, savagedavages said:

've been thinking a lot about cache difficulty lately (not terrain which is a different attribute) because the official guidance is too qualitative in my opinion – e.g. 'easy', 'relatively easy', 'mild', etc. – and therefore based on individual experiences. Since I’m an analytical data nerd at heart, I think a better way would be to quantify cache difficultly as a quotient of finds to total play logs (excluding reviewer activity, notes, enables/disables, maintenance, etc.) where total play logs equals Finds plus DNFs. In this way the difficulty rating would be a ratio representing how often the cache is (or isn't) found on average.

For example, there's a D-5 close to our house hidden back in 2015. There are 85 total play logs (39 Finds, 46 DNFs) as of today including our own DNF we logged a few days ago. Some quick division tells us this cache (on average) is found only 46% of the time and DNF'ed the other 54%. If a cache is DNF'ed more often than it's found, I think a D-5 rating is appropriate.
 

I also judge a cache difficulty that way and have on more than one occasion mentioned in my log, that say 50% of logs are DNFS (also mention that a lot of people don't log DNFs, so almost certainly there are more), and so no way is that cache a 1.5D as it has been rated. More like say a 4D. I agree with what you wrote. That is a VERY GOOD way to judge difficulty; finds against DNFs and add a few more DNFs for the lazy and spineless who won't log them.

I will alter the rating of my caches after publication, depending on if people have difficulty finding the cache, or they have less difficulty than I thought they might.

Naturally the CO is likely to think their cache is easy to find, because they know where it is. Others don't have that insider information. More COs should think of that.

Link to comment
1 hour ago, Goldenwattle said:

I also judge a cache difficulty that way and have on more than one occasion mentioned in my log, that say 50% of logs are DNFS (also mention that a lot of people don't log DNFs, so almost certainly there are more), and so no way is that cache a 1.5D as it has been rated. More like say a 4D. I agree with what you wrote. That is a VERY GOOD way to judge difficulty; finds against DNFs and add a few more DNFs for the lazy and spineless who won't log them.

 

Maybe it's just me, but many of the DNFs on my caches have nothing to do with factors affecting the cache's D rating. One of my multis, a 2/2.5, has 4 DNFs from 36 finds but none of those DNFs have anything to do with either the concealment of the container or the difficulty of the field puzzle for working out the coordinates:

 

image.png.abbe28dc914b032f873f827471240489.png

 

That cache's D2 rating comes almost entirely from the field puzzle, the hide itself would have been a D1 had it been a traditional, but none of those DNFs were due to the field puzzle either, they were all attributed to the nearby camper (a tent some 20 or 30 metres from GZ), the weather, a spider or a lack of time before the train home.

 

On the other side of the coin, 31 of my hides (excluding events) have had no DNFs. Should they all be rated D1?

Edited by barefootjeff
  • Surprised 1
  • Helpful 1
Link to comment
15 hours ago, barefootjeff said:

 

Maybe it's just me, but many of the DNFs on my caches have nothing to do with factors affecting the cache's D rating. One of my multis, a 2/2.5, has 4 DNFs from 36 finds but none of those DNFs have anything to do with either the concealment of the container or the difficulty of the field puzzle for working out the coordinates:

 

image.png.abbe28dc914b032f873f827471240489.png

 

That cache's D2 rating comes almost entirely from the field puzzle, the hide itself would have been a D1 had it been a traditional, but none of those DNFs were due to the field puzzle either, they were all attributed to the nearby camper (a tent some 20 or 30 metres from GZ), the weather, a spider or a lack of time before the train home.

 

On the other side of the coin, 31 of my hides (excluding events) have had no DNFs. Should they all be rated D1?

With my caches, it's rarely like that. They simply couldn't find it.

Link to comment
4 hours ago, Goldenwattle said:

With my caches, it's rarely like that. They simply couldn't find it.

 

Yes, exactly, that's why an algorithmic method for rating cache difficulty based on DNF-to-find ratio would mostly be a "one size fits none". By way of further example, I just got back from doing a routine check on one of my older caches, a field puzzle mystery (GC61HCN) published in August 2015 and rated difficulty 2.5. Part of that difficulty is for the puzzle element and part is for the placement which is inside a reasonably large honeycombed cave. It's a bit of a search at GZ but the description and hint limit it to a relatively small area and the container is a decent size (a 400ml Sistema) with just a small rock providing some camo. To date it's had 32 finds, received 14 FPs and has yet to have any DNFs. I wonder what savagedavages proposed algorithm would make of that?

  • Upvote 2
Link to comment
14 hours ago, Goldenwattle said:

I don't agree with that comment at all. Most caches these days are not in remote bushland and most get more finds than yours do and the system would work better.

 

All I'm saying is there are other factors besides container concealment that can contribute to the D rating, especially for non-traditional caches which globally make up a quarter of all hides. Those factors, like puzzle difficulty, the number of stages preceding the final, the complexity or time needed to complete an EarthCache or virtual task, or even those traditionals where there's some manipulation required to access the log once the container is spotted, don't contribute to DNF logs. People generally don't DNF puzzles they can't solve, multis that take too much time to attempt, ECs that look too complicated or challenge caches they don't qualify for, yet all those things are supposed to be taken into account in the D rating. How is your DNF-to-find ratio algorithm supposed to rate those?

 

Conversely, there are plenty of things that can lead to a DNF log that aren't related to a cache's D-rating, the most common being muggles in the vicinity of the cache. The other big factor is a searcher's prior experience and that's entirely an individual thing. In the USA, LPCs are ubiquitous but for someone who's never seen one before and isn't aware the covers on lamp post bases can be lifted up, they'd be pretty daunting. I have a lot more trouble finding "easy" caches when away on holidays because hiding styles are often different in different areas. When I first started, I DNFed a bison tube on a fence because I had no idea what a bison tube was and had a mental picture of something perhaps resembling a bison's horn. Once I'd done a bit of online research and discovered what they are, I went back and immediately spotted it. A lot of the time we look for what we expect to see, not what's really there.

 

Sometimes a high D rating can mean the searchers come prepared to make the extra effort needed to get their signature in the log and end up succeeding even if it takes a lot longer than a P&G. Is a challenging D3 cache with no DNFs really such a bad thing?

 

I'm curious that you say there are lots of D1.5 caches that have 50% DNF logs. Why isn't the CHS pinging those? Isn't that supposed to be a sure sign the cache is missing and needs to be either fixed or archived? Are CHS pings no longer being followed up by reviewers? But maybe the problem that really needs to be addressed is the default D/T setting of 1.5/1.5 on the cache submission page. It forces you to select a cache size but allows you to just completely skip over the D/T rating if you can't be bothered changing it, or even realise it's meant to be set to something other than the default if your cache isn't just a quick and easy roadside find.

 

Edited by barefootjeff
  • Upvote 1
  • Helpful 2
Link to comment
2 hours ago, barefootjeff said:

I have a lot more trouble finding "easy" caches when away on holidays because hiding styles are often different in different areas.

A lot of caches with low difficulty ratings are considered "easy" because they're in "the usual spot". That's great if you're experienced with that type of hide and know where the usual spot is. But as you've pointed out, it doesn't help geocachers who are new to that type of hide (or who are just new to geocaching in general). Any hide style that has a name (or worse, an acronym) is likely to fall into this category.

 

2 hours ago, barefootjeff said:

A lot of the time we look for what we expect to see, not what's really there.

Some of the caches that took me the longest were hidden near "the usual spot", but didn't use "the usual spot". One was hidden near several "usual spots", but didn't use any of them. It can take experienced geocachers time to let go of where they "know" the cache must be hidden and start searching the rest of GZ.

 

3 hours ago, barefootjeff said:

Is a challenging D3 cache with no DNFs really such a bad thing?

One of my favorites was a D4 hidden-in-plain-sight cache. The description warns, "If you don't care for EVIL hides, you may wish to pass on this one!" It got a LOT of DNF logs. But this was a T1 traditional cache in a suburban park with a bit of a reputation, so a lot of geocachers wanted to drop by to try it out. Other D4 caches that I've found haven't gotten nearly as many DNF logs.

Link to comment

It being a wet day, I decided to do a bit of data analysis so I looked at all my finds on D3 traditionals that are still active, of which there are 35, and calculated the ratio of DNFs to finds for each one. Here's a scatter plot of the result:

 

image.png.2a85a3153cba5c554f39d802d809c4f3.png

 

So what's the CORRECT ratio of DNFs to finds for a D3 traditional? Even if you took an average or median it'd be pretty meaningless given the spread of results.

Edited by barefootjeff
  • Upvote 1
Link to comment
9 hours ago, Goldenwattle said:

Likely not because then someone finds it.

 

My understanding is a find doesn't clear the CHS, it just improves the health score a little. I once got a CHS ping on a 6-week old 2/5 multi that had one find and one DNF. The explanation I was ultimately given was that the 100% DNF to find ratio was considered excessive for its cache type and D/T rating and it needed to get more finds. I'd imagine that if a 1.5/1.5 traditional is getting 50% DNFs the CHS would be jumping up and down regardless of what the most recent log is. More recent reports on the forum suggest that it does if the cache is under-rated for the frequency of DNFs it gets.

Edited by barefootjeff
Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...
×
×
  • Create New...