cache rating feedback (related to handicap accessibility topic)

[+Geo-Ferrets]

After bringing a physically limited friend along on a hunt for a cache rated 2.5/2 that turned out to be a 1/3 according to the GeoCache Rating System (http://www.clayjar.com/gcrs/), I've been putting some thought into how the community can protect itself from rating misunderstandings. This is directly related to the current discussion about a handicap icon, since more reliable ratings will assist in deciding whether a cache is appropriate to visit.

 

The solution I've come up with has two parts:

 

1. When reporting a new cache, have the hider fill out the GCRS questionnaire, rather than just selecting arbitrary numbers from a drop list. This will make the hider pay more attention to various aspects of accessibility, resulting in a more accurate initial rating.

 

2. When logging a find, give the finder the option of filling out the same questionnaire. Then use that information to recalculate the rating as the average of the ratings provided by the hider and all finders.

 

This automatic feedback system would change the cache rating from a static value specified by the hider into a dynamic value that reflects the consensus of the community. The result being a rating system that is more reliable and therefore more useful to all geocachers, especially those with physical limitations.

 

So what do you all think?

 

Dave

[+Sissy-n-CR]

Seems like a good idea.

 

However, the rating system looks at hard you think it is.

 

If I were to implement a finder's questionaire I'd be asking different questions based on his actual experience.

 

For instance:

Instead of asking what he thinks the difficulty is, ask how long did it take them to find the cache. I think most people will be surprised that most caches, IMHO, are a difficulty of only 1, maybe 1.5, very rarely over 2. Of course, we've found caches in 2 minutes where multiple others have logged DNFs on.

Instead of asking what rating he would give the terrain, I'd ask how far did he hike and what the terrain what like i.e. flat, hilly, smooth, unimproved, had to bushwach, etc.

I'd ask if he had to use any gear beyond the normal GPS, compass, map, printout stuff that most people take with them.

 

Some of the rating parts of very much subject, especially the how long will it took for someone to find the cache--everybody is different. That's why I'd ask concrete questions, "how long from being onsite did it take you to find the cache?" "From where you parked, using your GPS's trip odometer, what was the distance?" Stuff like that.

 

This way, we'd discover how hard it actually was for someone and not what they think of how hard it should have been.

 

All of the above aside, I use a rating system of basically asking how they enjoyed the cache and give them an opportunity to provide anon (if they want), direct feedback. I ask to be perfectly honest as it will help me improve the caches. If you want to check them out HERE is the link to our caches. If you want to try one out, do "Russellville Six," it doesn't get much traffic, just rate it about the same as the average it already has.

 

I ask for anon feedback as it will generally elicit more helpful responses.

 

CR

 

[+Markwell]

When the Clayjar system was being discussed in length many moons ago (June of 2001?), we talked about whether or not people should mandate people filling out the form and have the system generate the numbers, or to do it as we do now. I was a BIG supporter for having the form as a tool, not mandating the form.

 

The reason is that the system can be off. I have a four-stage multi cache that every part of the cache could be found by someone in a wheel chair. However, the distance of the trail is a little over a mile. According to the rating system, it would be a 2 or 3 terrain. I have to mitigate that with some common sense, and bring it down to a 1.5 terrain with a comment about the distance involved.

 

As far as the seekers filling out the terrain and difficulty and having that be averaged, if I have a 5/5 cache or even 4.5/4.5, and someone doesn't know how to fill out the form and gives it a 1/1 (or just wants to "mess" with the cache because they didn't find it), I wouldn't want their numbers to drag down my 4.5/4.5. The ONLY way I would support it would be if the numbers of the seekers were displayed differently, i.e. "Cache hider rates this as 4.5/4.5, Cache seekers rate this an average of 3.25/4.75."

 

Once you start going that direction, you have to start weighing the complexity of implementation vs. the perceived benefit. Most people rarely look at the current terrain/difficulty ratings. I've seen so many cachers (myself included) that got to a cache and couldn't find it, and posted "I didn't realize that I would need a boat to get to this cache."

 

Nice idea, but I'm just not sure how practical it is.

 

Markwell

Chicago Geocaching

[+Mopar]

I gotta go with Markwell here. I wouldn't want one mistake or prankster throwing off my cache rating.

Besides, read the cache logs. How many people write 5 word logs? Those same people would just click thru 1/1 if you made them rate the cache.

Now, CR's "Rate-A-Cache" idea on his cache pages I LOVE. That could be expanded to let the finders SUGGEST a rating to the cache hider. Even as it is set up now, I think it's a great idea, and CR should probably post a new thread explaining how it's done.

 

Tae-Kwon-Leap is not a path to a door, but a road leading forever towards the horizon.

[+Geo-Ferrets]

quote:

---

Originally posted by Markwell:

I was a BIG supporter for having the form as a tool, not mandating the form.

---

 

Fair enough. Requiring the form is the least important part of my suggestion. The real key is the community feedback.

 

quote:

---

Originally posted by Markwell:

According to the rating system, it would be a 2 or 3 terrain. I have to mitigate that with some common sense, and bring it down to a 1.5 terrain _with a comment about the distance involved_.

---

 

I see your point, but no rating system is without issues like this. And is how you handle it really so different from having the system rate it a 2 or 3 and including a comment about the terrain and distance?

I'd also point out that wheel chair access is not the only aspect of accessibility. My friend isn't in a wheel chair, but she would have difficulty with the mile of trail on many days. For her case, a 2 or 3 would be an accurate rating.

 

quote:

---

Originally posted by Markwell:

As far as the seekers filling out the terrain and difficulty and having that be averaged, if I have a 5/5 cache or even 4.5/4.5, and someone doesn't know how to fill out the form and gives it a 1/1 (or just wants to "mess" with the cache because they didn't find it), I wouldn't want their numbers to drag down my 4.5/4.5.

---

 

Remember, this is a running average. That 1/1 would only knock you down to a 3/3, and then only if it was the FIRST visitor to rate the cache. As others visited the cache, the 1/1 would become less and less significant as the average converges on the community assessment of your cache, whether that be 4/4, 5/5, or 6/6.

 

quote:

---

Originally posted by Markwell:

The ONLY way I would support it would be if the numbers of the seekers were displayed differently, i.e. "Cache hider rates this as 4.5/4.5, Cache seekers rate this an average of 3.25/4.75."

---

 

Fine by me. That was actually my original idea, but as I was writing that post I decided that a single rating was a simpler solution. But I've got no problem with having a dual rating.

 

quote:

---

Originally posted by Markwell:

Once you start going that direction, you have to start weighing the complexity of implementation vs. the perceived benefit. Most people rarely look at the current terrain/difficulty ratings.

---

 

That's one reason I went with a single rating, because it should be easier to implement. But I would strongly disagree with your assessment of the benefit. I'd say that the amount of attention people pay to the ratings depends on how important the ratings are to them. Those of us who need to make certain our party only attempts a 1 or 2 terrain look very closely indeed.

As for those who don't look at the ratings... If it causes them problems, they'll learn to look. And they'll be more likely to look if the ratings are reliable and consistent.

 

Dave

[+Sissy-n-CR]

Both Markwell and Mopar have very valid points.

 

Possibly, the function could be optional, which would make sense to me as some cachers simply might not pay attention to such things. No need to force them to make decisions based on faulty memory.

 

Second, maybe it should really be a tool to help the owner know if his thinking parallels the real world experiences of the finders. For instance, he could call up a page that has the logs of which the finders have put in a rating, there are check boxes to eliminate ones that are obviously out of whack. Then with a click of a button, he could get an averaged readings of the finder's experiences.

 

I wouldn't want to let the "official" rating of a new cache be determined by a few geophyte cachers taking all day to complete it and really throwing the ratings off.

 

I do like feedback, the more honest the better.

 

I'll post how I did the ratings thing on my pages soon.

 

[+Geo-Ferrets]

quote:

---

Originally posted by Sissy-n-CR:

If I were to implement a finder's questionaire I'd be asking different questions based on his _actual_ experience.

---

 

That's a good suggestion. I was sticking with the existing questionnaire in order to make implementation as easy as possible, since it already exists. But I suppose most of your changes would be just minor wording changes in the questions.

 

quote:

---

Originally posted by Sissy-n-CR:

All of the above aside, I use a rating system of basically asking how they enjoyed the cache and give them an opportunity to provide anon (if they want), direct feedback. I ask to be perfectly honest as it will help me improve the caches.

---

 

That's really neat! I'm glad to see you have that feedback system. However, it doesn't really address the issue that I'm looking at. Your system lets people give their overall opinion of the cache, which is nice to have. What I'm looking for is a more consistent and reliable difficulty and terrain rating. For those of us who need to pay careful attention to difficulty and terrain for our group, an overall opinion of the cache is poor guidance.

 

Dave

[+Geo-Ferrets]

quote:

---

Originally posted by Mopar:

Besides, read the cache logs. How many people write 5 word logs? Those same people would just click thru 1/1 if you made them rate the cache.

---

 

As I mentioned, the log rating would be an option. That way those people who can't be bothered would skip it entirely.

 

Dave

[+Geo-Ferrets]

quote:

---

Originally posted by Sissy-n-CR:

Possibly, the function could be optional, which would make sense to me as some cachers simply might not pay attention to such things. No need to force them to make decisions based on faulty memory.

---

 

As I originally suggested...

 

quote:

---

Originally posted by Sissy-n-CR:

Second, maybe it should really be a tool to help the owner know if _his_ thinking parallels the real world experiences of the finders. For instance, he could call up a page that has the logs of which the finders have put in a rating, there are check boxes to eliminate ones that are obviously out of whack. Then with a click of a button, he could get an averaged readings of the finder's experiences.

 

I wouldn't want to let the "official" rating of a new cache be determined by a few geophyte cachers taking all day to complete it and really throwing the ratings off.

---

 

The only problem with an owner tool is that it requires owner action before it is useful for cache browsers. That's why I was suggesting an automated system. And because it's an average, the few geophytes would have limited influence on the results.

 

However, perhaps Markwell's suggestion of a dual rating would be the best compromise. Then it would be available as a tool for the owner without automatically overriding the owner's rating, and yet still be available to everyone else as an automated system.

 

So... When can we have it?

 

Dave

[+Sissy-n-CR]

I'd probably add a break down of the ratings just in case there are oddball entries. You could mentally remove them to get a more accurate assessment.

 

CR

 

[+Geo-Ferrets]

Thanks to everyone for your thoughtful replies. Special thanks to CR for his "Rate-A-Cache", which inspired me to put together a working prototype for what I'd like to see.

 

Have a look at what I came up with...

 

Dave

[+Tsegi Mike and Desert Viking]

As someone with physical limitations, I find a description on the cache page more useful than any rating system. Any description that tells me if there is climbing, or the road is rough or is flat level terrain helps me decide if I am able to do it or not. Many people rate their caches inconsistently. What seems easy terrain to someone physically fit may be Mount Everest to me.

 

That moss-covered bucket I hailed as a treasure,

For often at noon, when I returned from the field,

I found it the source of an exquisite pleasure.

 

Samuel Woodworth The Old Oaken Bucket

[usmcski]

I agree with the need for better descriptions ....I took my nephew with me once...he has spinal muscular atrophy and is confined to an electric wheelchair.We did one cache and it was rater 2/2...should have been rated 1/1/...was so simple my other 10 year old nephew found it without using a GPS...the next one was a 1/1.5...the terrain looked great till we got halfway across a field and he got his chair stuck in some mud.

[+Sissy-n-CR]

quote:

---

Originally posted by Geo-Ferrets:

Thanks to everyone for your thoughtful replies. Special thanks to CR for his "Rate-A-Cache", which inspired me to put together a working prototype for what I'd like to see.

---

 

That's pretty cool!

 

One of these days I'm gonna figure out how to translate the ratings of my Rate-a-Cache to a .jpg file. That way I can call CGxxxxrating.jpg directly from the cache page and display the rating. If you can figure out a similar scheme, you wouldn't have to worry about IFRAMEs.

 

CR

 

[+parkrrrr]

quote:

---

Originally posted by Sissy-n-CR:

One of these days I'm gonna figure out how to translate the ratings of my Rate-a-Cache to a .jpg file. That way I can call CGxxxxrating.jpg directly from the cache page and display the rating

---

It's actually pretty easy if you have netpbm and freetype installed on your server. I have some Perl code I'd be willing to share, for what it's worth. Below, for example, is such an image:

 

[+Geo-Ferrets]

quote:

---

Originally posted by Sissy-n-CR:

One of these days I'm gonna figure out how to translate the ratings of my Rate-a-Cache to a .jpg file. That way I can call CGxxxxrating.jpg directly from the cache page and display the rating. If you can figure out a similar scheme, you wouldn't have to worry about IFRAMEs.

---

 

Yeah, I'd started to do it that way, but decided I'd rather get the functionality going than spend time mucking about with images and fonts and such. But as you say, "One of these days I'm gonna..."

 

Dave

[+Web-ling]

The biggest problem I see with the ClayJar Geocache Rating System is that there are too many variables. I always use the system, then tweak the numbers. I've had instances where a cache was severely over or underrated using the system, because there were factors the system doesn't take into consideration.