Jump to content

New Cache Rating System


addisonbr

Recommended Posts

We will complete our first attempt at a rating system by the end of the month. In lieu of a generic (and IMO unhelpful) 1-5 star rating system we have decided to let people rate a percentage of their finds as their favorites.

 

This favorite count will then be displayed on the cache listing. The beta feature will initially be introduced to Premium Members but will eventually opened up to all users.

 

We will also be exploring other ways to identify the best caches to encourage quality. Quality of the game is one of the major themes for 2011.

Link to comment

and once again: interesting timing :huh:

interesting due to the subject from a GPS manufacturer's new web site. ( I think that is vague enough not to get removed) ??? <_<

+1 for competition

I have to ask would we get it without?

 

Since Groundspeak had said on the Feedback site some time ago that the rating system was under review, and then it went into planning, I would say yes. These things don't happen overnight. I daresay the same goes fro the API and all the other "coincidences". I rather doubt internal discussions, design, coding, testing, tweaking, and implementation can be done within a couple of days just because another site came online. In my 30 years in IT, the only code I've seen cranked out that fast was junk code you wouldn't want in production.

Link to comment

and once again: interesting timing <_<

interesting due to the subject from a GPS manufacturer's new web site. ( I think that is vague enough not to get removed) ??? :)

+1 for competition

I have to ask would we get it without?

- 1 for repetition .... I've looked and here in the Fort Worth/Dallas area the ones they have posted arn't original caches.... Just GC.com caches with there codes. Several I have logged already, thats how I know that they are not new and just copies. :D

 

I just bought a new GPSr I hope Groundspeak doesn't drop support for there recivers. :huh:

 

Wish this thing had a spell check

Edited by sdrawkcab
Link to comment
personally i don't think a simple favorite count is gonna cut it.

I think a favorite count will correlate with quality, but that it will also correlate with total finds. It will be interesting to see how that plays out in practice. My best guess is that the highest rated caches in most areas will be the easiest caches, that don't suck. Which is useful in its own way, but of course gives different results than other ratings might.

 

I have a couple of notable caches in a nearby park with waypoints very close to each other. One gets found a LOT. The other is a multi that requires a minimum 4-6 hour investment and gets found about 10% as often.

 

The ratings on GCVote are interesting. The former has a lower average score (about 3.3), with a little less than a quarter of the votes being 4s and 5s. The latter has a perfect 5 for average score. But the *total* votes for that cache are still less than half of the 4s and 5s voted for the former. Which is of course a phenomenon driven almost entirely by their relative find rates.

 

In this particular case, I think the 1-5 rating gets it right, rating the multi much higher. The 'favorite count' method on its own might suggest that the other cache is superior. I don't think it means that every micro or guardrail cache at a rest stop will get rated high just because a lot of people find it. But I think it could turn into a list of 'easy caches, that don't suck'.

 

Regardless, I'm looking forward to taking it for a spin.

Link to comment

I find the last statement most interesting. We knew they where planning some sort of rating system. Although, I think they are rushing it out now, we were getting one anyway. " Quality of the game is one of the major themes for 2011." Is a very interesting statement to me. I can only hope they are trying to figure out a way to limit PNG's (whether they are power trails, LPC or other parking lot type hides). I think that they are not just poor caches, but have a higher likelyhood to cause problems with landowners, LEO and businesses and prompt regulations/restrictions that will hinder geocachings future.

Link to comment

I find the last statement most interesting. We knew they where planning some sort of rating system. Although, I think they are rushing it out now, we were getting one anyway. " Quality of the game is one of the major themes for 2011." Is a very interesting statement to me.

 

I also found that theme interesting. Rather than single out certain types of caches which some might rate as lower quality than other types, I wonder one other topics in the Feedback system they're considering to improve the quality of the game. I think we also saw a few the the quality issues related to the web site when the souvenirs feature was launched. That prompted someone to submit a Feedback issue on making a beta test site available so new features such as souvenirs or a cache rating system could be tested but a smaller, selected group of users before they're put into production. The topic had quite a few votes that last time I looked.

 

What other features should GS consider to "improve the quality of the game"?

Link to comment

I find the last statement most interesting. We knew they where planning some sort of rating system. Although, I think they are rushing it out now, we were getting one anyway. " Quality of the game is one of the major themes for 2011." Is a very interesting statement to me.

 

I also found that theme interesting. Rather than single out certain types of caches which some might rate as lower quality than other types, I wonder one other topics in the Feedback system they're considering to improve the quality of the game. I think we also saw a few the the quality issues related to the web site when the souvenirs feature was launched. That prompted someone to submit a Feedback issue on making a beta test site available so new features such as souvenirs or a cache rating system could be tested but a smaller, selected group of users before they're put into production. The topic had quite a few votes that last time I looked.

 

What other features should GS consider to "improve the quality of the game"?

 

I didn't single out those types of caches because they are lower quality( even if they are), but because of the other reasons I stated. Which complies to the "improve the quality of the game" statement. I think it complies for the lower quality comment as well, but we've all seen the numerous threads arguing both sides of that. I'm focusing on the other reasons.

Link to comment

We will also be exploring other ways to identify the best caches to encourage quality. Quality of the game is one of the major themes for 2011.

 

I am so pleased to hear this.

 

And yes, the timing is very interesting.

 

(edited for tagfail)

Edited by narcissa
Link to comment
We will complete our first attempt at a rating system by the end of the month. In lieu of a generic (and IMO unhelpful) 1-5 star rating system we have decided to let people rate a percentage of their finds as their favorites.

 

This favorite count will then be displayed on the cache listing. The beta feature will initially be introduced to Premium Members but will eventually opened up to all users.

 

We will also be exploring other ways to identify the best caches to encourage quality. Quality of the game is one of the major themes for 2011.

 

Why are we dumbing down the site?

Link to comment
We will complete our first attempt at a rating system by the end of the month. In lieu of a generic (and IMO unhelpful) 1-5 star rating system we have decided to let people rate a percentage of their finds as their favorites.

 

This favorite count will then be displayed on the cache listing. The beta feature will initially be introduced to Premium Members but will eventually opened up to all users.

 

We will also be exploring other ways to identify the best caches to encourage quality. Quality of the game is one of the major themes for 2011.

 

Why are we dumbing down the site?

 

How does a favorites list dumb down the site?

 

I see it as alerting hiders to quality caches and rewarding those hiders that produce quality. Hopefully the outcome will be more quality hides.

Link to comment
How does a favorites list dumb down the site?

 

I see it as alerting hiders to quality caches and rewarding those hiders that produce quality. Hopefully the outcome will be more quality hides.

the problem is that a favorite count is a totally meaningless number.

 

a cache that's been favorited 10 times and has 10 finds would be highly "awesome", but another cache that's been favorited 10 times and has been found 100 times is probably gonna be somewhat lame.

Link to comment
the problem is that a favorite count is a totally meaningless number.

 

a cache that's been favorited 10 times and has 10 finds would be highly "awesome", but another cache that's been favorited 10 times and has been found 100 times is probably gonna be somewhat lame.

I'm not sure that it's totally meaningless. I suspect it will correlate both with quality and with quantity.

Link to comment
How does a favorites list dumb down the site?

 

I see it as alerting hiders to quality caches and rewarding those hiders that produce quality. Hopefully the outcome will be more quality hides.

the problem is that a favorite count is a totally meaningless number.

 

a cache that's been favorited 10 times and has 10 finds would be highly "awesome", but another cache that's been favorited 10 times and has been found 100 times is probably gonna be somewhat lame.

 

You're assuming that you know that it won't be weighted.

Link to comment
How does a favorites list dumb down the site?

 

I see it as alerting hiders to quality caches and rewarding those hiders that produce quality. Hopefully the outcome will be more quality hides.

the problem is that a favorite count is a totally meaningless number.

 

a cache that's been favorited 10 times and has 10 finds would be highly "awesome", but another cache that's been favorited 10 times and has been found 100 times is probably gonna be somewhat lame.

 

Pehaps they plan to use these numbers to provide some kind of "favorites index" (FI). Simply dividing the number of finds by the number of favorites, over time, would provide a simple form of such an index. The 10 for 10 would rate a 100 and the 10 of 100 would rate a 10. You could then put a criteria selection in the PQs to pick only caches with an FI of, say, 75-100.

Link to comment
I'm going to wait and see the actual implementation before I form an opinion. I know, crazy thought.

 

I'm optimistic. I think it could really make a change for the good and the data will actually come from the community itself. We would have only ourselves to blame.

You are such a loose cannon!! :huh:
Link to comment

My own list of favorites is already out there for all to see.

 

What other features should GS consider to "improve the quality of the game"?

 

I'd be in favor of limiting the number of times someone can post to the same forum thread over a period of time. Too many threads become useless when they devolve into a petty spat between a very small number of posters.

Edited by wimseyguy
Link to comment
I'm going to wait and see the actual implementation before I form an opinion. I know, crazy thought.

Perfectly reasonable, although if there's a chance opinions can influence what gets rolled out in a few weeks I wouldn't mind being a part of the conversation.

 

(I have no idea if there's a chance opinions can influence what gets rolled out in a few weeks, but, maybe.)

Link to comment

Since Groundspeak had said on the Feedback site some time ago that the rating system was under review, and then it went into planning, I would say yes. These things don't happen overnight. I daresay the same goes fro the API and all the other "coincidences". I rather doubt internal discussions, design, coding, testing, tweaking, and implementation can be done within a couple of days just because another site came online. In my 30 years in IT, the only code I've seen cranked out that fast was junk code you wouldn't want in production.

+1

Link to comment
How does a favorites list dumb down the site?

 

I see it as alerting hiders to quality caches and rewarding those hiders that produce quality. Hopefully the outcome will be more quality hides.

the problem is that a favorite count is a totally meaningless number.

 

a cache that's been favorited 10 times and has 10 finds would be highly "awesome", but another cache that's been favorited 10 times and has been found 100 times is probably gonna be somewhat lame.

 

Yes, it would be better expressed as a ratio of total finds.

Link to comment

"Rate a percentage of their finds as favorites." Hmmm... Wonder what the percentage will be? 1 out of 10? 1 out of 25? 1 out of 100? Time to remind me that I've found another 100 caches and can pick one as a favorite? Back-rating six years of finds? Well worth the effort, but it will take time. Can I mark as a favorte an archived cache? Hmmm... Which are my 300 favorite caches?

Hey! There will be problems, of course, but I like the idea!

Link to comment

"Rate a percentage of their finds as favorites." Hmmm... Wonder what the percentage will be? 1 out of 10? 1 out of 25? 1 out of 100? Time to remind me that I've found another 100 caches and can pick one as a favorite? Back-rating six years of finds? Well worth the effort, but it will take time. Can I mark as a favorte an archived cache? Hmmm... Which are my 300 favorite caches?

Hey! There will be problems, of course, but I like the idea!

Of course, there's also the fact that cache quality isn't static. A cache that was awesome when found a while back may have devolved into trash. Those dozen people who marked it as a fave will never know that it's no longer awesome, so future cachers will continue to be led by bad info.

Link to comment

A favourites-list thing can yield results similar to the GCVote 1-5 scale. How about a percentage scale? For example, if I've visited 100 caches and put 16 of those on my favourites list, then the system could give each of those favourites 92% (= 100-16/2) of a vote from me, and each of the non-favourites get 42% (= (100-16)/2) of a vote from me. Take the average votes from each cacher to get the score.

 

The nice thing about this scheme is that the size of your favourites list is entirely up to you.

Link to comment
the problem is that a favorite count is a totally meaningless number.

 

a cache that's been favorited 10 times and has 10 finds would be highly "awesome", but another cache that's been favorited 10 times and has been found 100 times is probably gonna be somewhat lame.

As others have pointed out, you're assuming that the favorite count won't be weighted.

 

But even if it isn't weighted, the number isn't completely meaningless. If everyone were to assign their favorites randomly, then you would expect roughly one favorite for every 20 finds. So a cache with roughly one favorite for every 10 finds is still above average. However, I would expect most people not to assign their favorites randomly, so a cache with any favorites is likely to have some redeeming quality for someone.

 

The question is whether that redeeming quality is something that you value. If I mark a micro-cache with clever 4-star camouflage as a favorite, is that really going to matter to someone who's looking for larger caches with "treasure" for the kids, or to someone who's looking for easy finds at the end of difficult hikes, or to someone who's looking for something else entirely?

Link to comment
"Rate a percentage of their finds as favorites." Hmmm... Wonder what the percentage will be? 1 out of 10? 1 out of 25? 1 out of 100?
This isn't definitive, but a comment made in passing by Jeremy on the feedback site makes it sound like it might be 1 in 10:
Jeremy

@luzian I don't think this will be a problem. Since people are logging 1 out of 10 finds as their favorite, even the "good" caches will have at least a point or two. But this is the first attempt at improving the quality of the game.

Link to comment
the problem is that a favorite count is a totally meaningless number.

 

a cache that's been favorited 10 times and has 10 finds would be highly "awesome", but another cache that's been favorited 10 times and has been found 100 times is probably gonna be somewhat lame.

As others have pointed out, you're assuming that the favorite count won't be weighted.

 

But even if it isn't weighted, the number isn't completely meaningless. If everyone were to assign their favorites randomly, then you would expect roughly one favorite for every 20 finds. So a cache with roughly one favorite for every 10 finds is still above average. However, I would expect most people not to assign their favorites randomly, so a cache with any favorites is likely to have some redeeming quality for someone.

 

The question is whether that redeeming quality is something that you value. If I mark a micro-cache with clever 4-star camouflage as a favorite, is that really going to matter to someone who's looking for larger caches with "treasure" for the kids, or to someone who's looking for easy finds at the end of difficult hikes, or to someone who's looking for something else entirely?

 

The people that are actually interested in a cache rating will just have to read the cache pages on caches that are rated higher, or hopefully it is integrated into the query filters and you can filter rating and whatever, size, hike etc.. I do that now, I hope the new system makes it easier. The people that don't care won't be affected.

Link to comment

Hmm. I like the idea of a cache rating system. I was hoping for something like a general 1-5 star rating but I'll play along with it because at least it's something.

 

I'm interested to see how this will work with the different areas (high cacher density vs. low cacher density). Like here in the boonies. A neato cache might be exceptionally in the boonies and all 5 of us locals have found it but it's going to look bad because only the 5 of us thought it was cool versus a cache in a higher cacher dense area where it might not be cool but because it's the coolest of the available options 100 people might like it.

 

Then I also wonder about the folks who don't go after just any cache and end up with the vast of majority of their caches found being awesome with a few duds mixed in. Limiting which ones they can essentially rate means some awesome caches won't get a vote they deserve. Which is again why I favored the 1-5 system. Then if all one finds are awesome caches (with a few duds) then they appropriately rate those awesome caches. Especially if it's a cacher who goes to remote caches or difficult terrain caches which may get visited once in a long while.

Link to comment
Then I also wonder about the folks who don't go after just any cache and end up with the vast of majority of their caches found being awesome with a few duds mixed in. Limiting which ones they can essentially rate means some awesome caches won't get a vote they deserve.
In the case of someone extremely selective who goes after only the best caches, their favorites list is going to give you the best of the best.

 

If someone gives every cache a top rating (whether that is 5 of 5 stars, or adding it to a favorites list), then that doesn't give me any new information that I wouldn't get from a list of their finds. Maybe they really are seeking only the cream of the crop. Maybe they're just extremely undiscerning. The only way to tell is to evaluate the information that other people have provided.

Edited by niraD
Link to comment

I like the idea of favorites much better than assigning an arbitrary number. It is more of a positive approach and will mean more to me. I play another location-based game that started out with a rating system but is dropping it because it has not proven to be very useful. Of course I just realized that I aleady have 72 local favorites listed in my bookmark, not to mention the ones outside of my area that are listed in my profile. It might be hard to pick and choose.

 

In any event I am intrigued by Groundspeak's commitment to making 2011 the year of quality. Nothing but the best in lamp post hides, with decibel ratings to take into account the squeak factor? Or something that would move the game beyond dropping a cache on private property simply because it is 528 feet from the last one?

Edited by mulvaney
Link to comment
The question is whether that redeeming quality is something that you value. If I mark a micro-cache with clever 4-star camouflage as a favorite, is that really going to matter to someone who's looking for larger caches with "treasure" for the kids, or to someone who's looking for easy finds at the end of difficult hikes, or to someone who's looking for something else entirely?

I think in these situations you'd be best-served by comparing the ratings for the similar types of caches you're interested in.

 

Like on Yelp. My wife and I happen to love dive taco stands. Most people don't. I know that the ratings for these places will be lower than how we feel about them. But if I compare the ratings of different dive taco stands relative to each other, I'll probably be able to pick out the best of the bunch.

 

At least that's how I approach it. If I'm only interested in easy finds at the end of difficult hikes, I can run a PQ for low difficulty, high terrain caches and compare the ratings for the caches that pop up.

 

It won't be perfect. But it might be better than operating in a vacuum.

Link to comment

Sure, we don't know the implementation details yet, but I, for one, am very happy that Groundspeak is making a strong effort to address cache quality. Like everything else in Geocaching history, it may take awhile before everything gets figured out, but the point is that Groundspeak is making it a priority. The same goes for stats...you can't ignore the fact that statistics are a widely-enjoyed meta-game, and I'm very pleased to see that we're going to start seeing official stats. Go :huh:!

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...
×
×
  • Create New...