Jump to content

Revision of Review process

Confucius' Cat

By Confucius' Cat
in General geocaching topics

Followers 4

Recommended Posts

Confucius' Cat

+Confucius' Cat

Posted
Posted

I was talking to anothe cacher tonight and this topic came up. I think a lot of responses will be predictable, but I thought I would bring up his ideas anyway. I think they have merit, but also (of course) associated problems.

 

Premises:

1. There are some caches placed in a questionable manner or in questionable places which are not easy to determine from written information. Further, caches placed by beginners are often lacking in some aspect which could be easily improved, even though they might not technically violate any guidelines.

 

2. There are caches placed which, while technically OK, are destined to be regarded by many as "lame".

 

3. The necessity for review prior to publication varies depending on the skill, experience, and reputation of the placer. i.e. newbies probably require closer checknig than old-timers in most cases.

 

I would like to limit debate on the premises if possible. If you disagree with the premises, the expected response would simply be "no changes to the current system would be justified".

 

Assuming these premises are valid then, would you favour modification of current reviewing practices along the lines I shall suggest or do you have other ideas?

 

Suggestions:

1. Those cache submissions that are simply "OK to go", would simply be published as is done today. But for those that the reviewer finds "need further information", a team of 5 volunteer cachers (hereinafter called "peer reviewers") would be dispatched to find and check out the new cache. They would check for whatever specific concerns the reviewer has, such as "is the cache too close to the RR?" or "will seekers be putting themselves in unnecessary danger or trespassing", confirm the ratings and coordinates, and note if they have any other concerns. Their concerns could be limited to guidelines or they could be given authority to judge quality issues as well. They would each vote yea or nay and the majority rules. These cachers would not be allowed to trade, change the cache without the owner's permission, or claim the FTF. They could however remove any items that are clearly prohibited and ask the owner to advise for return (at owner's expense) or disposal. They would not be allowed to log the assigned cache online until after the FTF.

 

2. The peer reviewers would be automatically assigned to perhaps the first 10 submissions by a newbie as a matter or policy.

 

3. "Veteran cachers" (definition and designation authority open to debate) could be designated to have automatic approval authority on their own submissions

 

Reasoning (numbers relate to suggestion numbers above):

1. The volunteer reviewer would not have to spend his/her own time investigating, writing emails, or discussing back and forth with the hider. Arguments over the reviewer's discretion would be reduced by having a more or less automatic peer review process. Simple problems could be found and corrected quickly prior to publication.

 

2. Until a cacher establishes a track record for following guidelines and shows they reasonably understand the concepts of caching, their submissions should be looked at a little more closely. This would also give the opportunity for more seasoned members to "mentor" the newbies and should help improve overall new cache quality.

 

3. Why do we spend approver time on people who have clearly shown that they do not need supervision? It would be kinda like granting them "limited approver" status- they can approve caches, but only their own. This authority could be revoked if abused.

 

Cons that I see:

Possible increased approval waits on newbie caches- maybe, maybe not.

Perhaps difficulty in getting trusted peer reviewers.

Well.... just plain resistance to change.

 

What thinks ye?

Link to comment
budd-rdc

+budd-rdc

Posted
Posted

What thinks ye?

 

1. On rare occasions, I've already seen this happen, although it wasn't a team of 5 additional volunteers, just a person or two. You're asking for a lot of resources here. This suggestion will turn Geocaching from fun into work for those 5 people. Why not take the time to recruit an additional full-time reviewer for the area?

 

2. Having a mentor ("peer reviewer") is not a bad idea, but it should be done locally, and a person-to-person contact should be established first. Not as efficient this way, but more effective. I'd feel weird if a mentor was assigned to me just after I signed up on GC.com.

 

3. Corruption is absolute. Besides, careful hiders in my area sometimes have their caches approved in less than 5 minutes (includine a few of mine), probably because the caches were easy to review. So this is already happening without spreading the corruption. I never underestimate what reckless creativity can do to this idea.

 

This authority could be revoked if abused.

That's called opening a can of worms. An abuser will likely feel he's entitled to his actions, so I see nothing but escalation of hostilities between the parties involved. It takes just one bad apple to revoke the idea itself.

 

I won't dismiss your premises, based on the increasing number of complaints about "lameness" this past year. I appreciate your attempt at trying to find solutions to the problems. However, I balk at your suggestions which do not account for regional differences. Some areas complain there are too few hides, but others complain there are too many. Geography vary from region to region, and so does the definition of lameness. I think the best way to tackle your premises is to encourage the creation of local Geocaching organizations, with Groundspeak providing an easy-to-find page with a list of them, so the newbies can turn to them for help.

Link to comment
Happy Humphrey

+Happy Humphrey

Posted
Posted

Even if you could recruit 5 volunteers for these duties around here, you'd probably delay the cache approval by months rather than days!

Quality checking is harder than you think. Although some I've found have been better than others from my own point of view, there are several reasons for placing a cache; I respect the right of others to place one in such a way that it doesn't appeal to me.

 

HH

Link to comment
CoyoteRed

CoyoteRed

Posted
Posted

Hmmm...

 

A "go, no go" rating system assigned to 5 people per area. Depending on the people, it could severely change the landscape of an area. Suppose 3 of the 5 don't like hard puzzles or are mobility challenged and don't like strenuous hikes. What if their tolerance of lameness is very high? (That's not to mention the major problem of them actually being able to do the cache to begin with.)

 

The major problem with this is cronyism. Not that it doesn't happen already, but it could certainly have the potential of getting out of hand.

 

Personally, while not perfect, I'd rather stick with the present system.

Link to comment
gorgon9380

+gorgon9380

Posted
Posted

no changes. Locally, the '5 reviewer' thing would be a bust. I can think of at least 5 guys that would *Love* to be on the review team, then call about 40 of their friends so they could all march right up to the cache and claim a find w/o even breaking out the GPS. This would be a boon for the High Numbers folks.

Link to comment
carleenp

+carleenp

Posted
Posted

It is an interesting idea, but I just don't see it working for a number of reasons. The main reasons being that a system of volunteers who actually check things and make some sort of judgment on quality is likely going to be time consuming, inconsistent, and hard to coordinate. Second, even if it can be made to work people would throw fits about the change, causing more headaches than it would all be worth. I think disagreements abut the process would also arise more often. Finally, the current system really isn't that bad. I tend to think that things should not be fixed unless they are truly broken. Everyone can find things they might want to do differently or better, but that doesn't equate to a broken system. Although perhaps part of the view of whether it is broken is based on your intial premises.

Link to comment
Stunod

+Stunod

Posted
Posted

It is an interesting idea, but I just don't see it working for a number of reasons. The main reasons being that a system of volunteers who actually check things and make some sort of judgment on quality is likely going to be time consuming, inconsistent, and hard to coordinate. Second, even if it can be made to work people would throw fits about the change, causing more headaches than it would all be worth. I think disagreements abut the process would also arise more often. Finally, the current system really isn't that bad. I tend to think that things should not be fixed unless they are truly broken. Everyone can find things they might want to do differently or better, but that doesn't equate to a broken system. Although perhaps part of the view of whether it is broken is based on your intial premises.

 

What she meant to say was "no changes to the current system would be justified"

Link to comment
welch

+welch

Posted
Posted

I can see you've thought about this a bit, but I don't really see it working out that well. I tend to think automatically assigning peers reviewe and doing it for Ten caches would be overkill, and a wasting peer reviewers time. If they haven't figured out a lot of the do / do not things in the first couple (and to ask someone for advice when in doubt) you'll probably have to review all their caches, or at least everytime they place new cache types etc.

 

However, I think the biggest problem would be overseeing the peer reviewers. There are occasionally complaints about the reviewers taking too long to review, or being bias, or being mean to a certain person by insisting the person follow the guidelines. If you increase the reviewers by 5 fold you'd likely get an increase increase in complaints. Maybe more since the 'wait' would also include 5 actual visits to the location and coming back online to vote aye or nye. And how do you deal with the peer reviewers that differ from the guidelines maybe 3 of the 5 say knives are ok in caches out in the woods, or that buried means no digging holes more than a couple inches deep so it doesn't affect 'grade'. Seems like a lot of time could be spent trying to find the perfect juries reviewers. (btw- FTF are pointless, but why wouldn't the peer reviewers get to log it? After the owner one of them was the first to find it, and someones bound to take an issue with offical/unoffical/postgc.compublishing FTF junk)

And oh dang, a quality review would really tick people off. I dislike 'lame' caches as much as the next cacher, but differences in definitions are bound to cause arguements...

 

I like the ideas of spending extra time on those that seem likely to mess something up. And to their be cachers that will assist on cache problems (vactions caches, caches that don't get listed but have TB in them), but I think these things already sorta happen (right?). Its just more person to person and not some automatic system.

 

welch

Link to comment
wimseyguy

+wimseyguy

Posted
Posted

I’m always amazed that nobody suggests simply lying to the reviewer. :laughing: After all, if being untruthful is acceptable when logging an alleged ‘find’ on a cache, why not use that same standard when hiding one?

 

No, no changes are needed.

 

Don't think it's not already done.

And probably done more by the more experienced hiders than the newer ones. :rolleyes: And fixing the problems created by these inexperienced hoders is what this complex system seems to be geared toward correcting. At least that is what I gathered before I quit reading the whole long complex post. :(

Link to comment
TheAlabamaRambler

+TheAlabamaRambler

Posted
Posted

I can see you've thought about this a bit, but I don't really see it working out that well. I tend to think automatically assigning peers reviewe and doing it for Ten caches would be overkill, and a wasting peer reviewers time. If they haven't figured out a lot of the do / do not things in the first couple (and to ask someone for advice when in doubt) you'll probably have to review all their caches, or at least everytime they place new cache types etc.

 

However, I think the biggest problem would be overseeing the peer reviewers. There are occasionally complaints about the reviewers taking too long to review, or being bias, or being mean to a certain person by insisting the person follow the guidelines. If you increase the reviewers by 5 fold you'd likely get an increase increase in complaints. Maybe more since the 'wait' would also include 5 actual visits to the location and coming back online to vote aye or nye. And how do you deal with the peer reviewers that differ from the guidelines maybe 3 of the 5 say knives are ok in caches out in the woods, or that buried means no digging holes more than a couple inches deep so it doesn't affect 'grade'. Seems like a lot of time could be spent trying to find the perfect juries reviewers. (btw- FTF are pointless, but why wouldn't the peer reviewers get to log it? After the owner one of them was the first to find it, and someones bound to take an issue with offical/unoffical/postgc.compublishing FTF junk)

And oh dang, a quality review would really tick people off. I dislike 'lame' caches as much as the next cacher, but differences in definitions are bound to cause arguements...

 

I like the ideas of spending extra time on those that seem likely to mess something up. And to their be cachers that will assist on cache problems (vactions caches, caches that don't get listed but have TB in them), but I think these things already sorta happen (right?). Its just more person to person and not some automatic system.

 

welch

 

What he said was "no changes to the current system would be justified"! :laughing:

Link to comment
BlueDeuce

+BlueDeuce

Posted
Posted (edited)

I can see you've thought about this a bit, but I don't really see it working out that well. I tend to think automatically assigning peers reviewe and doing it for Ten caches would be overkill, and a wasting peer reviewers time. If they haven't figured out a lot of the do / do not things in the first couple (and to ask someone for advice when in doubt) you'll probably have to review all their caches, or at least everytime they place new cache types etc.

 

However, I think the biggest problem would be overseeing the peer reviewers. There are occasionally complaints about the reviewers taking too long to review, or being bias, or being mean to a certain person by insisting the person follow the guidelines. If you increase the reviewers by 5 fold you'd likely get an increase increase in complaints. Maybe more since the 'wait' would also include 5 actual visits to the location and coming back online to vote aye or nye. And how do you deal with the peer reviewers that differ from the guidelines maybe 3 of the 5 say knives are ok in caches out in the woods, or that buried means no digging holes more than a couple inches deep so it doesn't affect 'grade'. Seems like a lot of time could be spent trying to find the perfect juries reviewers. (btw- FTF are pointless, but why wouldn't the peer reviewers get to log it? After the owner one of them was the first to find it, and someones bound to take an issue with offical/unoffical/postgc.compublishing FTF junk)

And oh dang, a quality review would really tick people off. I dislike 'lame' caches as much as the next cacher, but differences in definitions are bound to cause arguements...

 

I like the ideas of spending extra time on those that seem likely to mess something up. And to their be cachers that will assist on cache problems (vactions caches, caches that don't get listed but have TB in them), but I think these things already sorta happen (right?). Its just more person to person and not some automatic system.

 

welch

 

Pretty much what I was thinking. People already 'take issue' with reviewers, could you imagine if you added peers into the mix?

 

I think there is enough checking by peers to see if the cache meets guidelines. It's just done after the cache has been posted, by the first few finders.

 

edit: note to self, proofread before posting

Edited by BlueDeuce
Link to comment
BlueDeuce

+BlueDeuce

Posted
Posted (edited)

Pretty much what I was thinking. People already 'take issue' with reviewers, could you imagine if you added peers into the mix?

The reviewers are peers.

 

Once upon a time there were no reviewers, and there were very few problems.

 

Well, okay. I'm referring to the peer Groundspeak reviewers who have access to complete the submission process and all the responsibilities therein.

 

Not the OTHER peers who don't.

 

I guess the OP request is to increase the reviewers by five fold.

Edited by BlueDeuce
Link to comment
Criminal

+Criminal

Posted
Posted

Pretty much what I was thinking. People already 'take issue' with reviewers, could you imagine if you added peers into the mix?

The reviewers are peers.

 

Once upon a time there were no reviewers, and there were very few problems.

Ouch, that hurt.

I think you're reading too much into that Moose. :laughing:

 

The reviewers didn't cause the problems. With the huge growth of the game, it was inevitable that people were going to get careless in their desire to hide caches and the small staff in Seattle wasn’t going to be able to keep up.

Link to comment
TrailGators

+TrailGators

Posted
Posted

One thing they should do to help the reviewers out is to mandate that waypoints be entered for all mystery caches and multi caches by a certain date or they will get archived. I ran into two multis this weekend that had two waypoints from each within 15 feet of each other. Of course, we found and followed the waypoint from the disabled multi and wasted a bunch of time.... bangheadsc2.gif

Link to comment
Renegade Knight

+Renegade Knight

Posted
Posted

...Pretty much what I was thinking. People already 'take issue' with reviewers, could you imagine if you added peers into the mix?...

 

I think there is a place for peers, just not as proposed in this post. In writing a concept for an approval process I laid out a spot for peer review. Later and independantly Jeremy aslo bounced the idea in the forums. As things are, the peer review is in the guidelines to take a dispute to the fourms. That peer review can be brutal, but on occasion it does work to the benefit of the cacher.

Link to comment
BlueDeuce

+BlueDeuce

Posted
Posted (edited)

...Pretty much what I was thinking. People already 'take issue' with reviewers, could you imagine if you added peers into the mix?...

 

I think there is a place for peers, just not as proposed in this post. In writing a concept for an approval process I laid out a spot for peer review. Later and independantly Jeremy aslo bounced the idea in the forums. As things are, the peer review is in the guidelines to take a dispute to the fourms. That peer review can be brutal, but on occasion it does work to the benefit of the cacher.

 

Pretty much what I was thinking. People already 'take issue' with reviewers, could you imagine if you added peers into the mix?

I think there is enough checking by peers to see if the cache meets guidelines. It's just done after the cache has been posted, by the first few finders.

 

Edit: Of course I am talking about the peers that visited the cache, which may not be the peer who reviewed it.

Edited by BlueDeuce
Link to comment
Renegade Knight

+Renegade Knight

Posted
Posted

....I think there is enough checking by peers to see if the cache meets guidelines. It's just done after the cache has been posted, by the first few finders....

 

That's another ball of wax. The version that I was discusing was the peer review of a contested cache listing. As for the community we have an obligation to take care of obviouse issues with a cache. My experience though is about 10:1 of the people who are wrong about their assumptions vs 1 who's right and has a valid issue for me to address on my cache.

Link to comment
BlueDeuce

+BlueDeuce

Posted
Posted

....I think there is enough checking by peers to see if the cache meets guidelines. It's just done after the cache has been posted, by the first few finders....

 

That's another ball of wax. The version that I was discusing was the peer review of a contested cache listing. As for the community we have an obligation to take care of obviouse issues with a cache. My experience though is about 10:1 of the people who are wrong about their assumptions vs 1 who's right and has a valid issue for me to address on my cache.

 

Sorry, I assumed that my original post would be taken in complete context.

 

I'll re-state it.

 

Right now you have people who review caches and that take the brunt of complaints/questions/abuse, I think adding more people into that process simply adds more opportunity for the submitter to take issue with whatever review is returned. Considering that in most cases an experienced reviewer can properly assess a cache, any issue that might have been missed by a reviewer, for whatever reason, is usually discovered by the first few experienced cachers and reported. That is the peer review process I am referring to.

 

(That is, the peers who don’t initially review caches)

Link to comment
Guest
This topic is now closed to further replies.
Followers 4
Go to topic listing
×
×
  • Create New...