New Pocket Query: Archived Caches

[+russellvt]

It would be useful to add the attribute "Archived Cache" (similar to "Is Not Active") to the pocket queries page. I'd say it should be on the Google Maps page (though unselected by default) but that might be overkill. This feature would be useful for those that maintain things like GSAK databases and they need/want to know when caches on their list (read: possibly already in their GPS) have either been marked inactive or simply archived -- sometimes there is no time in-between, or the two happen very quickly from one another.

 

I know that I (like others) have had a few occasions where I've spent way-too-much-time looking for a recently archived cache (which I missed because it wasn't on the map for the area I was targeting, but I already had it in GSAK and didn't cross-verify the caches from that angle before I left on my caching day). Adding this would help premium members better optimize their time.

 

[+Maingray]

Can't see it happening, Groundspeak have no interest in us maintaining an offline database. There are plenty of GSAK tools / macros to allow people to check for archived caches, such as deleting "stale" caches. Check out the GSAK support forum.

[+Markwell]

Part of my FAQ

[+root1657]

Can't see it happening, Groundspeak have no interest in us maintaining an offline database. There are plenty of GSAK tools / macros to allow people to check for archived caches, such as deleting "stale" caches. Check out the GSAK support forum.

 

I'd have to disagree with at least part of that. All too often when people ask for features on this site, they are told that they dont need it on the site because they can already do it in GSAK. In your post you claim that they have no interest in helping us maintain our GSAK, but then it comes down to a catch 22 of sorts. May of the requested features should be integrated into the website, else make consessions for people doing that work on thier own...

[+Renegade Knight]

Can't see it happening, Groundspeak have no interest in us maintaining an offline database. There are plenty of GSAK tools / macros to allow people to check for archived caches, such as deleting "stale" caches. Check out the GSAK support forum.

 

GSAK is limited to the formerly active caches that are now archived. What it can't find is the archived caches in an area.

 

Grounspeak does have an interest in allowing us to see archived caches. Two actually. First, the history in the logs. That's reason enough to be able to call up and read them. When you or your buddies kick off this is part of the legacy you have left behind. Especially if you wrote more than TNLNSL.

 

The other reason is again the history. This time though the reason for the cache being archived. Muggled, Blown up. Land manager policy. Past caches containe good information on for the placment of future caches.

 

They may have other reasons they don't want to do this, but they do have some good reasons for doing it as well.

[+StarBrand]

Can't see it happening, Groundspeak have no interest in us maintaining an offline database. There are plenty of GSAK tools / macros to allow people to check for archived caches, such as deleting "stale" caches. Check out the GSAK support forum.

 

I'd have to disagree with at least part of that. All too often when people ask for features on this site, they are told that they dont need it on the site because they can already do it in GSAK. In your post you claim that they have no interest in helping us maintain our GSAK, but then it comes down to a catch 22 of sorts. May of the requested features should be integrated into the website, else make consessions for people doing that work on thier own...

See this quote from jeremy (owner of the website)

 

http://forums.Groundspeak.com/GC/index.php...t&p=1588633

Edited May 19, 2008 by StarBrand

[+russellvt]

I'd have to disagree with at least part of that. All too often when people ask for features on this site, they are told that they dont need it on the site because they can already do it in GSAK. In your post you claim that they have no interest in helping us maintain our GSAK, but then it comes down to a catch 22 of sorts. May of the requested features should be integrated into the website, else make consessions for people doing that work on thier own...

See this quote from jeremy (owner of the website)

 

The sheer irony of this, of course, is that when you create a PQ and download caches directly to your GPS, what you are doing in essence is creating an offline database. You can still take that data off the GPS or manipulate it in any way you so desire. Simply because it doesn't reside on someone's PC (in GSAK or anything else for that matter) is pretty much irrelevant. You can even still pull those same waypoints back from your GPS in to any number of applications...

 

Of course, further irony here (part of the catch-22 root1657 pointed out) is that by not providing the functionality, they're essentially encouraging others that want/need to use these offline databases to come up with their own (possibly automated) mechanisms to verify the status of caches. I'm sure, however, that Groundspeak would likely prefer that people didn't do such things, as it only helps to drive up load on their webservers as more and more people with large databases end up pounding their way through their DBs in-attempts to make sure a cache they're planning on hunting hasn't been disabled or archived. Of course, I also can't speak for anyone at Groundspeak, though they've listed spidering as against the ToS, it's probably not often worse than someone just opening up a ton of windows (in a non-automated fashion) so they can click through the "GPX" update.

 

Just seems like a reasonable request, as people really don't want to waste time on looking for caches that no longer exist... and truly, there's not really much differentiating something like GSAK and whatever waypoints I already have stored in my GPS without the benefit of some form of middleware. And, like I said, it'd probably help reduce the load on the Groundspeak webservers/database.

[+tozainamboku]

I'm sure, however, that Groundspeak would likely prefer that people didn't do such things, as it only helps to drive up load on their webservers as more and more people with large databases end up pounding their way through their DBs in-attempts to make sure a cache they're planning on hunting hasn't been disabled or archived. Of course, I also can't speak for anyone at Groundspeak, though they've listed spidering as against the ToS, it's probably not often worse than someone just opening up a ton of windows (in a non-automated fashion) so they can click through the "GPX" update.

 

Just seems like a reasonable request, as people really don't want to waste time on looking for caches that no longer exist... and truly, there's not really much differentiating something like GSAK and whatever waypoints I already have stored in my GPS without the benefit of some form of middleware. And, like I said, it'd probably help reduce the load on the Groundspeak webservers/database.

I find it amazing that everyone who asks for a way to get archived caches in their pocket queries is sure that Groundspeak would prefer that they didn't put such a load on the servers by using one of the other methods to determine which caches in their offline database had been archived. Perhaps it is true that you could get the same caches you are currently getting using fewer pocket queries if you could get the caches that have been recently archived. But what is also true is that some one could use this technique to enable them to maintain a offline database containing far more caches than they currently can. Not having to get all the caches in an area will allow people to maintain an offline database for a much larger area with many more caches. Some people would no doubt take advantage of this and in doing so perhaps wipe out any savings on the load on server. But probably more importantly, should that person be dishonest and decide to violate the TOUs and the license agreement for downloading of GPX files, they could start sharing this data and perhaps even duplicate a portion of Groundspeak's database and compete directly with Groundspeak (essentially the main reason that the TOUs prohibit spidering). Groundspeak is making it difficult to maintain a large offline database in order to protect their rights to the collection of data. You may maintain a smaller but still generous copy of the Groundspeak database for your personnal use, however. Perhaps if you can propose a way to prevent abuse of a PQ of archived caches it would be a reasonable request.

[+russellvt]

I find it amazing that everyone who asks for a way to get archived caches in their pocket queries is sure that Groundspeak would prefer that they didn't put such a load on the servers by using one of the other methods to determine which caches in their offline database had been archived.

Let's see... download a large page (something around 1MB, all told, last I looked), or send me the text portions of the same data... seems like the later would be a lot less intensive just in terms of bandwidth, alone (nearly 50MB versus like, what, 256k maybe?). Either way, I still need to get the same data (for my own personal use, of course).

 

Perhaps if you can propose a way to prevent abuse of a PQ of archived caches it would be a reasonable request.

Well, personally I see far less abuse potential for a list of "what used to be" than what's currently already allowed... meaning, I think it's far-more-abuseable to be able to PQ the same 2500 caches every day and, by process of elimination, deduce the "missing" (archived?) caches than to let me get 2000 of those same caches and 500 caches that "no longer exist."

 

Then again, I guess I fail to see why someone would specifically/otherwise want this data for illicit purposes; guess I must be missing something. I mean, truth by told, there are enough tools out there these days that detecting a crafty person with malicious/illegitimate ideas is already the proverbial "needle in the haystack," so to speak (especially when that "needle" looks like just another piece of hay). Personally, I'd rather just ask and hope the-powers-that-be might agree.

 

I guess perhaps I should have just left this topic to "adding them back on to the map" instead of the "dreaded PQ." eh? *laugh* And here I thought maybe the PQ would get a bit less resistance... at least that way the option would be for paid subscribers, only. Guess I figured wrong, there.

Edited May 22, 2008 by russellvt

[+tozainamboku]

I find it amazing that everyone who asks for a way to get archived caches in their pocket queries is sure that Groundspeak would prefer that they didn't put such a load on the servers by using one of the other methods to determine which caches in their offline database had been archived.

Let's see... download a large page (something around 1MB, all told, last I looked), or send me the text portions of the same data... seems like the later would be a lot less intensive just in terms of bandwidth, alone (nearly 50MB versus like, what, 256k maybe?). Either way, I still need to get the same data (for my own personal use, of course).

I understand quite well that it takes less bandwidth to get just what changed instead of getting all. I'm reasonably sure that Groundspeak understands this as well, yet Jeremy continue to indicate he won't be providing an archived caches PQ. My statement is meant that Groundspeak is not as interested in saving bandwidth as in controlling the number of up-to-date caches you can have in your offline database. It may be that they are not worried about abuse, but this seems the most likely reason. Jeremy has taken steps in the past to shut down other sites that repackaged Geocaching.com data without permission. One site used screen scraping to produce up-to-date maps that people found very useful in getting an idea of where cache dense areas were. Others produced statistical information including leader boards. Jeremy allows the current leader boards to continue only because they produce fairly limited statistics that Jeremy isn't interested in having on the Geocaching.com website and they do this without continuously scraping the site so he doesn't see it as interference (or at least not worth dealing with). Groundspeak sees PQs as a way to give users the ability to download a reasonable number of caches for people to use. If users choose to keep these caches in an offline database for a significant time without refreshing all the data, they run the risk that some caches will have been archived. Each cacher can determine for themselves how often to refresh this data and eliminate the caches that have been archived. Allowing the user to get an update of just caches that have been archived would make it easier to keep a larger offline database. Instead of getting the same 2500 caches everyday, one could build up an database of 25000 caches or more and get the caches that have been updated everyday including those that were archived that day. I'd bet you'd only need one 500 cache query per day to maintain a 25000 cache database. With a little effort one could probably keep a up to date copy of a significant portion of the Geocaching.com database.

[+russellvt]

Allowing the user to get an update of just caches that have been archived would make it easier to keep a larger offline database. Instead of getting the same 2500 caches everyday, one could build up an database of 25000 caches or more and get the caches that have been updated everyday including those that were archived that day. I'd bet you'd only need one 500 cache query per day to maintain a 25000 cache database. With a little effort one could probably keep a up to date copy of a significant portion of the Geocaching.com database.

 

I'd still contend that the only thing that really changes here is the ease at which you maintain these databases... really, there's not much difference between it and in pulling the same PQ every N days. As early as the second time you request the "same" dataset (or even a significantly overlapping one for that matter), you should already know basically what you expect to see, the only exception being the furthest outliers from a center point or a particular query (which can be calculated on each request, essentially based on the delta of changes). And, you'd not even need to screen-scrape.

 

So, to put it as simply as possible, any cache that was listed in a previous query that's "closer" than any other point is likely listed as archived... anything further out than (or a relatively small average of a few of them) simply is ignored as not being included in the dataset you were sent. It's really not that hard if your goal is to copy the database -- I'm just looking to make my life easier, that's all (and unfortunately I've seen caches archived much faster than I might otherwise like to admit).

 

So really, I think all that's happening (in my opnion) is sacrificing that ease at which legitimate or paid users are able to update whatever caches they're caching (in the sense of a "maintained database" go). Again, in my opinion... for all that's worth (I know, probably nothing, at this point).

 

Then again, neither one of us can speak for Jeremy, let alone Groundspeak, can we? *smirk*

 

(though perhaps this thread's received enough attention that someone in some official position might be able to further enlighten us... *shrug*)

[+Miragee]

<snip>

So really, I think all that's happening (in my opnion) is sacrificing that ease at which legitimate or paid users are able to update whatever caches they're caching (in the sense of a "maintained database" go). Again, in my opinion... for all that's worth (I know, probably nothing, at this point).

 

<snip>

If you get Notifications for "Archived" caches, and "Disabled" caches, you can keep the data you are putting into your GPSr as current as possible. I use several "Date Placed" PQs to get a wide search area coupled with those notifications and the "Last .gpx Update" filter in GSAK to keep my data current.

 

It really is easy-peasy once you get used to understanding how to filter out caches that didn't update with the last set of PQs.

[+russellvt]

If you get Notifications for "Archived" caches, and "Disabled" caches, you can keep the data you are putting into your GPSr as current as possible. I use several "Date Placed" PQs to get a wide search area coupled with those notifications and the "Last .gpx Update" filter in GSAK to keep my data current.

 

It really is easy-peasy once you get used to understanding how to filter out caches that didn't update with the last set of PQs.

/slaps forehead

 

This entire thread and you're the first to suggest something as bonehead-simple as that... doh! Guess I got entirely too tunnel-visioned with the entire "bulk" idea... something about "forest of sake of the trees," eh? Similarly, I sometimes think others might get too locked in to arguing a point rather than simply stepping back, thinking, and trying to actually help with a given issue.

 

Thanks Miragee! You rock!

 

(also thanks to trainlove who also replied privately to me with another good line of reasoning)

Edited May 22, 2008 by russellvt

[+sTeamTraen]

This entire thread and you're the first to suggest something as bonehead-simple as that... doh!

 

Actually, you had an even simpler reply within one hour of your original post:

 

<snip>

Check out the GSAK support forum.

 

Your issue is basically with GSAK, where this question is very definitely an FAQ. Although pretty well any situation where you accumulate data over time from a source suffers from the same issue, namely, having supplied you with any given data an arbitrary time ago, how does the system tell you to stop using it? This is actually a real-life academic computer science issue on which some very high-powered people have spent a lot of thought. (Ask anyone who sends out their software on CDs and would like people to stop using versions before March of this year because they have a bug in them.)

 

In theory, to prevent anyone ever having any out-of-date data (in this case, cache archived and the database not updated), one of the following would have to be available:

 

1) A mechanism to let you know, for every cache that has ever been sent to you on any PQ which you have ever run, or that you have saved manually from the GPX file link, that that cache is now archived. After all, Groundspeak does not know what you chose to do with that data, so they would have to assume that it's part of your offline database. That in turn would require every cache to have an associated field, listing every site user who had ever received a GPX copy of it, the exact timestamp when that happened, and what its status was at the time (after all, you might have downloaded the cache when it was archived, not seen it in a PQ when it was reactivated for a while, and now it's been rearchived). I'll leave the math as an exercise for the reader, but the numbers will quickly become eye-watering when you multiply by over 800,000 cache listings - we're certainly talking many, many gigabytes.

 

2) The ability to receive a fresh copy of every cache which is currently in your offline database - at least its status, "archived or not" - as one PQ. It would take quite some programming to allow you to specify that with a graphical interface. The only way I can see to do it would be for the site to allow you to upload a GPX file (presumably with some limit on the number of caches... start the bidding...) and it would send you the updated version of the data for every cache in it.

 

Given all that, I can see why Groundspeak doesn't want to be involved in helping people maintain their offline databases. It would be an endless and ultimately not very productive task.

Edited May 23, 2008 by sTeamTraen

[+russellvt]

This entire thread and you're the first to suggest something as bonehead-simple as that... doh!

Actually, you had an even simpler reply within one hour of your original post:

Actually, the only really "useful" piece of that reply, AFAIK (or can tell), are technically direct violations of the ToS (which I'd prefer to stay as clear of as possible, of course). Feel free to point me in a more specific direction, though, if my assessment is incorrect (sometimes folks can overlook the-most-obvious things (or be tunnel visioned away from there), as was kinda also proven in this case... *laugh*)

 

Note: the idea of "rotting" things off your list really isn't any simpler IMO, since it's technically inaccurate (ie. it's a workaround rather than a real/complete solution, at best).

 

<snip>

Check out the GSAK support forum.

Your issue is basically with GSAK, where this question is very definitely an FAQ. Although pretty well any situation where you accumulate data over time from a source suffers from the same issue, namely, having supplied you with any given data an arbitrary time ago, how does the system tell you to stop using it? This is actually a real-life academic computer science issue on which some very high-powered people have spent a lot of thought. (Ask anyone who sends out their software on CDs and would like people to stop using versions before March of this year because they have a bug in them.)

 

In theory, to prevent anyone ever having any out-of-date data (in this case, cache archived and the database not updated), one of the following would have to be available:

 

1) A mechanism to let you know, for every cache that has ever been sent to you on any PQ which you have ever run, or that you have saved manually from the GPX file link, that that cache is now archived. After all, Groundspeak does not know what you chose to do with that data, so they would have to assume that it's part of your offline database. That in turn would require every cache to have an associated field, listing every site user who had ever received a GPX copy of it, the exact timestamp when that happened, and what its status was at the time (after all, you might have downloaded the cache when it was archived, not seen it in a PQ when it was reactivated for a while, and now it's been rearchived). I'll leave the math as an exercise for the reader, but the numbers will quickly become eye-watering when you multiply by over 800,000 cache listings - we're certainly talking many, many gigabytes.

 

2) The ability to receive a fresh copy of every cache which is currently in your offline database - at least its status, "archived or not" - as one PQ. It would take quite some programming to allow you to specify that with a graphical interface. The only way I can see to do it would be for the site to allow you to upload a GPX file (presumably with some limit on the number of caches... start the bidding...) and it would send you the updated version of the data for every cache in it.

 

Given all that, I can see why Groundspeak doesn't want to be involved in helping people maintain their offline databases. It would be an endless and ultimately not very productive task.

 

Well, IMHO, you're way over-thinking what I was asking, here... I'm not asking for a complete and comprehensive list, by any means. I am (or was) just looking for a simple mechanism to toggle any caches that are already listed "active" on my GPS over to a symbol that makes it clear they are no longer available (on a fairly case-limited basis). From there, it makes it easier to delete/ignore things I no longer care about or wish to see. As previously stated, I tend to keep my GPS waypoint memory fairly full with caches around my "frequently traveled areas" just-in-case I have an unplanned moment/occasion to go for a hunt. Like others, I just have no real desire to go searching for a cache that's no longer there, ya know? *laugh*

 

Of course, since I pretty much have a weekly "full" (500) cache PQ(s) that define those area(s) and new caches are being placed often faster than I can "clear" them, simply letting the archived caches "rot" off the list doesn't really work for me (as now the outliers now appear to "rot" as new caches are placed -- though that kinda falls in to the "math issue" you point out, in a small way). It also involves the practice of bulk-deleting waypoints/caches from the GPS before I resend new waypoints to it, which I currently don't tend to do. Perhaps I need to think about how I do those updates, too, but currently I mostly just let waypoints get updated until I run out of memory, then I go looking for the "bulk delete by symbol" function.

 

The simplest solution, by far, would be a PQ that defines those same areas that are already defined in other PQs to give me a list of caches that have been archived/disabled within N+1 days of my last PQ (ie. 8 days for the sake of overlap). There's no real complexity here... at least, I surely don't see any need for it. And the system already has a computed list of all the caches in the same area (eg. X miles from a defined center point) -- I'm just asking for a status to be added, that's it... they're already intentionally "ignoring" those same caches I'm asking to see (ie. remove the "and not status=disabled" or similar from the SQL).

 

Largely, I think the real problem here eventually ends up becoming more of a "disabled cache saturation issue" as the list of disabled caches grow in an area rather than a "you shall not have a complete/perfect list of caches" issue... but again, I obviously can't speak for Groundspeak. The only use I have for maintaining any archived cache status, in the long run, are for those that I've either flagged as "interesting" or that I've found prior to their de-listing (and largely just the later)... that's it.

 

But, again, as was pointed out quite clearly a few posts up... I can already get the same info that I had requested (archived/disabled caches) in a more iterative fashion, and without violation of the ToS. I had just been reading too much about "pocket queries" when I came up with the original question/post and, hence, was kind of focused on the same concept and didn't think about the-most-simple form of the problem/issue, myself. And yeah, maybe I need to slightly adjust the way in which I cache, too... but, well, that's a slow adjustment as I figure out what works best for me. *laugh* (suggestions for improvement are, of course, always welcome)

 

Thanks again to everyone for their responses/input!