Jump to content

For Jeremy: Archived Caches Pq


CoyoteRed

Recommended Posts

As you may know I recently went back to downloading full PQs versus just pulling the caches that have changed in the past 7 days. It works wonderfully in ridding my collection of archived caches, but is grossly inefficient. Note the graphic of a recent and typical import:

 

e19579a0-02b4-40f2-a631-2a42a5a770e0.jpg

 

This seems to be typical of what I've been importing. Of the 1809 caches in the PQs, only 276 really mattered in updating the database.

 

I know the subject of PQs of archived caches has gone around and around several times, but I may have a solution. Tie the archived caches to the "Changed in last 7 days" box. This way only those caches archived in the last 7 days will be included.

 

Advantages to this solution are

  • It facilitates fresher data in that it doesn't leave archived caches "orphaned" in our database. Less likely that users of offline databases will be hunting archived caches. Good PR to land owners.
  • While it delays hiding archived caches it is only for 7 days maximum. That should be well within the window of operations of this site.
  • Would allow those, like me, to fall back to getting only the caches changed in the last 7 days thus reducing server load, bandwidth, the number of PQs run, etc.

Just a suggestion. Might allow some of us to reduce the number of PQs we need.

Edited by CoyoteRed
Link to comment
How do you get a PQ with 1809 caches in it?

I actually have my PQs delivered to a dedicated mail box and a script running to download those on a daily basis and stuff'em into a certain directory on the LAN. This allows me to not have to manually check the box every day and when I go to import into GSAK I don't have to wait on download speeds.

 

I've saved up to 60 files once before importing into GSAK. It's all automated except for actually opening GSAK and the import.

 

I'd like to be able to have a more friendly "set it and forget it" scheme and still have the same outcome.

Edited by CoyoteRed
Link to comment

We, too, would LOVE to have this functionality. We cache paperless and have, on more than one occasion, spent time fruitlessly looking for a cache, only to find out when I go to log our DNF (and we log ALL of them), that the cache was archived. :lol:

 

Mrs. Car54

Link to comment
I'd like to be able to have a more friendly "set it and forget it" scheme and still have the same outcome.

I've accomplished mostly the same thing using GSAK macros. I simply created a filter that I call "Recently Archived" that looks for ones that haven't been updated from a GPX file in the past x days (mine is 7, but you could use longer or shorter depending on how fresh your PQs are) and ignores my finds and hides. My startup macro simply runs this filter and deletes all waypoints in the filter results.

 

This does NOT delete caches that haven't been logged in x days, just those that are not touched by an updated GPX file - archived caches. Basically, instead of listing caches that have been removed my macro assumes that the absence of caches from a GPX file implies that they have been removed and dumps them from GSAK too. If you want, you could have GSAK move them to another database if you'd prefer, but I just get rid of mine completely.

Link to comment
I've accomplished mostly the same thing using GSAK macros.

Ummm... No.

 

This is not what I'm talking about. I'm talking about receiving Pocket Queries that have only those caches that have changed in the past 7 days. As you can see in my graphic above, only 276 caches changed any information in GSAK. The remaining 1533 caches where not needed and a waste of time and effort on the part of the Groundspeak servers.

 

Once I have an up-to-date database, I can use only those caches that have changed in the last 7 days and thus have a much smaller and fewer PQs. The only issue is getting the archived caches, too, so those aren't orphaned in the database.

Link to comment
I know the subject of PQs of archived caches has gone around and around several times, but I may have a solution.  Tie the archived caches to the "Changed in last 7 days" box.  This way only those caches archived in the last 7 days will be included.

I completely agree with this request and could not have said it better. If there were one change I would like the folks at GC.com to implement, this would be it.

 

The current "Updated in the last 7 days" PQ includes all caches that have had notes added to them which makes the query fill up fast. If we had a PQ that would include only those whose status had changed or where part of the static data on the cache (description, coordinates, etc.. ) had been changed that would be very nice.

Edited by Blanston12
Link to comment

Great suggestion {applause}

 

I too have spent a good while recently looking in vain for a cache only to find out layer it had been recently archived. Not enough to make me start using printouts, but still quite irritating. That was probably 3 or 4 caches worth of precious time! :o

Link to comment

Excellent idea CoyoteRed! Something I have done to try and keep on top of caches being archived is to create an Insta-Notify alert for any cache that is archived and/or disabled. This keeps me abreast of caches within a 50 mile radius of my home coordinates. I would like to see this capability expanded to include my entire state.

Link to comment

This is an excellent idea.

 

I know in the past Jeremy has resisted such requests because he doesn't think people should use offline data bases.

 

The fact is, however, we do.

 

I would run fewer PQs if I had this capability, because some of them are just to figure out what is recently archived.

I also think this is an excellent idea. Recently I spent a long time getting to and looking for a cache that had been recently archived.

 

I never knew the reason why Archived caches couldn't be included in PQs. :laughing: Since I have such a slow Internet connection (24K dialup is my only option), I depend on my GSAK database for information.

 

If I had to open the individual cache pages to check them, I would never get away from the computer. At 24K, it takes a long time to bring up the individual cache pages . . . :)

 

Thanks for proposing this CR. I would really appreciate the implemetation of this feature. thumbsup.gif

Link to comment

This is an excellent idea.

 

I know in the past Jeremy has resisted such requests because he doesn't think people should use offline data bases.

 

The fact is, however, we do.

In truth even if you don't use something like GSAK, the instant you get a PQ, or even print out a stack of geocache pages, you have an offline database. In fact unless you can establish a connection to gc.com in field you are using offline information.

 

So I think CR idea is a good one as well.

Link to comment

I never knew the reason why Archived caches couldn't be included in PQs.

They aren't there anymore. It makes perfect sense . . .

But they are still in our databases. If there was a limited time, like seven days from the time they are Archived, where they would come through the PQs with their new Archived status, it would prevent having stale data in the GPSr and PDA.

 

I think CR is proposing something that would work and would take a big load off the servers.

 

I've been waiting for more than an hour for a PQ to come through this morning :laughing: . . . is that because too many people are ordering more, and larger PQs than they really need just to keep their database more up-to-date? :)

Link to comment

 

I've been waiting for more than an hour for a PQ to come through this morning :laughing: . . . is that because too many people are ordering more, and larger PQs than they really need just to keep their database more up-to-date? :)

 

Jeremy's wishing that we don't have databases won't change the reality.

 

I ordered 3 PQs today to track down some newly archived caches. One much smaller PQ could have answered the question, if Jeremy allowed us to get that information.

 

It makes perfect sense for us to have that option...

Link to comment

The Boss has said "no" to this so many times, I just don't see it changing.

I, too, have requested 'shootdown' lists in the past and have explained why they make sense to me but I've accepted "no" and moved on.

 

Arguments about system load may be well intentioned, but I think there's an implicit answer to that, too: you can get 5 PQ's a day of no more than 500 caches. That's the number Groundspeak has decided to provide for that cost. Use them as you see fit.

Link to comment
Use them as you see fit.

 

This is pretty much what I've resigned to doing. I had offered the idea in the OP as I hadn't seen that before.

 

While the angle Jeremy mentions is perfectly valid, you could change the inflection a little and it becomes, "HEY! That cache is not there anymore!" To which we reply, "Okay, stratching that one off our list."

 

But, if it's not that big of a deal to Jeremy... ~shrugs~

Link to comment

. . . I think CR is proposing something that would work and would take a big load off the servers.

 

I've been waiting for more than an hour for a PQ to come through this morning :lol: . . . is that because too many people are ordering more, and larger PQs than they really need just to keep their database more up-to-date? :laughing:

I don't want to hijack this thread, but I'm still waiting for that single PQ. :) I only get PQs when I need them. I am going caching tomorrow and would like to have this data.

 

Did someone forget to feed the hamsters? :laughing: Are too many PQs being requested because the weekend is approaching? :)

Link to comment
Use them as you see fit.
This is pretty much what I've resigned to doing.

I hope you recall that I have offered to write a program that, given a new GPX file and a GPX file of existing caches, will generate a (possibly incomplete) list of caches that have disappeared.

 

Of course, since it would necessarily be run outside of GSAK, I doubt there will be much interest.

Link to comment
Use them as you see fit.

 

While the angle Jeremy mentions is perfectly valid, you could change the inflection a little and it becomes, "HEY! That cache is not there anymore!" To which we reply, "Okay, stratching that one off our list."

 

I too wish the "past 7 days" would help us remove archived caches. Egads, CoyoteRed and I agree on something. Heh heh heh

 

Jeremy, couldn't the PQ of caches changed in the past 7 days have the listings of archived caches only contain the waypoint number, and nothing else? If the coords aren't there, the description isn't there, etc, then GC wouldn't be sending out information about a cache that isn't there.

 

We could use GSAK to easily identify nameless, coord-less caches in the PQ and remove them from our offline databases.

Link to comment

Yeah, I remember that, but it won't help in the same way as what I was suggesting.

 

As it is I get a PQ of all active, including unavailable, caches in our area. I filter on GPX files exported in the last week and anything beyond that is assumed archived.

 

Similar to what you're talking about and is about the same result.

 

In essence, it's simply throwing away the old data and using the new. Of course, not exactly the same as I keep the older logs, but that's about the only difference.

 

The OP scheme is more of a differential scheme where you're not pushing around the whole database of an area, only that which has changed. The Groundspeak infrastructure would be pushing a much smaller dataset.

Link to comment

I really don't see what the driving force is behind the need to have every single cache that has ever existed in your area in a private database. I do see that it is wanted by some tho. I haven't read all of this thread or the other several ones on it but had a thought. We can get notifications of new caches (immediately and/or weekly). Would getting similar type notifications for caches being archived be a possible solution?

 

BTW: I've never had the problem of archived caches in my GSAK database because I always use only my new PQs instead of adding them to the existing database.

Link to comment

I really don't see what the driving force is behind the need to have every single cache that has ever existed in your area in a private database.

 

I think most people only want the active caches plus the archived caches they've hunted or are tracking for whatever reason.

 

We can get notifications of new caches (immediately and/or weekly). Would getting similar type notifications for caches being archived be a possible solution?

 

This is already possible and I'm sure other people are using it as am I. This method is especially good for paper cachers, who can use the notification to be able to purge their binders.

 

BTW: I've never had the problem of archived caches in my GSAK database because I always use only my new PQs instead of adding them to the existing database.

 

Resulting in your PQs probably being larger and more frequent than those designed only to add to and update a database. There was a time not long ago when PQs were extremely unreliable due to load issues. We were told that the determining factor in measuring the load was number of caches returned.

Link to comment

I really don't see what the driving force is behind the need to have every single cache that has ever existed in your area in a private database.

 

I think most people only want the active caches plus the archived caches they've hunted or are tracking for whatever reason.

But finding out about archived caches can be done in a number of ways. Active ones can be tracked thru notifications. Found caches can be done thru the all found PQ. Tracked caches (whether watched or bookmarked) also allow for you to be notified when there are logs posted to them.

 

We can get notifications of new caches (immediately and/or weekly). Would getting similar type notifications for caches being archived be a possible solution?

 

This is already possible and I'm sure other people are using it as am I. This method is especially good for paper cachers, who can use the notification to be able to purge their binders.

This is true, I forgot aout the notification feature. I don't use it at all. But wouldn't doing that solve the problem of people wanting to know when a cache is archived?

 

BTW: I've never had the problem of archived caches in my GSAK database because I always use only my new PQs instead of adding them to the existing database.

 

Resulting in your PQs probably being larger and more frequent than those designed only to add to and update a database. There was a time not long ago when PQs were extremely unreliable due to load issues. We were told that the determining factor in measuring the load was number of caches returned.

I run 3 PQs a week and each one has about a 400 count. That doesn't seem too frequent or too big. They only include caches I have not found. If I want the ones I have found, there's the 'all find' PQ. If a cache is archived it won't show up in my unfound one, but then again, it's archived, why would I want to look for it?

 

There are ways already in place to get notified of and/or identify recently archived caches. Yes, a PQ may be a little more helpful but TPTB have said time and again that it ain't gonna happen...

 

I guess I just don't see the real need for it.

Link to comment

Nice suggestion CR, especially if the archived caches can be set to return GC Id and archived status only. My PQ's, like yours could then be set for only caches with changes in the last 7 days - much reducing load. I expect this will only happen if there's a real drag on the servers. Even then, I'm not sure enough people will understand how it works for it to be useful to the site; it's apparent from reading this thread that a number of responders don't really see the difference between filtering the archived from your own database and only querying the changed in the first place.

Link to comment
I guess I just don't see the real need for it.

 

While it's a moot point, the need is there. I've gone over many of reasons before. Using a search on the subject will provide with a great deal of material, no need to rehash it here.

 

The active-caches-only angle and the 5-log-limit contribute to either higher bandwith or stale data depending on how a user approaches setting up their queries. Both are problems. So, yes, I think I do see the need.

Link to comment

I expect this will only happen if there's a real drag on the servers.

 

It's probably never going to happen - and we've seen plenty of performance issues on this site.

 

I don't think there is any high priority on the site to reduce bandwidth through reducing page reloads etc. It took a long time to go and disable the PQs for people who weren't using them - yet given all the concentration of attention on PQs, you'd think that one of the first analyses would have been PQ traffic versus site login frequency, to determine likely PQs going unused. Look how long it took to implement the client-side encrypt and decrypt hints and even then it was still broken for days even in the most popular browser, IE. There are a lot of things which could boost global performance like implementing web services. Same thing with pocket queries. Imagine how many bytes saved per month as people can switch to last updated and get archived.

 

I would guess the bandwidth costs haven't been worked out. I think Nielsen's book gives a cost comparison - like if you save this many kbytes per page load - this equates to this much profit. I think the old GIF Wizard optimizer use to show your savings per visitor, too. This ain't Amazon.com, but someone's losing money.

Link to comment

I run 3 PQs a week and each one has about a 400 count. That doesn't seem too frequent or too big.

 

Assuming those 3 PQs are disjoint (not the same query), if you had maintained an offline database and switched the PQ to last updated in 7 days, using the same percentage of activity which CR had (14%), and saying 1K per cache, that's:

 

3 x 400 x (100-14)/100 K wasted every week = 1032 (over a MB of data transferred - before zipping - but a server still has to do the zipping, which is also a cost)

 

Multiply up by number of people doing the same thing.

 

I'm sure you get the idea: if everyone switched to this method - gc.com would save 84% of it's bandwidth costs due to PQs.

 

Of course, that's ideal potential savings. In actuality, the savings would not be that large, but people who can optimize their queries and move the work off gc.com's servers can still reduce gc.com's cost to process PQs significantly.

 

For example, I have seen estimates that traditional webmail solutions (not AJAX ones like gmail) cost 10 to 20 times the bandwidth of traditional POP3 solutions which are really lightweight. I imagine a similar savings for web services-based interaction instead of web pages.

Link to comment

We, too, would LOVE to have this functionality. We cache paperless and have, on more than one occasion, spent time fruitlessly looking for a cache, only to find out when I go to log our DNF (and we log ALL of them), that the cache was archived. :anicute:

 

.

Edited by baloo&bd
Link to comment
Nice suggestion CR, especially if the archived caches can be set to return GC Id and archived status only. My PQ's, like yours could then be set for only caches with changes in the last 7 days...

I agree with this 100%. I don't want the description or hint, just the GC code and archive status; just for the recent past.

 

Although: It would be useful to have any final logs by owner or reviewers so that we have an indication as to whether something is problematic with the cache site. Many times the archiving is because of muggle or safety problems; that would be very useful information.

Edited by Hynr
Link to comment
Guest
This topic is now closed to further replies.
×
×
  • Create New...