Jump to content

Question for PQ users


mambero

Recommended Posts

I was not a cacher when I started here, and while I have enjoyed finding my first few caches, there is still much about the culture I don't understand. Developers are normally not encouraged/allowed to communicate directly with the users of their software, but that is not the case at Groundspeak, as I recently/traumatically discovered. So, now that I have access to the collective, I have some questions.

 

While finding ways to optimize PQ generation, I've heard from several users that it's important they receive the PQs at the same time every day. Why is this?

Is it

a ) because you head out the door just after your receive it

b ) the time you have set aside to merge the PQ into your tool-of-choice

c ) if you don't receive it at that time, you know something is wrong with the system

d ) other: ______________________________ (please fill in the blank)

 

One of the ways we can speed up the processing of data is by caching the data for caches. Cached data is not real time. And if you think about it, PQs are just a form of cached data. As soon as we send it, it can become stale. The longer it sits in your inbox, the more stale it becomes. The longer it sits on your device the more stale it becomes. The question is, how fresh does it need to be when it leaves our server. Obviously, fresher is better.

 

Let's say we were able to get to this {b}hypothetical{/b] point of delivering PQs based on TZ, which would mean you would receive your PQs sometime between 2-6am ish. Would you care if it came at the same time every day? Would you care if the data in the caches was one or two hours old since it may already be several hours since we sent it to you and it's unlikely there would have been much activity on the cache at midnight? Of course there are edge cases where occasionally a cache log will be updated very late in the evening, but it doesn't appear to be the norm, and if we could dramatically reduce the load on the db, it could improve our chances of processing around the clock.

 

disclaimer: Because I'm an employee at Groundspeak, it's easy misread my words as promises or directions we're headed. I'm just on a fact gathering mission here. I want to understand the community better so that I can keep your needs in mind, and I want to collect pros & cons to ideas as early as possible, which means that often, my questions are just probing.

Edited by drewburlingame
Link to comment

I can't speak for others, but this is how I run my pocket queries. I have several pocket queries designed. I normally tend to check specific ones to run as late in the day as I can. That way I can have the latest data for the next day. I generally will do this after the reviewer has done his thing for the day so I can pick up any newly published caches. I will always run five PQs. Another reason I do it at the end of the day is so that if something comes up during the day, I can create and run a PQ (or more) if needed. If I ran the PQs by schedule, they would run in the morning and I would not be able to do on the fly PQs since my 5/day would have already ran.

Link to comment

My personal response (and thanks, btw, for asking!) - I don't care what time of day they are run / received. Though it would be preferable if they arrived at the same time.

 

Speaking as someone from "across the pond", it would seem to me that the most sensible solution to spread server load would be to run based on my home location TZ. Say, starting at 2am for each TZ so that the data is as "fresh" as you can realistically make it.

 

Cacheing worries me slightly - though by their nature, offline DBs or GPX files are stale from the moment they are published, we do at least expect the GS data to be accurate. If you're talking about cacheing data so that in the "worst case" scenario the data is 30 minutes out of date, that would be fine. If the data was 24 hours out of date, not fine. Somewhere in between is the fine line to be drawn.

 

I would then also stress that it would make SOOOOO much sense to show my local time on the various pages that display it, so that if I want to schedule a cache for Saturday, I don't have to think about what time that means in Groundspeak-terms.

 

Sure you'd still get some peaks and troughs depending on cacher density around the world, but you wouldn't be trying to run all of the european PQs at the same time as all of the US.

 

HTH!

 

Matt

Edited by teamhillside
Link to comment

b and c.

 

My scheduled PQs tend to run while I sleep and when I get up in the morning I merge and update my database. It's likely all the caching activity from yesterday will be reflected, but I've never considered this to be absolutely required (some people will log late anyway.)

 

2-6am ish would be fine. I wouldn't care when in that interval. 1-2 hours old would never really be a problem.

 

(MyFinds is an exception. When I run that it is normally because I have just logged finds and would want them included in the PQ.)

Link to comment

I can't speak for others, but this is how I run my pocket queries. I have several pocket queries designed. I normally tend to check specific ones to run as late in the day as I can. That way I can have the latest data for the next day. I generally will do this after the reviewer has done his thing for the day so I can pick up any newly published caches. I will always run five PQs. Another reason I do it at the end of the day is so that if something comes up during the day, I can create and run a PQ (or more) if needed. If I ran the PQs by schedule, they would run in the morning and I would not be able to do on the fly PQs since my 5/day would have already ran.

 

Hey, that's a point I hadn't thought of at all. Run the PQs at the end of the day so that you have as many "on demands" as possible the next day. It would be a trade off with not having all the info for that day. Do you find that you miss many cache logs with this approach. I've been told that it's common for cachers to log their find later in the evening, and that it's important to gather those logs to see the DNFs.

Link to comment
Let's say we were able to get to this hypothetical point of delivering PQs based on TZ, which would mean you would receive your PQs sometime between 2-6am ish.
I realize that you're in the early stages of planning this out, but I have a quick question on the TZ scheduling.

 

Would the scheduling be based upon the TZ of the Home coordinates set up in your profile? If so, how would that work when you're on vacation. Let's suppose that I live in NY, and go on vacation in Hawaii for 2 weeks. While I'm there, would the PQs I have set up for caching in Hawaii run on the NY schedule (many hours earlier than the time in Hawaii)?

 

Actually, that might be more of a concern for those traveling east - if you're in Europe, the PQs would end up running 5-6 hours later than desired.

Link to comment
Let's say we were able to get to this hypothetical point of delivering PQs based on TZ, which would mean you would receive your PQs sometime between 2-6am ish.
I realize that you're in the early stages of planning this out, but I have a quick question on the TZ scheduling.

 

Would the scheduling be based upon the TZ of the Home coordinates set up in your profile?

 

http://www.geocaching.com/account/ManagePreferences.aspx

 

It would make sense for it to be based on what you set in your preferences. (Which you could change when going to Hawaii.)

Link to comment

A and C. I like to have all the day's PQs before I leave in the morning. But I don't mind if some of them are early but others I'd like a late as possible before my departure time. Maybe have a way to mark certain PQs as "Allow to run early"?

 

I don't like the TZ idea for various reasons. A post in another thread suggested using the PQs coords to figure out which time to run it. That could be computed when the PQ is saved if server load is an issue.

Link to comment

I do not live in a very cache rich area (in addition I have found a majority of the nearby caches). I do not travel much and do not have time to cache on most weekdays.

 

I run one saved PQ weekly. 30 miles gets me 300 unfound caches. I run it on Thursdays so obviously by Saturday my data is stale. But I'd prefer to have my data and have a route planned then grab at the last second.

 

When I take a trip it's very similar I run the PQ a couple days ahead so I have all the data then I use GSAk and Google earth to plan my route. I prefer having a route to maximize my efficiency and fun.

 

Some have suggested caching data for large cities and that may be a good idea. Plus they can use live PQ's to supplement this data.

 

<My guess is that everyone is different and there will be no solution for all. Splitting up the time zones should help for saved PQ's and the 1000 limit will make those in cache rich areas happier.>

Link to comment
Let's say we were able to get to this hypothetical point of delivering PQs based on TZ, which would mean you would receive your PQs sometime between 2-6am ish.
I realize that you're in the early stages of planning this out, but I have a quick question on the TZ scheduling.

 

Would the scheduling be based upon the TZ of the Home coordinates set up in your profile? If so, how would that work when you're on vacation. Let's suppose that I live in NY, and go on vacation in Hawaii for 2 weeks. While I'm there, would the PQs I have set up for caching in Hawaii run on the NY schedule (many hours earlier than the time in Hawaii)?

 

Actually, that might be more of a concern for those traveling east - if you're in Europe, the PQs would end up running 5-6 hours later than desired.

 

My thoughts now are that the PQ screen allows you to select which time and TZ to queue the PQ for, and default it to the cachers TZ. The immediate downside I see is that people will assume that's when their PQ will be processed. When a large group of PQs are assigned the same time, it may take several hours for the queue to process. Perhaps we could get away with a not saying that you're PQ may take several hours to process. If we included last queued & processed time on the PQ screen, you'd be able to adjust your time accordingly. I just had a mental picture of an entire country alternating their PQ time an hour forward one night and an hour back the next, over and over.

Link to comment

I do not live in a very cache rich area (in addition I have found a majority of the nearby caches). I do not travel much and do not have time to cache on most weekdays.

 

I run one saved PQ weekly. 30 miles gets me 300 unfound caches. I run it on Thursdays so obviously by Saturday my data is stale. But I'd prefer to have my data and have a route planned then grab at the last second.

 

When I take a trip it's very similar I run the PQ a couple days ahead so I have all the data then I use GSAk and Google earth to plan my route. I prefer having a route to maximize my efficiency and fun.

 

Some have suggested caching data for large cities and that may be a good idea. Plus they can use live PQ's to supplement this data.

 

<My guess is that everyone is different and there will be no solution for all. Splitting up the time zones should help for saved PQ's and the 1000 limit will make those in cache rich areas happier.>

 

Is there a market on ebay for unused PQs? Maybe you can rent out your extra 34 PQs/week.

Link to comment

I do not live in a very cache rich area (in addition I have found a majority of the nearby caches). I do not travel much and do not have time to cache on most weekdays.

 

I run one saved PQ weekly. 30 miles gets me 300 unfound caches. I run it on Thursdays so obviously by Saturday my data is stale. But I'd prefer to have my data and have a route planned then grab at the last second.

 

When I take a trip it's very similar I run the PQ a couple days ahead so I have all the data then I use GSAk and Google earth to plan my route. I prefer having a route to maximize my efficiency and fun.

 

Some have suggested caching data for large cities and that may be a good idea. Plus they can use live PQ's to supplement this data.

 

<My guess is that everyone is different and there will be no solution for all. Splitting up the time zones should help for saved PQ's and the 1000 limit will make those in cache rich areas happier.>

 

Is there a market on ebay for unused PQs? Maybe you can rent out your extra 34 PQs/week.

I was under the impression that doing so would be a TOU violation, covered under "thou shalt not share thy data."

Link to comment

Is there a market on ebay for unused PQs? Maybe you can rent out your extra 34 PQs/week.

I was under the impression that doing so would be a TOU violation, covered under "thou shalt not share thy data."

And (if we're taking this idea seriously) for thirty bucks a year anyone can get another 40 of their very own.

Link to comment

I do not live in a very cache rich area (in addition I have found a majority of the nearby caches). I do not travel much and do not have time to cache on most weekdays.

 

I run one saved PQ weekly. 30 miles gets me 300 unfound caches. I run it on Thursdays so obviously by Saturday my data is stale. But I'd prefer to have my data and have a route planned then grab at the last second.

 

When I take a trip it's very similar I run the PQ a couple days ahead so I have all the data then I use GSAk and Google earth to plan my route. I prefer having a route to maximize my efficiency and fun.

 

Some have suggested caching data for large cities and that may be a good idea. Plus they can use live PQ's to supplement this data.

 

<My guess is that everyone is different and there will be no solution for all. Splitting up the time zones should help for saved PQ's and the 1000 limit will make those in cache rich areas happier.>

When you say caching data, do you mean maintaining a database for your home area? Maybe it's just me, but I thought most GSAK users did that?

 

The way I read what drewburlingame said about caching data, the idea was that GS could supply (slightly) old data to users. If that happened, where would we get "live PQs"?

 

Why not run Thursday's PQ again Saturday? Or Friday night?

 

Why not take one minute in the morning to open one GoogleMap of the day's caches and see if any box is grayed out, missing, or new (a thing I forget to do most days, and that's why I'm getting on someone else's case about it)?

Link to comment

I might be a minority because I cache for the location and not that numbers. Most of my scheduled PQ's are based on a bookmark list. For me any cache on a bookmark list is usually a place I want to visit instead of a cache I want to find. That being said I don't generally care if cache data is old. In fact I only run my query once a week. What I do look for is if I get my query on the day it was supposed to run. If so I know the PQ generator is running.

 

Now for the question you didn't ask.

I would love a way to move caches from one bookmark list to another so my pocket query accuratly reflects the statues of a cache. For instance if a cache is archived I still want to see the location. If I have a pocket query of archived caches I can load them on my gps differently so that I know that I shouldn't be looking for a cache and instead just the location is of interest. I would also like the pocket query to include the bookmark note as a log so I know what it is I am going to see at the location.

Link to comment

Is there a market on ebay for unused PQs? Maybe you can rent out your extra 34 PQs/week.

I was under the impression that doing so would be a TOU violation, covered under "thou shalt not share thy data."

And (if we're taking this idea seriously) for thirty bucks a year anyone can get another 40 of their very own.

 

We weren't taking this idea seriously. It was a joke. I've never been much of an emoticon guy, but it looks like I'll need find the "I'm joking" emoticon.

Link to comment

The way I read what drewburlingame said about caching data, the idea was that GS could supply (slightly) old data to users. If that happened, where would we get "live PQs"?

 

One of my points is that "live PQs" is a misnomer in that, as soon as the data is extracted from our database, it becomes stale data. As we're generating the gpx file, as it's being sent to your inbox, as you drive/hike to your destination, new caches, logs, etc could be added. Caches could be archived. The only way to know is to check the site. If you can access the site, you're less likely to need a gpx file.

 

As long as we know the PQ is stale, the question becomes, how stale can it be. Obviously less stale is better, and one of the goals is to give you the freshest data possible. Currently, it's live data and we'd like to keep it that way. The downside is that our window for generating PQs is restricted to times the database is not under load from the web site. Copying that data to a second database would remove the restriction while generating PQs, but copying the data is also resource intensive for the database. If we can copy updated items every X minutes (where X is 30 or 45 or ...), we can reduce database load enough to run PQs around the clock.

 

So, it becomes a trade-off. Receive your PQs when you want them with slightly stale data, or receive your PQs when we can send them with fresh data.

 

[disclaimer: This is still all hypothetical]

Link to comment

Far as scheduled PQ's are concerned, I really don't care because my life is not ordered in that manner. But when I run a PQ I want it to run *now*, not at midnight or some time related to my timezone. As long as what you do does not upset my apple cart I will be happy. As for the PQ's I run they consist of bookmark based PQ's, newly defined CAAR PQ's, saved regular PQ's, and newly defined regular PQ's. All of my PQ's have the day to run unchecked and have the uncheck when run option set. When I want fresh data I just check the current day and wait for the data. Ideally I don't have to wait long.

 

And thank you for asking for the community's ideas. I see no other way for the developers to gain the information they need for direction other than the forums or attending events and talking to folks.

Edited by jholly
Link to comment

Is there a market on ebay for unused PQs? Maybe you can rent out your extra 34 PQs/week.

I was under the impression that doing so would be a TOU violation, covered under "thou shalt not share thy data."

And (if we're taking this idea seriously) for thirty bucks a year anyone can get another 40 of their very own.

 

We weren't taking this idea seriously. It was a joke. I've never been much of an emoticon guy, but it looks like I'll need find the "I'm joking" emoticon.

Are you serious?

Link to comment

First, thanks drewburlingame for asking us this, even though you didn't say anything is happening.

 

Answering for myself, I use PQs to keep my GSAK up to date. Most run weekly, some twice, a few three times. Sure, I'd like them to be perfect, but the fact is I never sit waiting for them and sometimes they don't get loaded into my system for quite a while.

 

And then I use GSAK as a backdrop to the live data - when I'm using it to plan a day, I hit f4, look at the live web pages and put the caches on my watchlist. So it makes no difference to my life if you cache them every thirty minutes. I'd feel more comfortable if you just ran the PQs later in the day, but it makes no real difference.

 

But one day in the next year or two I will follow the trend and become paperless, and then my answer will be different. Then - I guess - my answer will be "sure, cache most of it but I'd like to be totally up to date in regard to publication, archival, enabling and disabling".

 

[edit to remove a proposal which - when I thought about it - was what GS already does :anicute: ]

Edited by Cairngorm
Link to comment
Would you care if it came at the same time every day?
No. My first priority is getting the PQ in the first place.

 

Would you care if the data in the caches was one or two hours old since it may already be several hours since we sent it to you and it's unlikely there would have been much activity on the cache at midnight?
Of course not. The folks who are clamoring for the absolutely freshest data really don't get what fresh data really is.

 

First, there are two different pieces of data we're talking about. One is the complete PQ. That's the data of all the caches within that PQ and how fresh it is on a whole. The second is how fresh the data is on any one particular cache. A single cache's data is as fresh as the last log and nothing more. This could be years old in rare cases.

 

Of course there are edge cases where occasionally a cache log will be updated very late in the evening, but it doesn't appear to be the norm, and if we could dramatically reduce the load on the db, it could improve our chances of processing around the clock.

I think it is much more important to actually get the PQs than risk not getting them because you're wanting the latest "just in case."

 

How I pull PQs:

I maintain an OLDB (Off Line Data Base) because there have been times where PQs don't run and was without the latest data and had to use old data. Now, I just go ahead and keep all of my old data. This also helps with 5 log limitation, too. There have been plenty of times where older logs help with hints to find the cache--some are simply accurate coords.

 

I pull PQs through out the week of the areas we like to cache in. A few day old data is no big deal. If I miss a brand new cache I'll get it on the next trip. The only data that I like to have the latest data on is the local stuff and that's really just for grins.

 

What I done is create an environment where I don't have to worry if Groundspeak is going to have my data on Saturday morning. If they do, fine. If not, that's fine, too. I've got data. I can load up the car and haul behind to where ever the winds take us. I can even change my mind and turn around and go somewhere else. I'm not planning anything on GC.com days before. We just go.

 

As a result, when I read on the forums the teeth gnashing of not getting PQs, I simply shrug and move on. I've got mine.

 

Getting the very latest data could be successfully accomplished if your boss would allow some sort of indication that caches are getting archived. It could be set up for use on only PQs that are marked to provide data of only that caches updated in the last 7 days. It could also be a simple list of waypoints of caches that have been archived in the last 7 days. Let GSAK sort that out. That way folks could be getting full data sets earlier in the week and differential PQs at the last minute for those last minute publishings and archivals. Then, if that data doesn't come before they're heading out of the house they'll at least have data that is only a couple days old instead of a week.

 

Getting the archived data thing past your boss is the issue, really.

 

Anyway, to answer your question: a few hours old data is much better than risking not getting any data at all. I'd explore the spreading out of processing by user provided TZ information.

 

Further saving could be made with canned PQs (discussed plenty of times here on the forums). Also, enhancing and encouraging OLDBs use even more by optimizing differential data. (Like only sending the data that has changed and not everything. When looking at differential PQs, by far the most that is changed is a single additional log. Why send the whole cache data, descriptions, etc. Massive saving on differential PQs if fully explored. Like file sizes of less than 5% of the full dataset. Add the idea of optimized differentials to canned PQs and the work load is cut massively.) This is just an idea for when you guys overhaul the PQ system and I understand it's not likely to happen anytime soon.

Link to comment

As long as we know the PQ is stale, the question becomes, how stale can it be. Obviously less stale is better, and one of the goals is to give you the freshest data possible. Currently, it's live data and we'd like to keep it that way. The downside is that our window for generating PQs is restricted to times the database is not under load from the web site. Copying that data to a second database would remove the restriction while generating PQs, but copying the data is also resource intensive for the database. If we can copy updated items every X minutes (where X is 30 or 45 or ...), we can reduce database load enough to run PQs around the clock.

 

So, it becomes a trade-off. Receive your PQs when you want them with slightly stale data, or receive your PQs when we can send them with fresh data.

 

[disclaimer: This is still all hypothetical]

why not give the users an option? if the PQ page lets them choose whether they want the PQ to return fresh live data, or data that's let's say up to 60 minutes old, then you could reward them for choosing the second option by counting this PQ only as half a PQ towards the daily limit, or something like that.

Link to comment

Just my 2 cents worth.

We are in a cache rich area and we never know what dirrection the caching bug will bring us. We could go 2 miles south or 136 miles North. I run PQ's by date placed and update them once a week. I know the info could be a week old but that is fine.

 

So the "time of day" i get it, is no problem, as long as i get them to update GSAK.

 

I would rather run less PQ's and get more in them.

 

Thanks for asking.

Link to comment

How about letting weekly scheduled PQs run later in the day (because nobody's impatiently waiting for them) and letting new or manual requests run immediately? I realize this is a reversal of current policy but wouldn't it lighten the server load?

:anicute: That's how it works now or did I miss a memo?

Link to comment

why not give the users an option? if the PQ page lets them choose whether they want the PQ to return fresh live data, or data that's let's say up to 60 minutes old, then you could reward them for choosing the second option by counting this PQ only as half a PQ towards the daily limit, or something like that.

That's what I suggested, but I like the additional idea of a reward/incentive for choosing it.

Link to comment

How about letting weekly scheduled PQs run later in the day (because nobody's impatiently waiting for them) and letting new or manual requests run immediately? I realize this is a reversal of current policy but wouldn't it lighten the server load?

:anicute: That's how it works now or did I miss a memo?

I think currently the ones which have not run recently take priority? I'm suggesting the opposite.

 

[edit:]Wait . . . no, I'm not. Boy, I've been back and forward on this logic all morning. I need to make a chart. But I have to go do my US taxes instead.

Edited by Cairngorm
Link to comment

You asked so here is what I do an why.

 

I live on Long Island. To go "off Island" I want to run a PQ for the area I am visiting. I generally do it the night before the trip because I don't trust the PQ generator to give it to me before I head off in the morning.

Usually, if it's a new PQ it comes right away, but at least that gives it time. I don't really care that the data might be old by the morning. But I do want to get it, there are mornings I leave at 5 AM to drive 2-3 hours away for an event hike. I don't think the TZ idea is gonna fly.

 

I don't keep the database for those areas updated until I am heading there. There are areas I visit repeatedly, Boston, Delaware, CT,areas in New Jersey and the Tri States area of PA, NJ and NY.

 

If I want to be certain to get it ASAP because an excursion came up last minute I create a new one. The TZ rule would screw that up.

 

For my area I keep a GSAK database and run 5 PQ's . This will change when the raise from 500 to 1000 goes into effect. Anyway, the three oldest dated ones I run on Sunday. I run one for the Eastern end of the island on Saturday morning, it starts 50 miles out and I won't likely be just happening out that way, but....you never know.. The newer dated ones I run Friday for caches published during the week and Monday to get new caches that were published on the weekend. I don't really need them early in the day, but when they were new they were always there when I got up. They have gotten later and later as subsequent slowdowns and downtimes have happened. When they are very late I check time last run and I can tell if there seems to be a problem. I keep a database of this type because I find myself all over Long Island and don't want to be hunting a cache that's been disabled or had a number of DNF's which, before I learned how to keep it up to date happened numerous times. Caches that I deem not worthy of worrying about whether it ever gets repaired I put on ignore.

Link to comment
The TZ rule would screw that up.

Maybe I've missed something. Why would starting PQ generation based on TZ screw your plans up?

 

As far as I've seen they're talking about starting PQ generation at midnight your time instead of Groundspeak time. That would mean you'd be getting your scheduled PQ 3 hours earlier than you are now, if not sooner.

 

Unless they change anything else, everyone would benefit. Everyone east of Groundspeak would be getting their PQs earlier because they started generating earlier. Folks on Groundspeak time and west would also benefit by getting their PQ earlier because everyone east of them have already had theirs run and they aren't competing for slots with folks west of them. Win-win all around.

 

...if nothing else changes, that is. You should still be able to get brand new PQs immediately.

Link to comment
The TZ rule would screw that up.

Maybe I've missed something. Why would starting PQ generation based on TZ screw your plans up?

 

As far as I've seen they're talking about starting PQ generation at midnight your time instead of Groundspeak time. That would mean you'd be getting your scheduled PQ 3 hours earlier than you are now, if not sooner.

 

Unless they change anything else, everyone would benefit. Everyone east of Groundspeak would be getting their PQs earlier because they started generating earlier. Folks on Groundspeak time and west would also benefit by getting their PQ earlier because everyone east of them have already had theirs run and they aren't competing for slots with folks west of them. Win-win all around.

 

...if nothing else changes, that is. You should still be able to get brand new PQs immediately.

 

Maybe I am misunderstanding then.....when would one I create at 6 AM run? Would a new PQ still run right away as it does now?

Link to comment
Maybe I am misunderstanding then.....when would one I create at 6 AM run? Would a new PQ still run right away as it does now?

As far as I've read--and I'm pretty certain I've not read everything--the only thing they're thinking of changing along those lines is when the PQs run. Instead of starting at midnight Pacific, they had first floated starting a 8PM. Then floated starting generation based on time zone.

 

If that's all they're doing, then I don't see why brand new PQs wouldn't run immediately.

 

As far as generation, except for the start time, the scheduling scheme is pretty close to optimal. What is sent could be adjusted, but that's been complained about for years.

Link to comment

Maybe I am misunderstanding then.....when would one I create at 6 AM run? Would a new PQ still run right away as it does now?

 

Yes. When you create a new PQ, it will be put in a high priority queue to run a.s.a.p., just as we currently do. We're only talking about adjusting the scheduled PQ start times.

Link to comment

Maybe I am misunderstanding then.....when would one I create at 6 AM run? Would a new PQ still run right away as it does now?

 

Yes. When you create a new PQ, it will be put in a high priority queue to run a.s.a.p., just as we currently do. We're only talking about adjusting the scheduled PQ start times.

 

Good deal, go for it then. If it's really necessary to get something ASAP then folks can copy and rename one.

Of course there are always going to be those that want them NOW ALL THE TIME!! You aren't going to change that behavior. :anicute:

Link to comment

I would like to receive mt PQ's at the same time, especially on weekends. I get several PQ's during the week, save them, and load them into my GPS'r as needed.

 

I also would like to see the ability to run one mega PQ say on a weekly/daily basis similar to My Finds PQ. I know that PQ size will be increasing soon to 1000, but the ability to run 2000 or more in a specific PQ would cut down on my need for mulitple PQ's. I could then supplement this with a daily New Cache PQ to capture any new hides or hides that were re-enabled. I may still want seperate smaller PQ's depending upon what I am doing during the week, but a Mega PQ would be nice.

Link to comment

I can't speak for others, but this is how I run my pocket queries. I have several pocket queries designed. I normally tend to check specific ones to run as late in the day as I can. That way I can have the latest data for the next day. I generally will do this after the reviewer has done his thing for the day so I can pick up any newly published caches. I will always run five PQs. Another reason I do it at the end of the day is so that if something comes up during the day, I can create and run a PQ (or more) if needed. If I ran the PQs by schedule, they would run in the morning and I would not be able to do on the fly PQs since my 5/day would have already ran.

 

Hey, that's a point I hadn't thought of at all. Run the PQs at the end of the day so that you have as many "on demands" as possible the next day. It would be a trade off with not having all the info for that day. Do you find that you miss many cache logs with this approach. I've been told that it's common for cachers to log their find later in the evening, and that it's important to gather those logs to see the DNFs.

I don't worry about any logs that occur after the PQ runs. I'll get them the next time I run the PQ. My main concern is having a good list of the active caches and their correct coordinates. I maintain a GSAK DB and use the PQs to keep it reasonably up to date.

 

The hardest part is seeing when caches are archived that fall outside of my notification range of 50 miles. A way to get a list of recently archived caches would be very helpful.

Edited by Corfman Clan
Link to comment

While finding ways to optimize PQ generation, I've heard from several users that it's important they receive the PQs at the same time every day. Why is this?

Is it

a ) because you head out the door just after your receive it

b ) the time you have set aside to merge the PQ into your tool-of-choice

c ) if you don't receive it at that time, you know something is wrong with the system

d ) other: ______________________________ (please fill in the blank)

 

For me, it's A B & C. I wake up in the morning, process the PQs in my Inbox, update my GPS and head out the door. If they haven't arrived, I will be missing current data for a particular area and change the direction I travel that day as my data would be a full week old now. Also, if they haven't arrived, I typically assume there is a problem with the system.

 

The data I use for caching is almost always 1 to 6 days old, so a few hours or even a day doesn't matter to me if I could be sure I would find my daily dose of PQs every morning when I wake up. :)

Link to comment

To answer some questions about my original post:

 

I like to have a plan and there's very little change between Thursday and Saturday in my area. Most enabling and disabling happen early in the week. Like others have said I just need the coords and cache description, if I don't have the newest logs it's okay. If I miss a brand new cache it's okay.

 

I consider what we get right now "live" PQ's. And I saw in the other thread people wanting city (county, statewide even) data and they suggested having a weekly(monthly) cached data for a certain area. So I was just recycling that suggestion. Of course others have already suggested better options, and I'm no super techie so I don't know what would be best.

 

Sometimes I wish I lived in a much higher cache density (especially during my 100 day streak) and had to worry about more caches then I could download.

Link to comment

One thing that I would really welcome is to add the search criterion, "Updated in the last 2 days" in addition to the "Updated in the last 7 days" criterion. Either that or allow the number of days to be configurable. To me, that would make my PQs much more effective.

This is a concern here as well. I tried streamlining my process by utilizing the "Updated in the last 7 days" option. Since my PQs cover a 30 mile radius from home coordinates (by date placed), this easily translates to >500 caches returned. I tried splitting it up into two PQs based on date placed, but that just was a pain because the center point (median) of the cache list kept moving back and forth - some days more than 500 were in one list and much less than 500 in the other. The next day was the opposite. Going to three lists was just crazy-making.

 

Now, I like the TZ idea. It would spread out the PQ generation load across the globe.

 

Next, when do I want it generated? I prefer to get up in the morning, load my newest set of PQs to update my GSAK and export to the GPS units. Right now, the whole process takes me 15 minutes total from PQ receipt to out-the-door. If I don't get new PQs, then I'll go without updating.

 

The impact? Only if I can't find the cache I'm seeking causes an issue. With my phone, I can access the live database from anywhere I have phone service (choose wisely here!!). I've noticed that, on average, only about 10 caches are archived/disabled every day (out of a cache set of 5000). The odds of me heading for any one of them is remote.

 

Bottomline: Just send me live data in the early morning hours. 2am-6am is fine.

Link to comment

Living in North America, since the PQ run after midnight, they usually have the latest caches since most reviewers do a a lot of work in the evening. And the PQ are in my mail box by the time I get up in the morning. So while the date is stale a few hours by the time I get it, it is usually pretty accurate. I suppose some reviewers might be publishing caches at 2 AM so I might miss one or two caches once in awhile.

 

I live in a cache dense area, so in fact most of my data is stale by a few days when I use it. I run scheduled PQ throughout the week with the most recent caches run on Friday or Saturday just before the weekend. Older place by date PQ change less often than newer placed by dates.

 

I wouldn't mind slightly stale data on the scheduled PQs or have them run slightly later (just so long as the Friday/Saturday one arrive by 6AM.)

 

On the other hand I will often run an on demand query targeted to the explicit area or bookmark list for where I plan to cache just before I go out. These I would want to see run with the highest priority and use the latest data.

 

When the PQs are increased to 1000 caches, I might explore running fewer scheduled PQs and saving more PQ for areas where I cache where I might want to get the latest just before going out. The GSAK database would serve as a backup for whey I can't get on demand PQs and for storing notes and solved puzzles. But it could even have data a few weeks old. I would get 1000 caches in the area where I am going to just before heading out, if I knew that they would arrive reliably in a few minutes.

 

If I were the developer, I probably wouldn't worry about optimizing scheduled pocket queries, especially if you couldn't distinguish them from a saved query that only gets run when needed. Sounds like you already have some ideas on how to speed up the processing. The distributed cache you are talking about could have could expire data older than 10 or 15 minutes, and I don't think that will be too bad. You won't have problem except when someone gets an instant notification and then requests a PQ to find the new cache is not in it.

Link to comment

it looks like when it comes to the time of day when the PQ would be generated, it's impossible to please everybody. personally i would think giving the users an option of when they'd like to have the PQ run would be the best solution (something like having the choice between no preference, 0-6 am, 6-12 am, etc, without guaranteeing it, but rather stating it as "best effort"), but i guess that would rather cause additional load on the servers instead of easing it.

Link to comment

If I were the developer, I probably wouldn't worry about optimizing scheduled pocket queries, especially if you couldn't distinguish them from a saved query that only gets run when needed.

 

Wouldn't a saved PQ with the uncheck after run attribute checked be a saved pocket query that gets run only when needed?

Link to comment

 

While finding ways to optimize PQ generation, I've heard from several users that it's important they receive the PQs at the same time every day. Why is this?

Is it

a ) because you head out the door just after your receive it

b ) the time you have set aside to merge the PQ into your tool-of-choice

c ) if you don't receive it at that time, you know something is wrong with the system

d ) other: ______________________________ (please fill in the blank)

 

One of the ways we can speed up the processing of data is by caching the data for caches. Cached data is not real time. And if you think about it, PQs are just a form of cached data. As soon as we send it, it can become stale. The longer it sits in your inbox, the more stale it becomes. The longer it sits on your device the more stale it becomes. The question is, how fresh does it need to be when it leaves our server. Obviously, fresher is better.

Personally, I get Updates for the States around me. That is Updates to the MO, and KS PQs.

 

I completely erase both DBs every 6 Months, at which time, I take 3 days(will be less once PQ Size is increased YEAH!!!) and dedicate them to Database cleaning.(erase, and get all new data)

 

My Weekly PQs are all set up as such. Any and All, that I haven't found, I don't own, Has been updated in the last 7 days, and is Active (in State*either KS, or MO*). Each State takes 5 PQs sorted by Date(00-06,07-08,08-09,09,09-10) as needed to get just UNDER 500 for that PQ.

 

Thus, I am Always working on stale data. Heck, the last time I reloaded the GPS was over 2 Months ago(completely...), but That's what the Blackberry, and coord.info/gcCODE is for...

 

The thing to remember is that until there is a seamless integration with GSAK, and I can get PQs of Archived caches, the data will ALWAYS be 'Stale' it may be a few hours, or days(or weeks..months), but still the same STALE! I know the iPhone app links directly to the site, but it STILL has issues...(service area stuff), and there is NOT A Blackberry App! Seriously, the Blackberry STILL Controls the Smartphone market, but no app...

 

I'll try and get back on topic...

I would Like to have all my PQs hit my Inbox before 7a CST(5a PST) on the specified days. But, if I make a new PQ and it is After 7a, I get that PQ ASAP. Of course, this still falls under the 5/day rule.

 

The Steaks

Edited by eagsc7
Link to comment

Further saving could be made with canned PQs (discussed plenty of times here on the forums). Also, enhancing and encouraging OLDBs use even more by optimizing differential data. (Like only sending the data that has changed and not everything. When looking at differential PQs, by far the most that is changed is a single additional log. Why send the whole cache data, descriptions, etc. Massive saving on differential PQs if fully explored. Like file sizes of less than 5% of the full dataset. Add the idea of optimized differentials to canned PQs and the work load is cut massively.) This is just an idea for when you guys overhaul the PQ system and I understand it's not likely to happen anytime soon.

 

Can I suggest the first Few "CPQs"(Canned Pocket Queries)

1) All Active Caches within XX State

2) Archived caches within XX State from the last 6 Months

3) all archived caches within xx state

 

I know 3 will NEVER happen, but that's be nice to see the TRUE history of this GREAT SPORT!

Link to comment

Maybe I am misunderstanding then.....when would one I create at 6 AM run? Would a new PQ still run right away as it does now?

 

Yes. When you create a new PQ, it will be put in a high priority queue to run a.s.a.p., just as we currently do. We're only talking about adjusting the scheduled PQ start times.

 

Good deal, go for it then. If it's really necessary to get something ASAP then folks can copy and rename one.

Of course there are always going to be those that want them NOW ALL THE TIME!! You aren't going to change that behavior. :)

 

Well while we are updating the PQ system, we might as well add a 'run again' button to be able to instantly run a PQ again which has already run that day.

 

It's silly to have to 'copy' a PQ just so you can run it again. There should be a way to run a PQ more than once, which would of course up the 'run count' on that PQ to 2.

Link to comment

Fascinating topic, especially for a non-premium member.

 

Am I getting this right? The hypothetical plan is to cache PQ data on a second server/database. This data will be updated every XX minutes. As a result, PQ data may be a bit old when generated (by XX minutes + YY minutes for everything else such as receiving email, driving etc).

 

What data would be cached? All the information for the cache? This seems like a lot! Why not have some cache attributes (such as active/inactive or change of coords) which are 'high priority', and when changed by the owner they update the PQ cache immediately? Obviously, the PQ cache would still be update every XX minutes as well.

 

The problem then is to define the high priority attributes...

 

Just thinking!

Link to comment

I live in Australia and my preference is to have my PQ available for me early in the morning but to do this I have found that I have to copy the PQ and activate it for the previous day (by GC HQ time). so that it runs immediately.

 

If I tick a prerun PQ to run it will run sometime after about 5pm local time and as our reviewers and many cachers who are still returning home have not completed their logging and it could miss out on a lot of the important logs and new caches.

 

My preference would be that if I ticked the box for Thursday that it would run sometime after Midnight Wednesday night MY local time.

 

I also have the problem that if I want to run an ad hoc PQ late in the afternoon I do not know if I have to tick it for today or yesterday to get it to run immediately. (Normally I tick them both to be on the safe side).

Link to comment

I update my GPS on Friday. The PQs for older caches run earlier in the week. (No. I don't care that by Friday the data is four days stale.) The ones set to run on Friday, however, I would like it very much if they would arrive before I get up in the morning. Then I can update my GPS, and have it ready to go. (Currently not working on Fridays.) Yesterday's last PQ can through at 7:01, which isn't too bad. But, they have frequently come through later in the day. I run the New Caches in the Past Week on Thursday, so I am sure of having them on Friday morning. That means it's always a day old. That one I would love to have ready for me on Friday morning!

Link to comment

Off line Database from weekly PQ's from date placed for local caches..

 

A days caching sorted the night before.

(Friday night, for Saturdays trip, Saturday night for Sunday, or Friday night for Sat/Sun.)

 

If I need an up to date PQ I run a new one for the area I'm going to. Yes 12 hours old or more!

I think most weekend logging has been done and should be on a Friday PQ.

 

Nice if the weekly PQ's are there in the morning (UK time!)

Link to comment
Guest
This topic is now closed to further replies.
×
×
  • Create New...