Jump to content

PQs don't run

sejtam

By sejtam
in Website

Followers 4

Recommended Posts

sejtam

+sejtam

Posted
Posted

I have two PQs which refuse to run. They are both 1000s (and both will find 1000 caches).

They are both set to 'unset day of week after running'.

When I add them for a given day of the week, it is shown on the page

but the PQ never runs.

Other PQs I add later run w/o problem, but these two simply never run at all anymore.

Link to comment
Moun10Bike

Moun10Bike

Posted
Posted
Yes, on one of them a copy ran once, then also stopped. Haven't tried copying the other one yet.

 

sejtam, the PQ to which you link above is a query that is asking for all caches within 500 miles of San Francisco. That is a ginormous area in one of the densest cache areas on the planet. The issue is that the server is choking on such a huge request. Even if it worked, you would get what amounts to a random sampling of 1000 caches in that expansive area.

 

You should take a different approach in setting up your PQs. Adjust the criteria such that the results are returning just under the maximum number of caches (i.e. just under 1000 in this case). That will ensure that you are not overburdening the server with the search request and can get a workable PQ out of it.

 

It is in our plans to provide better feedback to users on the PQ pages so that it is apparent when the search criteria are out of bounds.

Link to comment
eagsc7

+eagsc7

Posted
Posted
Yes, on one of them a copy ran once, then also stopped. Haven't tried copying the other one yet.

 

sejtam, the PQ to which you link above is a query that is asking for all caches within 500 miles of San Francisco. That is a ginormous area in one of the densest cache areas on the planet. The issue is that the server is choking on such a huge request. Even if it worked, you would get what amounts to a random sampling of 1000 caches in that expansive area.

 

You should take a different approach in setting up your PQs. Adjust the criteria such that the results are returning just under the maximum number of caches (i.e. just under 1000 in this case). That will ensure that you are not overburdening the server with the search request and can get a workable PQ out of it.

 

It is in our plans to provide better feedback to users on the PQ pages so that it is apparent when the search criteria are out of bounds.

 

Correct me if I'm wrong, but it was my understanding(and experience) that if you have a From(coordinates/zip/GC#) that exceeds the number of caches in the area, that it actually finds the Closest 1000 caches and outputs that. I.E the 1000 caches you receive would be the 1000 closest to the center of San Fran(or wherever you put the coords).

 

The Steaks

Link to comment
sejtam

+sejtam

Posted
  • Author
Posted

Exactly. Note that that PQ works find on the map preview or preview page, one simply does not get a GPX file as it does not automagically.

 

Also, what I asked for is exactly what I want. Whenever I go there I want a list of caches that I may want to try.

 

I don't see how what people tell me to do has anything to do with the actual problem: it simply won't run when requested for a given day.

Link to comment
Let's Look Over Thayer

+Let's Look Over Thayer

Posted
Posted
Exactly. Note that that PQ works find on the map preview or preview page, one simply does not get a GPX file as it does not automagically.

 

Also, what I asked for is exactly what I want. Whenever I go there I want a list of caches that I may want to try.

 

I don't see how what people tell me to do has anything to do with the actual problem: it simply won't run when requested for a given day.

Might as well jump in, though it probably wont' do any good...

 

I had four PQ's set to run today. 1 ran almost instantly and I have the results. The other three have not run yet (despite the passage of time...)

 

All four PQs have run before (a week ago).

 

Unfortunately, the one I need the most for today is one of the one's that has not run.

Link to comment
The Klever Boys

+The Klever Boys

Posted
Posted

Mine haven't been running today either. I'm going to try and create a new PQ to see if that helps.

I've had the same problem today - trying to run some PQs that I created a couple of weeks ago - containing 1000 caches, which both ran ok last time.

These have not run, so obviously no email notification and no new download available.

I've created copies of them which have run fine, so got round the problem that way for the time being...

Link to comment
Moun10Bike

Moun10Bike

Posted
Posted

Correct me if I'm wrong, but it was my understanding(and experience) that if you have a From(coordinates/zip/GC#) that exceeds the number of caches in the area, that it actually finds the Closest 1000 caches and outputs that. I.E the 1000 caches you receive would be the 1000 closest to the center of San Fran(or wherever you put the coords).

 

Not correct. The query pulls data out of the database in order, meaning those caches with the lowest IDs get added first. Exceeding the limits of the PQ means you are only going to see the older caches, and those will be spread across the search area.

Link to comment
Moun10Bike

Moun10Bike

Posted
Posted
Also, what I asked for is exactly what I want. Whenever I go there I want a list of caches that I may want to try.

 

I don't see how what people tell me to do has anything to do with the actual problem: it simply won't run when requested for a given day.

 

You are misunderstanding. By running your PQ out to 500 miles from San Francisco, you are asking the system to return well over 100,000 caches. Since the maximum number of caches the PQ will return is 1000, you will get only the first 1000 listings encountered in the database - less than 1% or the total, generally the oldest in the database, and spread out across that vast area. And that's only if the PQ actually runs. Because it is trying to pull so many caches for the query, the PQ will most likely time out, especially if the system is under any load at all. When that happens, it marks the PQ for later attempt and moves on to others. Hence, it appears to never run since it is always timing out.

 

It appears that when I look at the PQs that are noted in the forums as failing to run, they are PQs that are exceeding the maximum limit of the PQ. I guarantee that if people would trim down their search results so that the numbers were more in line with the maximum allowable per PQ, we would see many fewer of these complaints.

 

As I mentioned above, we have plans to make it more clear to users when they are exceeding the limits of the PQ.

Link to comment
Corp Of Discovery

+Corp Of Discovery

Posted
Posted

Correct me if I'm wrong, but it was my understanding(and experience) that if you have a From(coordinates/zip/GC#) that exceeds the number of caches in the area, that it actually finds the Closest 1000 caches and outputs that. I.E the 1000 caches you receive would be the 1000 closest to the center of San Fran(or wherever you put the coords).

 

Not correct. The query pulls data out of the database in order, meaning those caches with the lowest IDs get added first. Exceeding the limits of the PQ means you are only going to see the older caches, and those will be spread across the search area.

 

Are you sure this is correct? I always run PQs based on a central point and I've always seen that they come thru based on distance from that point rather than the age of the caches. For instance- I run one from my home coords and ask for 1000 and the distance is set to 50 miles. There are well over 1000 caches in that area but I only get the 1000 closest...

 

If there is no 'from' point selected I can see that what you are saying could be true.

Link to comment
Corp Of Discovery

+Corp Of Discovery

Posted
Posted (edited)
Are you sure this is correct?

 

Straight from the mouth of the developer!

 

I'm sorry but there must be something being lost in translation. I just looked at the PQ I mention and the furthest one from the central point is 21.9 miles away. From what you are saying the 1000 caches in that PQ must all just happen to be also the 1000 oldest caches within 50 miles of my house- that is not correct. This cache is about 35 miles from me, does not show up in the PQ I am referring to, yet literally 100's of other caches placed more recently do. Does the system perhaps look at all the caches by date and then filter them by distance leaving the PQ with only the closest ones?

 

Edited to add: Could it be that it adds caches in lowest ID order if they are at the fringe distance of and equidistant from the center point of the PQ?

Edited by Corp Of Discovery
Link to comment
Moun10Bike

Moun10Bike

Posted
Posted
Are you sure this is correct?

 

Straight from the mouth of the developer!

 

I'm sorry but there must be something being lost in translation.

 

My apologies; you are correct. In the most basic of PQs (show me all caches within X distance of a given point), the results will be sorted by distance, and so you will only get the nearest caches in the PQ. However, once you start adding search criteria, you start encountering other methods of sorting and that's when you can get only the oldest caches (or to be precise, the smallest cache IDs), among other things. I actually had to sit down with the developer and both of us walk through the code in order to determine this; he had remembered it as always being the smallest cache IDs but obviously it is a complex system (and there has been a lot of change recently)!

 

Markwell, the way that caches along a route (CAaR) are determined is another beast altogether. As you have discovered, there are additional complexities with that. I'd have to grab yet another developer to walk through that one and then I'm sure they'd want to skin me alive for bugging them! :laughing:

 

In any case, the most important points I was making are still valid: if your query grossly exceeds the limits of what can be returned by the PQ, you are going to run the risk of losing unexpected caches (depending on the sorting done by your particular query) or - more importantly - of having your query time out and result in no PQ being generated. For example, we manually ran the SQL query for sejtam's San Francisco PQ and found that it initially returned 107,000 possible results and took 15 seconds to fully run. This was moments ago when the load on the database was low. Given that 30 seconds is the time out rule for regular PQs, it's not hard to imagine that it would run into time out issues during peak times. Again, the best approach is to try to set your search criteria such that the PQ returns a number of caches that is just below the maximum results allowed.

Link to comment
StarBrand

+StarBrand

Posted
Posted

I think some semi-intelligent feedback really is the answer here. Although I never really have to worry about it in my rural setting - I know how easy it is to overshhot the request for large areas. Better instructions and some kind of error feedback really is needed.

Link to comment
Markwell

+Markwell

Posted
Posted
you are going to run the risk of losing unexpected caches

That's a little problematic for cachers to troubleshoot. I would think that the distance sort would be the last component added to the results from any point-of-origin query. Filter down criteria of type, size, attributes, difficulty, terrain, geographical area to a list - and then the non-indexed field of "distance from point of origin" would be applied. I would think that the non-point-of-origin (geographical territories only) would then default to the incremental identity (GCID=40, etc.) and the higher numbers would then break off.

 

If just by adding other criteria this concept is broken... ew.

 

I would however love to see consistency added to these, as well as some type of sorting to the cache on a route results - even if it's just by GCID.

Link to comment
Corp Of Discovery

+Corp Of Discovery

Posted
Posted
Are you sure this is correct?

 

Straight from the mouth of the developer!

 

I'm sorry but there must be something being lost in translation.

 

My apologies; you are correct. In the most basic of PQs (show me all caches within X distance of a given point), the results will be sorted by distance, and so you will only get the nearest caches in the PQ. However, once you start adding search criteria, you start encountering other methods of sorting and that's when you can get only the oldest caches (or to be precise, the smallest cache IDs), among other things. I actually had to sit down with the developer and both of us walk through the code in order to determine this; he had remembered it as always being the smallest cache IDs but obviously it is a complex system (and there has been a lot of change recently)!

 

Markwell, the way that caches along a route (CAaR) are determined is another beast altogether. As you have discovered, there are additional complexities with that. I'd have to grab yet another developer to walk through that one and then I'm sure they'd want to skin me alive for bugging them! :laughing:

 

In any case, the most important points I was making are still valid: if your query grossly exceeds the limits of what can be returned by the PQ, you are going to run the risk of losing unexpected caches (depending on the sorting done by your particular query) or - more importantly - of having your query time out and result in no PQ being generated. For example, we manually ran the SQL query for sejtam's San Francisco PQ and found that it initially returned 107,000 possible results and took 15 seconds to fully run. This was moments ago when the load on the database was low. Given that 30 seconds is the time out rule for regular PQs, it's not hard to imagine that it would run into time out issues during peak times. Again, the best approach is to try to set your search criteria such that the PQ returns a number of caches that is just below the maximum results allowed.

 

Thanks for the clarification. B)

 

It seems that what it might come down to is to use the KISS principle when running PQs.

Link to comment
Moun10Bike

Moun10Bike

Posted
Posted
I would think that the distance sort would be the last component added to the results from any point-of-origin query

 

Not if, for example, you are splitting out caches by date placed within the search area. In that case, the results are delivered sorted by date. Overall, though, I agree with your point.

Link to comment
Westacular

+Westacular

Posted
Posted

The focus on sorting could be a bit misleading... the issue isn't so much the sorting, as it is, WHEN during the process of querying the database is the list of caches truncated down to 500 or 1000, and what their sorting is at THAT point.

 

That said:

 

I just ran a few tests, and the truncation appears to be the final stage regardless of query. So, whatever the final sort order for the query is, it's the caches at the end of that list that will fail to appear.

 

If the query involves a radius from an origin -- which applies to almost all of mine, and I expect most in general -- then the final sort order is ALWAYS the distance from that origin, and the results you get will nicely cut off at a certain distance (ie giving the 500 or 1000 closest to the origin), regardless of what the actual radius limit is set to.

 

I just tested this by previewing a couple of "placed between these dates" queries with the radius set to a few hundred kilometers from home coords. On the first, all of the 500 results were within 158km, and sorted by distance; on the second, where I expanded the date range by a several months, all of the 500 results were within 82km.

 

The only time this isn't the case is when a "from origin" filter isn't set, and "within [country/state/province]" is used instead (since at least one of the two options must be used). And yes, in that case, caches are sorted and truncated based on the cache ID (which roughly correlates with the cache's age, though there are exceptions.)

 

I haven't tested this with more complex combinations, but I expect that any other aspects (like cache size/type/attributes) would be part of the initial database query and wouldn't affect this behaviour.

 

So:

1) If you're searching for caches, and you include a radius and an origin, you'll always get the closest ones to that origin, even if the radius is set too large and you run into the number of caches limit.

 

2) If you don't set an origin (which necessarily means you're searching within a preset region), they will be sorted according to cache ID, and you'll get the first 500 or 1000 or whatever based on that.

 

3) Just because you CAN enter a huge radius doesn't me you should -- the servers will have to do all of the additional processing a wider radius requires, but you won't get any more actual caches from that. So if you want to overestimate it, don't do it by a huge amount -- it will just slow things down and possibly break them.

 

This behaviour makes sense, and I think is generally the right way to do this... although some more transparency or feedback to users about their queries would certainly be welcome.

Link to comment
Guest
This topic is now closed to further replies.
Followers 4
Go to topic listing
×
×
  • Create New...