Drastically reduce server load

[+Team Hildy]

tl;dr summary - Not only are PQs hard for newbies to learn, almost all of their features are redundant since users can easily use their own database programs like GSAK, smartphone apps, or even their GPSr to do all that sorting for them. It would be far more efficient for Groundspeak to subdivide the world into regions, run one or two large queries for all active caches in each region every day, and make the resulting .zip files available for download to premium members. This would also make it easier for users to keep their databases up to date - a win / win for everyone.

 

 

--------------------------------------------------------------------------------------------------------

PQs are complicated beasts that use up a lot of server resources, which necessitates the 5 per day limit on running them. However, most of what a PQ does is totally redundant since the members who use them most are downloading them into GSAK or a similar database that can take care of all the complicated sorting that cachers like on an individual basis. Usually you can even load 10,000 or more caches into your GPSr and use it to sort for you. In fact, what most cachers want from PQs most of the time is just all of the active caches in a giant area, but they have to run multiple individualized PQs (and use lots of server resources) to do so.

 

EASY SOLUTION: Just have some lackey at Groundspeak set up regional queries for large areas, set them all up to run once or twice a day, and make the results available for download. If there are 1 million caches in North America (just guessing), and you subdivide the continent into 200 regions with somewhere around 5,000 caches per region, that covers all 1 million caches. So, the server runs 200 extra large queries per day, and then stores the results in 200 different files. Users could just click on the region or regions in which they're interested and download a copy of that day's query for their region. Surely multiple downloads of zipped files will take far less resources than generating hundreds of thousands (millions?) of PQs and uploading them to users every day.

 

I can forsee a map with the regions outlined, and cachers just click on the region they want in order to download the pre-generated file of all the caches in that region. A region may consist of half a state in sparsely populated areas like Western Montana, or just 1/8 of a metro area in a place like Chicago, depending upon the density of caches in the area. Instead of learning to set up 10 or 20 PQs all covering the same 200 mile radius and delineated by cache placement date to get around the 1,000 cache limit, cachers could just click on the part of the map they want and maybe set it up to e-mail them a link to the file every week, or twice a week, or however often they feel the need to update their database.

 

Consider a mega event where thousands of cachers all run essentially the same PQs for the area over the course of a weekend. The server has to do the same work over and over for all those cachers, and the cachers have to do all the work of setting up the PQs. Wouldn't it be better to have a lackey run a query of 5,000 or even 10,000 caches just one or two times a day, and then make the results of that query available for easy download?

Edited June 15, 2012 by Team Hildy

[jholly]

I never knew the 5 a day limit on PQ's is because they were complicated beasts and use lots of server resources. Learn something new every day. I always thought it was because Groundspeak didn't want the world downloading their database and uploading it to that squirrel site. Silly me. As for PQ's being complicated, well I just read Markwell's excellent treatise on PQ's and never looked back. Good luck with your proposal.

[+Delta68]

Looks like a solution to a problem which doesn't exist!

 

PQs work great at the moment. I can set one up and within minutes it will be available for download. If the process struggles Groundspeak can probably just chuck another server at it. The PQ server is separate from the web servers.

 

Apart from that, why would anyone want to download a list of 5-10,000 caches which would include all their finds and hides as well?

 

 

Mark

[+Team Hildy]

Looks like a solution to a problem which doesn't exist!

 

PQs work great at the moment. I can set one up and within minutes it will be available for download. If the process struggles Groundspeak can probably just chuck another server at it. The PQ server is separate from the web servers.

 

Apart from that, why would anyone want to download a list of 5-10,000 caches which would include all their finds and hides as well?

 

 

Mark

It doesn't cost anything to download the list that includes your finds and hides, and you can use GSAK or whatever DB software you use to filter the out when you load up your GPSr. Duplicating the process on Groundspeak's servers is redundant and inefficient.

 

Sure, you and I can set up our PQs easily enough, but I can tell you that it was a long learning process for me. Simply clicking a couple of links to get everything you used to get with all those custom PQs would make our expertise obsolete, but it would also make the game more accessible to new members.

[jholly]

Looks like a solution to a problem which doesn't exist!

 

PQs work great at the moment. I can set one up and within minutes it will be available for download. If the process struggles Groundspeak can probably just chuck another server at it. The PQ server is separate from the web servers.

 

Apart from that, why would anyone want to download a list of 5-10,000 caches which would include all their finds and hides as well?

 

 

Mark

It doesn't cost anything to download the list that includes your finds and hides, and you can use GSAK or whatever DB software you use to filter the out when you load up your GPSr. Duplicating the process on Groundspeak's servers is redundant and inefficient.

 

Sure, you and I can set up our PQs easily enough, but I can tell you that it was a long learning process for me. Simply clicking a couple of links to get everything you used to get with all those custom PQs would make our expertise obsolete, but it would also make the game more accessible to new members.

Given the growth I would say it is pretty accessible. And yes, it does cost to download your finds and hides, at least it costs bandwidth. So your trading server cycles generating PQ's for server cycles providing downloads. This may come as a shock, but an awful lot of people that cache do not use GSAK or other DB software, they just stuff the PQ directly on to their gizmo. Your approach would eat up a lot of the gizmo storage. And a lot of members are not premium members so PQ's are not available to them. They download from cache pages or search results. And given the flexibility of the PQ's, I might well have a PQ that is tailored to my needs that is not available in the one-size-fits-all lackey generated PQ. So I would still be generating PQ's. Not to mention I would still be generating cache along the route PQ's.

[+Chokecherry]

If I had to rely on GSAK I would never have have run the 2 PQ's I have run (caches along a route). I don't do PQ's because I had pick caches from maps for one and two after looking at GSAK I realize my technological limitations.

[+frinklabs]

Wouldn't the mass-download have to include every archived cache, to allow you to reproduce the equivalent of a My Finds query?

 

I think what makes the My Finds query unique over a regular one is that it includes archived caches you have found.

[+JesandTodd]

I don't think PQs were difficult to learn.

 

And also, I don't want to get results from somebody else's query. I want my queries that are set to my specs. No thank you to your idea. I'll keep my PQs the way that they are

[+tozainamboku]

The idea of canned queries has been brought up a number of times. The problem is that dividing the world into regions that have roughly the same number of caches is a lot like reapportioning congressional districts after the census. Only the count of cache is continuous - not every 10 years - and the density of caches varies greatly, leaving some areas where 5000 cachess is a relatively small area and others where 5000 covers a huge area. In addition, you're likely to get people who live near the border of a canned area. Perhaps they don't want all of the area they live in, but they may want all or part of the neighboring area. And if the regions get adjusted as the numbers of caches change, someone near the border may have trouble determining which region they are in from one day to the next.

 

On top of this many cachers have developed customized PQs to get just the caches they are interested in. And cachers who have found most of the nearby caches may request caches they haven't found to extend the range of caches they can get.

 

Certainly if there were canned queries, people should still be allowed to get custom queries if they want. My guess is that canned queries would not result in any reduction of server load because they would account for a small portion of what was run.

 

If you force cachers to use the canned queries and do the filtering on their own computers, you'll likely get a lot of complaints that this far more complicated than figuring out PQs. It also encourages people to maintain a large offline database that is hard to keep up to date. This something that Grounspeak does not want to happen. Limiting the numbers of caches you get in pocket queries has the effect of reducing the number of caches people will retain in GSAK or other databases, and thereby get people to rely on the Groundspeak data more.

 

It may be true that there is a learning curve in setting up PQs. The biggest issue seems to be people requesting queries with conflicting criteria, so no caches are returned. It would probably be a better use of resources to improve the PQ interface to provide more support to build queries and check that they return the expected caches before submitting them.

[+palmetto]

you can use GSAK or whatever DB software you use

 

Many many many cachers use no DB software, they load PQs directly to their devices.

 

I have no idea if there's merit to your suggestion re reducing server load, but I am quite certain that it would be a disaster for Groundspeak to stop providing the current level of individual customization of PQs.