Pocket Queries and large cache populations.

[+ClayJar]

Okay, so I was chatting away while slinging code on Watcher, and something came up in the discussion that seemed like a possibly useful idea. Since none of us could markwell the topic, I figured I'd post it here.

 

The Problem:

 

The most efficient way to get a full state's worth of caches is to do a pocket query by state; however, when the state grows to more than 500 caches, you can no longer use one query to fetch all the caches. It *is* possible to segment by type, but once the state has grown past 500 traditional caches, even that becomes impossible. Using multiple overlapping radial queries is possible, but it is far from efficient in both its use of pocket queries and also its use of the database (since the queries *must* overlap and therefore *will* include duplicates, plus the distances must be calculated for each cache in each query).

 

One Possible Solution:

 

The cachers using the pocket queries are going to find a way to get all the caches they're interested in. At present, that means they are putting extra load on both the database and themselves. This situation could be solved by creating the facility to combine pocket queries. If you want to return up to 1000 caches, you would spend not *one* pocket query point but *two* pocket query points on it, and it would return up to 1000 caches.

 

The problem with allowing cachers to spend more than one pocket query point on a query is simple. Resources. Since I am not in any way privy to how the system is setup, I cannot say whether returning 1000 caches for one pocket query would tax the system more than returning three pocket queries totalling somewhere around 1200-1500 caches in order to get the same full set. Elias, Jeremy, et al, I leave that question for you to reply to. (If fewer but larger queries would indeed tax the database more, please do tell us so that we can use it to markwell this should it ever come up again. )

 

An Alternate Solution

 

Choosing overlapping circles is both hard and inefficient. "Rectangles", on the other hand, are quite easy to create (just pick two points on a map) and efficient to filter (A < coord < . [Note: They're not perfect rectangles, due to the fact earth is round, but they're good enough.]

 

If you could add an "and is inside this rectangle" to the pocket queries, it would be the next best thing to combining queries. It would be simple enough to use, and it would eliminate the overlap (and distance calculation), which should mean that the database load is actually *decreased*.

 

Anyway, just a pair of ideas about a subject that does not matter yet to me (but that does matter to many cachers). If you would care to comment, Elias, Jeremy, or whomever, I and my fellow GPX coders and users would be most humbly grateful. (And feel free to post that you want to postpone commenting; it'll make it easier for us to control ourselves... somewhat. )

 

(Incidentally, they told me to ask if half-stars might be added to the pocket queries, since many of us would like, for example, terrain >= 4.5, or even a split at PQ1: <=3.5 and PQ2: >=4)

[+parkrrrr]

There's some potential for overlap with "rectangle" queries as well, if a cache falls exactly on a boundary line, but it's far less than with circles.

 

Anyway, I just wanted to register my heartfelt support for this idea.

 

[This message was edited by Warm Fuzzies - Fuzzy on December 23, 2002 at 10:49 AM.]

[+sbell111]

A 'rectangle' search would also be handy when planning a trip.

 

Whenever I feel blue, I start breathing again.

[+ClayJar]

quote:

---

Originally posted by sbell111:

A 'rectangle' search would also be handy when planning a trip.

---

Now that you mention it, Team Ninefruits was just asking me about whether he could use the pocket queries to search for caches along his route. If he could have drawn several long rectangles, that would have done nicely, indeed.

[+Web-ling]

quote:

---

Originally posted by ClayJar:

quote:

---

Originally posted by sbell111:

A 'rectangle' search would also be handy when planning a trip.

---

Now that you mention it, Team Ninefruits was just asking me about whether he could use the pocket queries to search for caches along his route. If he could have drawn several long rectangles, that would have done nicely, indeed.

---

What would really help in planning a trip would be the ability to draw rectangles that are on a diagonal, where the sides are not necessarily latitude or longitude lines. Yea, I realize that would be harder to program, but it shouldn't be that much harder.

 

[+byonke]

quote:

---

_The Problem:_

 

The most efficient way to get a full state's worth of caches is to do a pocket query by state; however, when the state grows to more than 500 caches, you can no longer use one query to fetch all the caches. It *is* possible to segment by type, but once the state has grown past 500 traditional caches, even that becomes impossible. Using multiple overlapping radial queries is possible, but it is far from efficient in both its use of pocket queries and also its use of the database (since the queries *must* overlap and therefore *will* include duplicates, plus the distances must be calculated for each cache in each query).

---

 

What works for me is to specify only the state. I then get the first 500 geocaches in chronological order. For the next 500 I use the last date of the first 500 for the starting date for the next 500. If it goes above 1000, just continue on. Hope this all makes sense.

 

Brian

[+Boundertom]

As the number of caches grows, this is definitely a problem that needs to be addressed. Clay Jars idea of "spending" more than one query point an a search would work for me. I have previously suggested to Jeremy that an additional filter for North/South/East/West of a particular longitude/lattitude would be very helpful. This would be a lot easier to set up and more efficient than the circular searches. I know nothing about programming, but it doesn't seem to me that this would be too hard to accomplish. I would actually like to see a combination of both ideas.

[+Jamie Z]

Sort of off-topic, but an idea that may be useful for someone looking for rectangular (or more specifically square) search area.

 

Couldn't you simply do a circular search for caches and then filter out those with coordinates outside your desired rectangle?

 

Still inefficient, but it's a rectangle.

 

Jamie

[+Lil Devil]

quote:

---

Originally posted by Jamie Z:

Couldn't you simply do a circular search for caches and then filter out those with coordinates outside your desired rectangle?

---

Thats kinda what I do now. I open the file in ExpertGPS, open a map view, then use the selection tool to select all the caches that are not in the area I care about.

 

Yeah it works, but the computer should do that for me. Then again, figuring the corners of the rectangle to specify in the PQ would also take some work.

 

Lil Devil