Jump to content
Sign in to follow this  
Followers 12
Jeremy

Apology: Weekend Pocket Query Issues

Recommended Posts

The idea is for those who wake up first to get the first PQs, so I believe by Latitude is correct, and we may also parse into longitude chunks. .

 

 

I think we all understand what you are trying to achieve but proccessing by latitude will produce PQ's for everyone at the south Pole first, then the equator, then the North pole?

 

 

I don't want to teach my granny to suck eggs but..... surely you need to process by Timezone i.e. Longitude first i.e. Aisia, Europe then the USA

Edited by Big Wolf

Share this post


Link to post
What would you like the indication to look like? "This is one of your Sunday PQs" in the e-mail?

How about "Pocket Query : [PQ name] scheduled for [date]" ?

 

So if I ask for a new PQ to run Saturday night it will count as Saturday, even though I may already have received a Sunday PQ earlier? I guess the only thing that would make this clear is if the date run date indicated the day charged...or maybe a new column indicate "charged to".

 

(I don't really care about what the e-mail says...it's the online list of PQs where it will make a difference to me.)

 

If you don't indicate the day charged, there will really be no way to be sure what day was charged except memory.

 

And if I ask for too many PQs to run on Sunday, will I be told that even if some ran early on Saturday?

Share this post


Link to post

Thanks to all at Groundspeak for their excellent explanations (Drew) and acknowledgement/aplologies (Jeremy) of the PQ issues...and everyone else who is working behind the scenes.

 

It's never a good thing when something breaks unexpectedly, but sometimes it leads to new ways of looking at problems and some great innovations. It should be interesting to see what you guys come up with.

 

Thanks again--I received the "my finds" PQ first thing this am and everything is working great--keep up the good work.

Share this post


Link to post

So, any guesses on how long it will be before a thread is opened whining about stating 5000 caches a day is less than someone can find needs in a day?

How long until we have a post whining about people who ask for new things?

 

Oh. It already happened. Wow, that was fast!

ROTFLOL!!

 

Oii... I REALLY Like that I will have PQs with 1000 results, but that only decreases the Number of PQs for my "Specified Area". It'll make trips all the better...

 

WOOHOO!!! I Guess I got MY Birthday Present EARLY!! Course, its only a few years Late... YEAH Anyways!!

 

The Steaks

Share this post


Link to post

For you tech types, we'll be using a distributed cache.

Woohoo! A new Cache type! :)

 

On a serious note... you guys rock. Really. What a great company, both leadership and staff.

 

Thanks.

Share this post


Link to post

Thanks for the update. My caching trip was thankfully saved by a data saving friend who had coverage for the area :).

 

Can't wait for 1,000 Pqs!! Awesome addition guys!

Share this post


Link to post

For you tech types, we'll be using a distributed cache.

Woohoo! A new Cache type! :)

 

On a serious note... you guys rock. Really. What a great company, both leadership and staff.

 

Thanks.

I had the same thought.

 

I also had the same thought all weekend that it would be nice if Jeremy posted an apology and perhaps even read the riot act to the developers to reduce the chance of it happening in the future. It seems he did both.

 

drew - if you are going to process the PQ by timezone then you do mean longitude. Remember the day starts at the International Dateline 180° E longitude (+180) and counts down to 0° longitude (Greenwich meridian) then keeps counting down to 180° W longitude (-180). It may be better to use the timezone in the users profile. Schedule the PQ to run at midnight in the profile and count the number of PQs per day based on the timezone as well (But don't have changes to the timezone take effect for 24 hours to keep people from changing timezones to get more PQs in a day). It may cause a little problem if the queue fills up with caches from Eastern TZ causing the other US/Canada TZ to begin executing a little later. (My guess is that the timezone defaults to Pacific TZ if not set so those PQs would be scheduled as they now are.) I would guess all the European and Asia PQs will have been processed before the US starts and the Australian geocachers would love not having to worry about when to run their PQs yesterday in order to get them today.

Share this post


Link to post
It may be better to use the timezone in the users profile. Schedule the PQ to run at midnight in the profile and count the number of PQs per day based on the timezone as well (But don't have changes to the timezone take effect for 24 hours to keep people from changing timezones to get more PQs in a day).
That would seem to be easier to understand than just starting them before midnight and leaving us to try to figure out what day the PQ was charged against.

 

It also would spread out the workload, it seems, since everybody's PQs would queue up at midnight local time.

 

No idea how hard that would be to implement.

Share this post


Link to post

Jeremy, thanks for the good news! PQ2 with 1k of caches will make things easier, especially in crowded cities and on bigger round trips (like my own though europe in summer 2010)

Share this post


Link to post

Over the weekend there were a lot of issues with Pocket Queries being generated. For that I sincerely apologize.

 

In an attempt to speed up the Pocket Query Generator, the application(s) that processes tens of thousands of Pocket Queries a day, there was a cascading number of issues that resulted in duplicate Pocket Queries going out, Pocket Queries without logs and Pocket Queries not being sent out for a certain amount of time. Also in the process, Pocket Queries were brought down to a trickle, resulting in Pocket Queries not going out at all.

 

We're taking this issue very seriously at Groundspeak, and we'll be focusing on increasing the capabilities over this next weekend to ensure this never, ever happens again. By increasing this capacity we'll also be prepared to offer Pocket Queries with 1,000 results instead of 500. This will happen by the anniversary of geocaching (May 2, 2010). As those veterans to geocaching know, I'm not usually one to offer dates, but this one is solid.

 

Again, I offer my apology for this weekend's issues.

 

I only had a minor issue (no logs) this past weekend and that's only one of a few times I've ever had problems. Thanks for the info, apology and great news about the increase. You and the rest of the crew do a great job- keep it up.

Share this post


Link to post

Sent request for new my finds PQ and email arrived very soon after from Nicole saying it had been reset so just wanted to say a big thank you to

all you guys out there who put in all the hard work keeping us lot happy.

It's all very much appreciated :)

Share this post


Link to post

drew - if you are going to process the PQ by timezone then you do mean longitude. Remember the day starts at the International Dateline 180° E longitude (+180) and counts down to 0° longitude (Greenwich meridian) then keeps counting down to 180° W longitude (-180). It may be better to use the timezone in the users profile. Schedule the PQ to run at midnight in the profile and count the number of PQs per day based on the timezone as well (But don't have changes to the timezone take effect for 24 hours to keep people from changing timezones to get more PQs in a day). It may cause a little problem if the queue fills up with caches from Eastern TZ causing the other US/Canada TZ to begin executing a little later. (My guess is that the timezone defaults to Pacific TZ if not set so those PQs would be scheduled as they now are.) I would guess all the European and Asia PQs will have been processed before the US starts and the Australian geocachers would love not having to worry about when to run their PQs yesterday in order to get them today.

 

Great idea! I add a second to the suggestion.

Share this post


Link to post

Again, I offer my apology for this weekend's issues.

Thanks.

 

But what about the 'Saved GPX files'? This feature was introduced november 2009 and the iPhone app is still not able to download them. This bug is known for more than three months but there is not even a statement when a bugfixed app can be expected. It shouldn't be too difficult to fix that??

Share this post


Link to post

Jeremy - Thanks for the info and for continuing to mature your company and staff...

 

It is good to read that Groundspeak may start actually adhering to Change Controls and informing the customer base when the Change Controls are scheduled for.

 

It is good and bad from my perspective to hear PQs are moving to an upper bound of 1000 per PQ. Honestly I percieve this as a Band-Aid that will reduce some of the users perceived user experience burdens, but honestly just stages the community for another 10 years of the same basic design and no more fidelity. As I posted in another thread, I would like to see and think it would be advantageous for Groundspeak to instead of having a data bounding per tool to have a more general Bounding for all data delivery features and giving the end user the ability to invoke as many or few PQs CAARPQs etc until that Data Feature Bound is met.

Share this post


Link to post

* Another plan is to start the PQs to processing at 8pm PST. If we have time this week, we plan to prioritize processing by Latitude so those of you across the Pond and further will have your PQs during the day, and to allow our side of the Pond to still log their caches for the day.

We want to make sure the generator is stable this weekend, so processing by Latitude may get pushed, but it is the plan. Any thoughts on this?

 

Sounds great ... except I hope you mean you're processing by longitude not latitude :)

 

A>

 

The idea is for those who wake up first to get the first PQs, so I believe by Latitude is correct, and we may also parse into longitude chunks. We plan to get the run times down to about 6 hours, which will be an interesting challenge when we move to 1000 caches per PQ.

 

Oh, the sun now rises in the south? :huh: No, LONGitude. They are the ones running north to south, or south to north depending on your point of view. Latitudes run around the sphere east to west. The equator is latitude 0 degrees.

 

I stand embarrassedly corrected. Luckily I am not responsible for any geospatial code yet. Are there any links to common mistakes/misconceptions people make in this area, like say, switching lat & long? Another interesting link would be to a test developers should take before writing geospatial code. Any takers?

Share this post


Link to post

Unacceptable! I expect a FULL refund!

 

LOL! :) I'm totally busting chops. Please, no harm no foul. We survived without the "My Finds" query over the weekend. Worst case we would have had to go with paper caching this weekend. And I can wait for the next allowable my find query. No need to load your email boxes with a billion requests when I can wait for the automated response.

 

The site rocks, I fully support it and will continue to do so.

Share this post


Link to post

I stand embarrassedly corrected. Luckily I am not responsible for any geospatial code yet. Are there any links to common mistakes/misconceptions people make in this area, like say, switching lat & long?

FWIW, I used to bozo this frequently in my early days of writing such things. I finally remembered that LATItude are like the rungs of a LADDer - they're parallels.

 

I suppose you can screw this saying up and think they're like the rails of a ladder instead of the steps/rungs, but for some reason, I don't. Still, it's a corny mnemonic that you might find useful.

Share this post


Link to post

When I teach lat/long to scouts, I say that if you think of a belt going around a fat belly, those are "lats" - the LONGs are the "tall" lines going top to bottom, or bottom to top if your south of the equator. ;)

 

It really can be a hard concept tho - as your talking about going around the globe, which is kind of like the latitude lines, but you're actually changing the longitude. Confused yet?

Share this post


Link to post

a thank you from me, i thought my email server was being awkward but obviously not.

 

Look forward to the 1000 pq's dont know how more or less work it will create for the servers but it will mean i will be running less pq's if they become this big.

 

many thanks to you all at Groundspeak.

Share this post


Link to post
When I teach lat/long to scouts, I say that if you think of a belt going around a fat belly, those are "lats" - the LONGs are the "tall" lines going top to bottom, or bottom to top if your south of the equator. ;)

A math teacher from way back taught me an easy way to remember : longitudes are longer than latitudes. Once you visualize the lines it's easy to figure out which is which.

Share this post


Link to post
When I teach lat/long to scouts, I say that if you think of a belt going around a fat belly, those are "lats" - the LONGs are the "tall" lines going top to bottom, or bottom to top if your south of the equator. ;)

A math teacher from way back taught me an easy way to remember : longitudes are longer than latitudes. Once you visualize the lines it's easy to figure out which is which.

 

That is mostly true, but for the special case latitude there is no difference.

Share this post


Link to post
When I teach lat/long to scouts, I say that if you think of a belt going around a fat belly, those are "lats" - the LONGs are the "tall" lines going top to bottom, or bottom to top if your south of the equator. ;)

A math teacher from way back taught me an easy way to remember : longitudes are longer than latitudes. Once you visualize the lines it's easy to figure out which is which.

 

That is mostly true, but for the special case latitude there is no difference.

 

Longitude reminds me of Longcat. That's how I remember.

Share this post


Link to post
When I teach lat/long to scouts, I say that if you think of a belt going around a fat belly, those are "lats" - the LONGs are the "tall" lines going top to bottom, or bottom to top if your south of the equator. ;)

A math teacher from way back taught me an easy way to remember : longitudes are longer than latitudes. Once you visualize the lines it's easy to figure out which is which.

True except for the equator (and latitudes very near the equator). These are longer that the longitudes (the earth is fatter around the equator than around the poles). And even that is confusing because you are talking about longitudes as going all the way around the earth. Some people just look at at the longitude going halfway around the earth (from pole to pole). So all latitudes between 30 N and 30 S are longer than these longitudes.

 

With regard to other misconceptions, one that I was trying to emphasize in my previous post was the sign of west longitude being negative (and east being positive). And that the day starts at +180 and moves down to -180. Time is more confusing even than latitude and longitude as witnessed by the continuing issues with field notes.

Share this post


Link to post

Again, I offer my apology for this weekend's issues.

Thanks.

 

But what about the 'Saved GPX files'? This feature was introduced november 2009 and the iPhone app is still not able to download them. This bug is known for more than three months but there is not even a statement when a bugfixed app can be expected. It shouldn't be too difficult to fix that??

 

This has been fixed today. Sorry for the trouble Olfi.

Share this post


Link to post
When I teach lat/long to scouts, I say that if you think of a belt going around a fat belly, those are "lats" - the LONGs are the "tall" lines going top to bottom, or bottom to top if your south of the equator. ;)

A math teacher from way back taught me an easy way to remember : longitudes are longer than latitudes. Once you visualize the lines it's easy to figure out which is which.

That is mostly true, but for the special case latitude equator there is no difference.

Fixed.

 

Only the equator latitude line is as long as all the longitude lines. All the other latitude lines are progressively shorter as you move towards the poles.

Share this post


Link to post

Again, I offer my apology for this weekend's issues.

Thanks.

 

But what about the 'Saved GPX files'? This feature was introduced november 2009 and the iPhone app is still not able to download them. This bug is known for more than three months but there is not even a statement when a bugfixed app can be expected. It shouldn't be too difficult to fix that??

 

This has been fixed today. Sorry for the trouble Olfi.

 

I just tried it and it works beautifully......Thanks guys....

Share this post


Link to post

drew - if you are going to process the PQ by timezone then you do mean longitude. Remember the day starts at the International Dateline 180° E longitude (+180) and counts down to 0° longitude (Greenwich meridian) then keeps counting down to 180° W longitude (-180). It may be better to use the timezone in the users profile. Schedule the PQ to run at midnight in the profile and count the number of PQs per day based on the timezone as well (But don't have changes to the timezone take effect for 24 hours to keep people from changing timezones to get more PQs in a day). It may cause a little problem if the queue fills up with caches from Eastern TZ causing the other US/Canada TZ to begin executing a little later. (My guess is that the timezone defaults to Pacific TZ if not set so those PQs would be scheduled as they now are.) I would guess all the European and Asia PQs will have been processed before the US starts and the Australian geocachers would love not having to worry about when to run their PQs yesterday in order to get them today.

 

Great idea! I add a second to the suggestion.

 

This is a great idea, and we have discussed it. There are a few reasons right now why we're discussing processing by longitude (pat's self on back)

1) most users haven't supplied a time zone

2) to use a distributed cache effectively, we should be asking for the same caches around the same time. i.e. If we process all PQs in Missouri at the same time, we're more likely to ask for the same caches for those PQs.

3) If you live in Seattle, but you're visiting Berlin and have created PQs in Berlin, wouldn't you prefer to receive those PQs around the same time, and (when we start processing PQs at midnight/tz) wouldn't you prefer to receive those based on the tz you're in instead of the timezone you're from?

Share this post


Link to post

This is a great idea, and we have discussed it. There are a few reasons right now why we're discussing processing by longitude (pat's self on back)

1) most users haven't supplied a time zone

2) to use a distributed cache effectively, we should be asking for the same caches around the same time. i.e. If we process all PQs in Missouri at the same time, we're more likely to ask for the same caches for those PQs.

3) If you live in Seattle, but you're visiting Berlin and have created PQs in Berlin, wouldn't you prefer to receive those PQs around the same time, and (when we start processing PQs at midnight/tz) wouldn't you prefer to receive those based on the tz you're in instead of the timezone you're from?

1) I would guess that if users haven't supplied a timezone, you would default to Pacific time and the PQs would be scheduled as they are now. It may mean you won't get the full advantange of the distributed cache but it would have the least impact on users wondering when their PQ is going to run. It's a trade-off that the Groundspeak team needs to make.

 

2) Most people in a given timezone are asking for caches that are nearby so you'd probably see a performance advantage in most cases by starting PQs base on the user's TZ. Certainly more so than just dividing the PQ into E and W hemisphere.

 

I can understand what you are doing now as first step, since it is easy to implement. Once you see that starting PQs off at different time is doable, then you might refine how these times are done. It seems reasonable to me that a first cut would be to look at the center coordinates of the PQ and split them by hemisphere. Run the non-negative longitudes at 8PM and the negative ones at midnight. I'm not sure how PQs that specify a geographic areas, bookmark lists, or CAAR will be scheduled. Don't forget about them.

 

3) I would guess that PQs for people visiting a different timezone would be a small portion of the PQs run. (I could be wrong). The extra advantage of running the Seattle cacher's PQ when the Berlin ones are run is probably very tiny overall. If the cacher is still in Seattle and is preparing in advance for the trip, they may very well prefer that it run after midnight Seattle time. If they are already travelling, I suppose they could change there TZ in the profile temporarily. Again its really just a trade-off between a small performance advantage and having a user confused about when the PQ will run and whether it counts against todays PQs or tommorrows.

 

I would say that the initial plan sounds good to get this working and that could be followed up with some changes to spread out the PQ throughout the day and make it easier for cachers who don't live in the Pacific TZ to know when their PQs will run.

 

The trip to Berlin got me thinking into the problem with field notes as well. The cacher may be logging caches in Berlin, maybe even uploading fields notes from there, and then returning to Seattle to log the caches. He probably would want to see the dates reflect the day in Berlin when the cache was found and not the Seattle dates. Something for whoever is implementing that fix to think about.

Share this post


Link to post

I think the biggest failure for us was our failure to communicate the issues over the weekend. Whenever we touch our production machines in the future, however trivial, we'll make sure to communicate this to you as well. In addition we'll continuously monitor the forums to make sure there are no issues coming up from our changes.

 

Most definitely keep your users informed.

I worked for a local ISP and when we had issues, we (as phone support) were never told that issues were occurring. This would make us change a users settings trying to figure out what was going. We could spend an hour trying to figure the problem out for the user and make all kinds of unnecessary changes to a users system and many hours of frustration.

 

When I knew there were issues, I would tell a customer that we were having difficulties and to try again in an hour. Most customers were ok with that and the frustration levels stayed sane! Also, the need for making changes on their end didn't happen and within that time frame everything was normally A-ok and working fine again.

 

So keep your customers informed, take everything they say with a grain of salt but at the same time realize that just because you have heard it 50 times, doesn't mean that you aren't going to hear it 500 more. Work with everyone and keep them informed and your feedback will be much more positive in the long run!

Share this post


Link to post

Apology accepted-)

 

As I said before, I can deal with something being broken/down if there is acknowledgment that someone is aware of it. Seems this problem has been solved so all is well.

 

1,000 cache PQ? Wow, I rarely run a 500 cache PQ so I guess I just need to expand my searches more ;)

 

Thank you for all the information from everyone at Groudspeak!!

 

David

Share this post


Link to post

So very good to hear from 'THE MAN' hisself on this issue.

I wasn't really affected, but it was curious to see six copies of the same PQ in my inbox when I returned from a camping trip on Monday.

Considering the exponential growth of Geocaching (approximately 500 new caches in one month ;) in my area) I can well imagine how doubling the available size of PQs will actually reduce the server load in the long run.

Share this post


Link to post

This is a great idea, and we have discussed it. There are a few reasons right now why we're discussing processing by longitude (pat's self on back)

1) most users haven't supplied a time zone

2) to use a distributed cache effectively, we should be asking for the same caches around the same time. i.e. If we process all PQs in Missouri at the same time, we're more likely to ask for the same caches for those PQs.

3) If you live in Seattle, but you're visiting Berlin and have created PQs in Berlin, wouldn't you prefer to receive those PQs around the same time, and (when we start processing PQs at midnight/tz) wouldn't you prefer to receive those based on the tz you're in instead of the timezone you're from?

1) I would guess that if users haven't supplied a timezone, you would default to Pacific time and the PQs would be scheduled as they are now. It may mean you won't get the full advantange of the distributed cache but it would have the least impact on users wondering when their PQ is going to run. It's a trade-off that the Groundspeak team needs to make.

 

2) Most people in a given timezone are asking for caches that are nearby so you'd probably see a performance advantage in most cases by starting PQs base on the user's TZ. Certainly more so than just dividing the PQ into E and W hemisphere.

 

I can understand what you are doing now as first step, since it is easy to implement. Once you see that starting PQs off at different time is doable, then you might refine how these times are done. It seems reasonable to me that a first cut would be to look at the center coordinates of the PQ and split them by hemisphere. Run the non-negative longitudes at 8PM and the negative ones at midnight. I'm not sure how PQs that specify a geographic areas, bookmark lists, or CAAR will be scheduled. Don't forget about them.

 

3) I would guess that PQs for people visiting a different timezone would be a small portion of the PQs run. (I could be wrong). The extra advantage of running the Seattle cacher's PQ when the Berlin ones are run is probably very tiny overall. If the cacher is still in Seattle and is preparing in advance for the trip, they may very well prefer that it run after midnight Seattle time. If they are already travelling, I suppose they could change there TZ in the profile temporarily. Again its really just a trade-off between a small performance advantage and having a user confused about when the PQ will run and whether it counts against todays PQs or tommorrows.

 

I would say that the initial plan sounds good to get this working and that could be followed up with some changes to spread out the PQ throughout the day and make it easier for cachers who don't live in the Pacific TZ to know when their PQs will run.

 

The trip to Berlin got me thinking into the problem with field notes as well. The cacher may be logging caches in Berlin, maybe even uploading fields notes from there, and then returning to Seattle to log the caches. He probably would want to see the dates reflect the day in Berlin when the cache was found and not the Seattle dates. Something for whoever is implementing that fix to think about.

 

All great points. Paul and I were talking about this issue of logging the caches from a different TZ a few weeks ago. I'm not sure if we have a story for that or not. We have a huge list of stories, so it's probable we do.

 

On the PQs, this discussion is touching on two different topics; delivering PQs in the users timezone, and processing PQs the fastest way possible. Right now, our goal is to make the generation of PQs as fast as possible. If we don't achieve this, moving to 1000 caches/PQ will not be possible. When you have as much data as we have, offer the range/flexibility of search options that we've offered, and have expectations of real-time data, it becomes difficult to optimize for all scenarios. One of the current constraints is the database. Some of the PQs put a lot of strain on it, so we try to run these process intensive searches during non-peak hours to make sure the site is as responsive as possible. There are solutions to this, but they all take time to implement and many of them will require extensive modification of our existing code base, which is not trivial and just not possible if we plan to hit our May deadline. As a geek, I'd love to change it all to the latest and greatest systems. Give me a moment to dream........ And I'm back. So, our safest option for now is utilize as much of the non-peak database time as we can. To be sure that our PQs include all the info from the previous day, we are splitting the PQs up into East & West hemispheres. The side-effect (not the goal) for our East Hemi brothers & sisters <flashing East Hemi Geocache gang sign> is that they'll get there PQs a few hours sooner. Actually, the real statement is that all PQs in the E.H. will be delivered a few hours earlier, and we assume that effects our East Hemi brothers & sisters <just got busted for flashing gang signs, but still means it>.

 

On a scale of 1-10, where "Nice To Have" is 1 and "Must Have" is 10, how important would it be to deliver PQs based on TZ? It would require some significant work, which would force us to postpone other often requested features. If it's high, then a different post should be created to contain the conversation.

Share this post


Link to post

Firstly, I would like to apologize sincerely for the frustration that the community has gone through during the weekend. Secondly, as the person responsible for the development and maintenance of geocaching.com, I totally own up the responsibility of this failure.

 

It's great to see such a response from all of you guys. It is refreshing to see that a business can admit it's mistakes and learn from them. And it's good to see that our comments don't fall on deaf ears. All we ask is to be kept in the loop.

 

1000 caches per pocket query - FANTASTIC NEWS.

 

Australian geocachers would love not having to worry about when to run their PQs yesterday in order to get them today.

 

Great idea! I add a second to the suggestion.

 

Wouldn't this be nice!

Edited by Big Matt and Shell

Share this post


Link to post

 

On a scale of 1-10, where "Nice To Have" is 1 and "Must Have" is 10, how important would it be to deliver PQs based on TZ? It would require some significant work, which would force us to postpone other often requested features. If it's high, then a different post should be created to contain the conversation.

 

You have given some excellent insights on the PQ generation process, but I wonder how you manage the requests. I'm very much a ad hoc PQ user. I have some predefined PQ's but don't run them unless I'm going in that direction. Or I may create one for a special trip, run it and then delete it. I have a couple bookmark based PQ's that I also run on a random order. All my PQ's are run once and then unchecked and sometimes deleted. So for me how quickly the PQ runs in real time is far more important than if it runs on some TZ based algorithm. I will, no doubt, completely destroy your cache ;)

 

Other users have a large number of scheduled PQ, and perhaps a TZ based algorithm would work best for them. Drawing on my OS experience, have you implemented queues for processing? If a PQ shows up and it is scheduled for today then put it in a FIFO queue and process it according to queue position. If a PQ shows up on Monday but is scheduled for Thursday it get tossed in the secondary queue. Perhaps you could expand a bit and if a PQ scheduled for today but does not have the uncheck when run option set it gets put on the front of the secondary queue. Those in the secondary queue get processed when the primary queue is empty, or by some resource sharing algorithm so the secondary queue does not get choked off. Naturally the secondary queue is a time based queue and perhaps this is where the TZ based processing happens. This will probably really upset the apple cart for you, but thought I would bring it up.

 

The difficult to process PQ's that you defer now could be placed in the secondary queue at an appropriate time based location. Yeah, that secondary queue is going to be a bit nasty, perhaps seven secondary queues would be better, one for each day of the week, but they will still be a bit nasty

 

Without having any real insight on your code base and all the nuances, it seems that improving the PQ processing speed is the most critical right now. Then perhaps a request management system and finally the new/extra features would be a way to proceed.

 

Just thinking out loud,

 

Jim

Share this post


Link to post

thanks for the update, would it be also possible to allow Archived caches to be included in PQ for at least 1 month after cache is archive, to allow incremental updates now that the PQ size has been increased.

Share this post


Link to post

On a scale of 1-10, where "Nice To Have" is 1 and "Must Have" is 10, how important would it be to deliver PQs based on TZ? It would require some significant work, which would force us to postpone other often requested features. If it's high, then a different post should be created to contain the conversation.

 

For me, -1. If implemented, it would be clearer what day was being charged for a PQ running. But since I'm GMT-5 it seems my repetitive PQs will not be running early anyway.

 

(I was confused last weekend when Sunday PQs were running Saturday night. After that, I couldn't tell if ad hoc PQ requests were not running because I was over the 5/day limit. Uncertain both on Saturday and Sunday how many PQs I was being charged for.)

Share this post


Link to post

 

(I was confused last weekend when Sunday PQs were running Saturday night. After that, I couldn't tell if ad hoc PQ requests were not running because I was over the 5/day limit. Uncertain both on Saturday and Sunday how many PQs I was being charged for.)

 

If you are over your limit, it gives you an appropriate error message on the PQ page.

Share this post


Link to post

 

(I was confused last weekend when Sunday PQs were running Saturday night. After that, I couldn't tell if ad hoc PQ requests were not running because I was over the 5/day limit. Uncertain both on Saturday and Sunday how many PQs I was being charged for.)

 

If you are over your limit, it gives you an appropriate error message on the PQ page.

 

I'm pretty sure I had more than 5 PQs run on Saturday because of the Sunday ones running early. There was no message, appropriate or not, when I tried to "cheat" and ask for some more PQs on Sunday. But since everything was crazy on the weekend, I couldn't tell what was expected and what wasn't.

Share this post


Link to post

Over the weekend there were a lot of issues with Pocket Queries being generated. For that I sincerely apologize.

 

In an attempt to speed up the Pocket Query Generator, the application(s) that processes tens of thousands of Pocket Queries a day, there was a cascading number of issues that resulted in duplicate Pocket Queries going out, Pocket Queries without logs and Pocket Queries not being sent out for a certain amount of time. Also in the process, Pocket Queries were brought down to a trickle, resulting in Pocket Queries not going out at all.

 

We're taking this issue very seriously at Groundspeak, and we'll be focusing on increasing the capabilities over this next weekend to ensure this never, ever happens again. By increasing this capacity we'll also be prepared to offer Pocket Queries with 1,000 results instead of 500. This will happen by the anniversary of geocaching (May 2, 2010). As those veterans to geocaching know, I'm not usually one to offer dates, but this one is solid.

 

Again, I offer my apology for this weekend's issues.

Thanks for the update, and looking forward to the 1000 queries, as now 500 only goes 24 miles from my house. ;)

Share this post


Link to post

All great points. Paul and I were talking about this issue of logging the caches from a different TZ a few weeks ago. I'm not sure if we have a story for that or not. We have a huge list of stories, so it's probable we do.

 

On the PQs, this discussion is touching on two different topics; delivering PQs in the users timezone, and processing PQs the fastest way possible. Right now, our goal is to make the generation of PQs as fast as possible. If we don't achieve this, moving to 1000 caches/PQ will not be possible. When you have as much data as we have, offer the range/flexibility of search options that we've offered, and have expectations of real-time data, it becomes difficult to optimize for all scenarios. One of the current constraints is the database. Some of the PQs put a lot of strain on it, so we try to run these process intensive searches during non-peak hours to make sure the site is as responsive as possible. There are solutions to this, but they all take time to implement and many of them will require extensive modification of our existing code base, which is not trivial and just not possible if we plan to hit our May deadline. As a geek, I'd love to change it all to the latest and greatest systems. Give me a moment to dream........ And I'm back. So, our safest option for now is utilize as much of the non-peak database time as we can. To be sure that our PQs include all the info from the previous day, we are splitting the PQs up into East & West hemispheres. The side-effect (not the goal) for our East Hemi brothers & sisters <flashing East Hemi Geocache gang sign> is that they'll get there PQs a few hours sooner. Actually, the real statement is that all PQs in the E.H. will be delivered a few hours earlier, and we assume that effects our East Hemi brothers & sisters <just got busted for flashing gang signs, but still means it>.

 

On a scale of 1-10, where "Nice To Have" is 1 and "Must Have" is 10, how important would it be to deliver PQs based on TZ? It would require some significant work, which would force us to postpone other often requested features. If it's high, then a different post should be created to contain the conversation.

 

If it's split into East Hemisphere and West Hemisphere, then you would split countries in half as to when they receive their PQs. The difference would be most notable in London (split in two by the Prime Meridian because that's where it was based). For example if someone living at the house at N51 33.319 E 0 0.008 moved across the street to N 51 33.332 W 0 0.008 (a 70ft difference), they would recieve their PQs several hours later.

It seems the best East/West line to split would be at 25 degrees West, past the UK, Ireland and Iceland (Iceland runs on UK time).

Share this post


Link to post

 

On a scale of 1-10, where "Nice To Have" is 1 and "Must Have" is 10, how important would it be to deliver PQs based on TZ? It would require some significant work, which would force us to postpone other often requested features. If it's high, then a different post should be created to contain the conversation.

 

You have given some excellent insights on the PQ generation process, but I wonder how you manage the requests. I'm very much a ad hoc PQ user. I have some predefined PQ's but don't run them unless I'm going in that direction. Or I may create one for a special trip, run it and then delete it. I have a couple bookmark based PQ's that I also run on a random order. All my PQ's are run once and then unchecked and sometimes deleted. So for me how quickly the PQ runs in real time is far more important than if it runs on some TZ based algorithm. I will, no doubt, completely destroy your cache :P

 

Other users have a large number of scheduled PQ, and perhaps a TZ based algorithm would work best for them. Drawing on my OS experience, have you implemented queues for processing? If a PQ shows up and it is scheduled for today then put it in a FIFO queue and process it according to queue position. If a PQ shows up on Monday but is scheduled for Thursday it get tossed in the secondary queue. Perhaps you could expand a bit and if a PQ scheduled for today but does not have the uncheck when run option set it gets put on the front of the secondary queue. Those in the secondary queue get processed when the primary queue is empty, or by some resource sharing algorithm so the secondary queue does not get choked off. Naturally the secondary queue is a time based queue and perhaps this is where the TZ based processing happens. This will probably really upset the apple cart for you, but thought I would bring it up.

 

The difficult to process PQ's that you defer now could be placed in the secondary queue at an appropriate time based location. Yeah, that secondary queue is going to be a bit nasty, perhaps seven secondary queues would be better, one for each day of the week, but they will still be a bit nasty

 

Without having any real insight on your code base and all the nuances, it seems that improving the PQ processing speed is the most critical right now. Then perhaps a request management system and finally the new/extra features would be a way to proceed.

 

Just thinking out loud,

 

Jim

 

My thoughts exactly. Are you sitting in the office across the street looking at my scrum board?

Share this post


Link to post

 

My thoughts exactly. Are you sitting in the office across the street looking at my scrum board?

 

Nope. But I am sitting at my desk across the sound :P

 

Jim

Share this post


Link to post

Personally I'm not particularly concerned if I get MY PQ before some guy in Japan or some guy in Belarus gets his.

I am concerned that my PQ should arrive within a reasonably predictable time-frame, so I can adjust accordingly. The wild time swings just leave us wondering what happened. :P

 

Do it via a diagonally sweeping spiral from the north pole to the south...just make it consistent! :laughing:

Share this post


Link to post

I regularly run ad hoc PQ's and as I live in Australia I always have to work out what time it is at HQ so that I click the correct day to run it.

 

Is it possible to get the current HQ time and day to show on the PQ page so that the correct one can be selected to get the PQ to run?

Share this post


Link to post

Over the weekend there were a lot of issues with Pocket Queries being generated. For that I sincerely apologize.

 

In an attempt to speed up the Pocket Query Generator, the application(s) that processes tens of thousands of Pocket Queries a day, there was a cascading number of issues that resulted in duplicate Pocket Queries going out, Pocket Queries without logs and Pocket Queries not being sent out for a certain amount of time. Also in the process, Pocket Queries were brought down to a trickle, resulting in Pocket Queries not going out at all.

 

We're taking this issue very seriously at Groundspeak, and we'll be focusing on increasing the capabilities over this next weekend to ensure this never, ever happens again. By increasing this capacity we'll also be prepared to offer Pocket Queries with 1,000 results instead of 500. This will happen by the anniversary of geocaching (May 2, 2010). As those veterans to geocaching know, I'm not usually one to offer dates, but this one is solid.

 

Again, I offer my apology for this weekend's issues.

 

Now if I could get Garmin to accept more than 2,000. I could to the whole world, Thanks for the update. ;)

Share this post


Link to post

By increasing this capacity we'll also be prepared to offer Pocket Queries with 1,000 results instead of 500. This will happen by the anniversary of geocaching (May 2, 2010). As those veterans to geocaching know, I'm not usually one to offer dates, but this one is solid.

 

Thanks for the increase, but can I get my birthday present(^^See above^^) from you on my birthday this year?? You'd only need to move the capacity up a few days....

 

The Steaks

 

P.S. 18 March is the date.

Share this post


Link to post

thanks for the update, would it be also possible to allow Archived caches to be included in PQ for at least 1 month after cache is archive, to allow incremental updates now that the PQ size has been increased.

 

I hate to be the bearer of bad news, but it has been clear that Groundspeak does NOT want to EVER give the community the possibility of searching for an archived cache. Thus, they will probably never give us the ability to do a PQ for archived caches, OR include recent archives in the PQs.

 

I do applaud you on thinking outside of the proverbial ammo can of requesting for the ability to create a PQ with Archived caches in it though.

 

The Steaks

 

*Been there done that. It's been beat into my head with many ammocans*

Share this post


Link to post

 

My thoughts exactly. Are you sitting in the office across the street looking at my scrum board?

The fact you use Scrum is interesting, hopefully it helps you meet the changing demands of geocaching.

Share this post


Link to post

With the 2nd of May rapidly approaching, does anyone know how the developments are going on the 1000 cache limit increase?

Share this post


Link to post

With the 2nd of May rapidly approaching, does anyone know how the developments are going on the 1000 cache limit increase?

 

I hear they are up to 675 now. :)

 

But maybe I hit it at the right times, but this weekend it seemed as if the generator was pumping out the PQ's in seconds.

Share this post


Link to post
Guest
This topic is now closed to further replies.
Sign in to follow this  
Followers 12

×
×
  • Create New...