Jump to content

Date in Field Notes


Avernar

Recommended Posts

Can someone please fix the date handling in the Field Notes upload? Please? I've complained about this before but it seems to be falling on deaf ears.

 

I wouldn't mind so much as I can compensate for the bug by altering the field notes file before upload, but every now and then something on the site changes and throws it off again. If I don't notice I'll accidentally log a whack of caches with the wrong date and have to go back and edit the log entries. Very frustrating.

 

This time there's a 13 hour offset being added to the timestamps. The only thing that should be added is my timezone offset in my profile (adjusted for daylight savings when DST is in effect).

 

Sorry for the harsh tone of this post but this is driving me insane as this has changed several times in the past, none of which were correct.

Link to comment

I'm having problems with dates too, although I think I have a different issue. I've just noticed this issue, so maybe it's a result of the latest site release issues.

 

On my field notes page, I had a few caches listed with a date of 01/17/10 and a couple dated 01/18/10. Since I wasn't ready to log the 01/17/10 caches yet, I decided to go ahead and get the 01/18/10 caches logged. When I went in to compose my log, the date had been changed to 01/17/10. This happened on both caches.

Link to comment

Are you by chance using a GSAK macro to first collect your field notes off of your device? I am seeing correct behavior right now with the "raw" field notes off of my Garmin, but if I use the "Colorado Import and Log" macro in GSAK first and then upload from there, my timestamps are off. I don't believe that the macros have adjusted to corrections we have made to the way the site interprets the field note timestamps.

Link to comment

No, not using GSAK for logs. I have a custom program which takes the time from the timestamp and places it as the first part of the comment. It also adjustes the timestamp as log.aspx doesn't use my configured timezone and things logged after midnight would have the previous day's date.

 

Here's a sample file to test the dates:

 

GC1WPWJ,2010-01-19T00:00Z,Found it,"00:00 "
GC1WPWJ,2010-01-19T01:00Z,Found it,"01:00 "
GC1WPWJ,2010-01-19T02:00Z,Found it,"02:00 "
GC1WPWJ,2010-01-19T03:00Z,Found it,"03:00 "
GC1WPWJ,2010-01-19T04:00Z,Found it,"04:00 "
GC1WPWJ,2010-01-19T05:00Z,Found it,"05:00 "
GC1WPWJ,2010-01-19T06:00Z,Found it,"06:00 "
GC1WPWJ,2010-01-19T07:00Z,Found it,"07:00 "
GC1WPWJ,2010-01-19T08:00Z,Found it,"08:00 "
GC1WPWJ,2010-01-19T09:00Z,Found it,"09:00 "
GC1WPWJ,2010-01-19T10:00Z,Found it,"10:00 "
GC1WPWJ,2010-01-19T11:00Z,Found it,"11:00 "
GC1WPWJ,2010-01-19T12:00Z,Found it,"12:00 "
GC1WPWJ,2010-01-19T13:00Z,Found it,"13:00 "
GC1WPWJ,2010-01-19T14:00Z,Found it,"14:00 "
GC1WPWJ,2010-01-19T15:00Z,Found it,"15:00 "
GC1WPWJ,2010-01-19T16:00Z,Found it,"16:00 "
GC1WPWJ,2010-01-19T17:00Z,Found it,"17:00 "
GC1WPWJ,2010-01-19T18:00Z,Found it,"18:00 "
GC1WPWJ,2010-01-19T19:00Z,Found it,"19:00 "
GC1WPWJ,2010-01-19T20:00Z,Found it,"20:00 "
GC1WPWJ,2010-01-19T21:00Z,Found it,"21:00 "
GC1WPWJ,2010-01-19T22:00Z,Found it,"22:00 "
GC1WPWJ,2010-01-19T23:00Z,Found it,"23:00 "

 

The Field Notes (fieldnotes.aspx) page shows 00:00 to 07:00 as being on the 18th so it's subtracting 13 then adding 5 for my timezone. If I set my timezone to GMT then 00:00 to 12:00 are on the 18th which is just the subtract 13.

 

On the log (log.aspx) page 00:00 to 12:00 are always on the 18th regardless of timezone.

 

A few days ago the subtract 13 wasn't happening so my program just had to subtract 5 hours from the timestamp so that log.aspx would have the correct date. This messes up the date on the fieldnotes.aspx page but that's just cosmetic.

 

And months ago the subtract 13 was also happening so it seems to come and go.

 

So basically what needs to be fixed:

 

1) Figure out where this 13 hour adjustment is coming from

2) Make log.aspx use my configured timezone

3) Make log.aspx and fieldnotes.aspx use the daylight savings setting when daylight savings is active

Edited by Avernar
Link to comment

What needs to happen is you need to stop adjusting the time stamps in the file with your custom program. We have made changes in the web site so that it expects the time stamps in field notes to be UTC time. Currently the site then converts and saves those to Seattle local time. You will see the timestamps on the field notes page as well as resulting logs in Seattle time.

 

Why Seattle time and not UTC time? Only because of precedence. Everything for the last 9 years has been stored in Seattle time and so that has become by default the standard for the database.

 

Moving forward, the site will be overhauled to adjust all of these times to the user's (or cache's) specified timezone, but we're not there quite yet.

Link to comment

What needs to happen is you need to stop adjusting the time stamps in the file with your custom program.

Sorry, not going to happen. That's what I did when I first got my Colorado and had to manually adjust the date on all logs between 12AM and 5AM. It's a royal pain because if I forget I have to go back and edit those logs to correct it.

 

I want the date on my log entries to be in my timezone as I put the time at the start of the log. So after midnight it looks like the cache was found 24 hours earlier.

 

We have made changes in the web site so that it expects the time stamps in field notes to be UTC time. Currently the site then converts and saves those to Seattle local time. You will see the timestamps on the field notes page as well as resulting logs in Seattle time.

No, I just explained in my previous post, it doesn't do this. Seattle is -8, not -13! The dates are neither my timezone or Seattle's.

 

The field notes page actually does use my timezone setting and prior to a few days ago it didn't do this weird -13 hours business so that was correct. The log entry page is what doesn't use the timezone setting and is what necessitates the adjustment of the timestamp before upload.

 

Why Seattle time and not UTC time? Only because of precedence. Everything for the last 9 years has been stored in Seattle time and so that has become by default the standard for the database.

It doesn't matter what it is in the database. The translation error is happening before the log is even saved. I can pick a completely different date before hitting the save button.

 

What difference does it make if my program adjusts the timestamp before the upload or I manually correct the dropdown box after? No difference. What's going to be saved in the database is the contents of that date box, not the original timestamp.

 

Moving forward, the site will be overhauled to adjust all of these times to the user's (or cache's) specified timezone, but we're not there quite yet.

It already does this on the field notes page. But this mysterious -13 adjustment that keeps on occurring and disappearing every 4-6 months is the basis of my complaint.

Link to comment

I think you are misunderstanding. When you upload the test file you did above (with no changes made by your program), you and everyone else who uploads the file will see 8 logs dated on 1/18 and the other 16 logs dated 1/19. This is because the system takes those timestamps as UTC (which they are) and converts them to Seattle time. Seattle is -8, so 8 of them are seen falling onto the previous day.

 

When you log one of those field notes that fall on 1/18, the log will be dated 1/18. That's because that is what the time was in Seattle, and that is currently the default time used throughout the site. Eventually, dates will be adjusted off of this standard to reflect the user's timezone.

 

Now, I'm not sure what your custom program is doing currently, but if you are seeing something else than what I describe above, it is likely because of what that program is doing. You can certainly choose to continue using it to change things now to reflect your current timezone, but that will come back to bite you in the future when we implement timezone adjustments of timestamps on the site.

Link to comment

From what I understand:

 

A few days ago, UTC is being treated as local time for logging. Avernar subtracts 5 hours so that his logs reflect the correct time.

 

Currently, UTC is being converted to PST (Seattle time) for logging. If Avernar subtracts another 5 hours this puts him at 13 hours behind UTC, and 8 hours off his time zone.

 

If Avernar now adds 3 hours to the time in the field notes (the difference between Ontario and Seattle), it should work correctly.

 

Eventually Groundspeak will fix the code to use the user's time zone (set in profile preference). After the code is fixed, Avernar should not need to do any adjustment.

 

 

Of course, if he moves to Ontario, California, problem would be solved too. And none of that pesky snow...

Edited by Chrysalides
Link to comment

The only thing I want changed in the field notes is for the user to have the ability to ignore notes before a date when uploading fresh.

 

I clear all of my cache notes from the website after logging and the next time it wants to import all of them again. I don't have the option to ignore all notes before a certain date when my cache notes (online) are empty.

Link to comment

I think you are misunderstanding. When you upload the test file you did above (with no changes made by your program), you and everyone else who uploads the file will see 8 logs dated on 1/18 and the other 16 logs dated 1/19. This is because the system takes those timestamps as UTC (which they are) and converts them to Seattle time. Seattle is -8, so 8 of them are seen falling onto the previous day.

No, I understand you perfectly. It looks like you are misunderstanding me. The field notes page is ALSO adding in my time zone. I just checked and the -13 the site was doing has been changed to -8 now (intentionally or not).

 

If I set my timezone in my profile to Grenwich Mean Time I do get 8 falling on the previous day.

 

However, if I set my timezone to -5 Eastern I only see 3 falling on the previous day.

 

When you log one of those field notes that fall on 1/18, the log will be dated 1/18. That's because that is what the time was in Seattle, and that is currently the default time used throughout the site.

For the log page it was -13 earlier today. Now it is -8 like you say.

 

Eventually, dates will be adjusted off of this standard to reflect the user's timezone.

It's already trying to do it for the fieldnotes.aspx page.

 

Now, I'm not sure what your custom program is doing currently, but if you are seeing something else than what I describe above, it is likely because of what that program is doing. You can certainly choose to continue using it to change things now to reflect your current timezone, but that will come back to bite you in the future when we implement timezone adjustments of timestamps on the site.

What I was seeing was happening with the clean test file I posted above so my program wasn't involved in all the things I just described.

 

The problem with retroactively showing previous logs in the user's timezone in the future is that the site is not recording a time for manual (non field note) log entries. You can't just default the time to 00:00 for those entries and treat them as UTC because it will show the wrong date for some logs for anyone not using Grenwich Mean Time. Those timestamps are effectively in the user's timezone for all intents and purposes and will have to be adjusted by their timezone setting before treating them as UTC. And don't forget to adjust for daylight savings time during the adjustment.

 

As for field note logs, I assume that you're implying that the time portion of the timestamp is somehow being passed hidden through the log screen to be saved in the database, correct?

Link to comment

Currently, UTC is being converted to PST (Seattle time) for logging. If Avernar subtracts another 5 hours this puts him at 13 hours behind UTC, and 8 hours off his time zone.

Nope. It was being adjusted by 13 hours by the site (not sure what time zone that is). NOW, it's doing it by 8 (PST).

 

The site did this before many months ago and I have a line commented out in the program to add the 13 before doing the -5 (or -4 if DST is in effect).

 

Now that the site is doing PST I'll just uncomment the line and change the 13 to and 8 and everything works for me.

 

What my complaint is that it keeps CHANGING. One month it's 0, another its 13. Now it's 8. Argh!

Link to comment

You can certainly choose to continue using it to change things now to reflect your current timezone, but that will come back to bite you in the future when we implement timezone adjustments of timestamps on the site.

Forgot to mention, this will bite whoever implements that feature. The site has not been consistent with it's adjustment. Like I said, for a while it was adjusting by 13, then not adjusting, then adjusting by 13 again and now by 8. So whatever the time portion of the timestamp in the database is, it's meaningless now.

 

You're going to have a lot of users with incorrect log dates...

Edited by Avernar
Link to comment

It's changed once that I know of and that's in this latest release. We now are properly taking the field note time as UTC whereas before we were inconsistently taking it as Seattle time or UTC depending on the situation.

 

I don't doubt that you've encountered an issue but I can't reproduce the problem you are describing. Your timezone setting is not involved in any way in how the timestamps in the field notes are interpreted at upload. They are taken as UTC and then converted into Seattle time. If you upload and *then* switch your timezone setting you will see a change in the time displayed on the field notes page, but that's only because the site is adjusting for the perceived change. The actual time associated with the log is unaffected.

 

Perhaps some bad behavior is being masked by the fact that I am in Seattle's timezone. Could you walk me through the precise steps you are taking, along with your account time settings and any chnges, in order to see the 13 hour offset you've described?

Link to comment
Your timezone setting is not involved in any way in how the timestamps in the field notes are interpreted at upload. They are taken as UTC and then converted into Seattle time. If you upload and *then* switch your timezone setting you will see a change in the time displayed on the field notes page, but that's only because the site is adjusting for the perceived change. The actual time associated with the log is unaffected.

My head hurts from reading this...

 

I did a quick experiment. Field note has 1 entry : 2010-01-20T00:04Z. Time zone preference set to PST (UTC-8). Uploaded.

 

Field notes display shows Jan 19 2010, as it is converted to Seattle time (unrelated to my TZ setting). When I click "Compose Log", the date is Jan 19, 2010.

 

Now I go to my profile and changed my time zone to NZ (UTC+12). I go back to field notes. Display now shows Jan 18 2010. Which doesn't make sense. It looks like it is adjusted the wrong direction.

 

To make matters worse, if I click on "Compose Log", it fills in the date with Jan 19, 2010.

 

Since change for this is being planned, all I can gather from this is : don't change your time zone after uploading field notes.

Edited by Chrysalides
Link to comment

Note: I edited this post as the one below invalidates most of what I wrote here.

 

Could you walk me through the precise steps you are taking, along with your account time settings and any chnges, in order to see the 13 hour offset you've described?

Try the experiment I described in the post below. If the site was always converting to just PST then in all three cases only the first 8 Compose Log links would be the previous day.

Edited by Avernar
Link to comment

Chrysalides just figured it out. The site *is* adding in the timezone from the profile during the upload in addition to the site PST correction.

 

I'm getting PST + EST for a total of 13. Changing the timezone after the fact just makes the field notes page confusing because it's not canceling out your timezone added during the upload.

 

I just tried an experiment using my sample file:

 

1) Set tz to GMT and upload file. Field notes page shows 8 (8 - 0) previous and Compose Log links has first 8 previous.

 

2) Set tz to EST and upload file. Field notes page shows 8 (13 -5) previous and Compose Log links has first 13 previous.

 

3) Set tz to PST and upload file. Field notes page shows 8 (16 - 8) previous and Compose Log links has first 16 previous.

 

The field notes page is using the timezone setting which is hiding the fact that the upload also used the timezone setting (in addition to the site timezone of PST).

Edited by Avernar
Link to comment
What my complaint is that it keeps CHANGING. One month it's 0, another its 13. Now it's 8. Argh!

Yeah, that would be frustrating. If it's broken, at least leave it broken at a consistent value so that we can correct for it on our side.

Looks like that was a false alarm. The fluctuation between 8 and 13 today was just me changing the my timezone between EST and GMT to figure out what was going on.

 

But it has changed several times in the past between no adjustment and the current (broken) adjustment.

Link to comment

I don't know if this will help or not, but I did experience an issue with this and couldn't figure out why. The following three lines come directly from my Oregon, unedited:

 

GC1VN20,2010-01-16T13:58Z,Found it,""

GC1WDZY,2010-01-16T14:12Z,Found it,""

GC1KWMQ,2010-01-16T14:52Z,Found it,""

 

The first posted as a 1/15 find and the next 2 posted as 1/16 finds. That is why you see that I edited the log for GC1VN20 clicky; I realized the date was wrong and changed it to the 16th after posting.

Link to comment

Sounds like you were manually correcting an error the site had for a long while.

 

Now they fixed the error and your corrections have created an error for you.

 

Nothing more than that.

No, it's actually a lot more than that. I didn't believe it at first, but after a few hours of experimenting, I agree with Avernar. He uncovered a particularly nasty bug.

 

The site tries to correct it, but in doing so made the situation worse.

 

Let's ignore changing time zone after uploading. That would seriously mess things up.

 

For this experiment, I set my time zone to Eastern (UTC-5).

 

I have a set of data. I'll list the date / time of the first 7:

  • 2010-01-20T07:58Z
  • 2010-01-20T08:58Z
  • 2010-01-20T09:58Z
  • 2010-01-20T10:58Z
  • 2010-01-20T11:58Z
  • 2010-01-20T12:58Z
  • 2010-01-20T13:58Z

Now, in EST, all these would be found on 2010-01-20. In PST, the first would be found on 2010-01-19, and the rest will be found on 2010-01-20.

 

If you upload to the site, that is how it is displayed - as PST. Moun10bike explained that. Not ideal, but we get it. I'd call it a bug, but since it is how it was designed to work, let's call it a feature.

 

769482416_upbB5-S.jpg

 

Now comes the really strange part.

 

When you click on "Compose Log", the date may not be the same as what was displayed on the Field Notes page.

 

If, however, I set my time zone to EST, before I upload my field notes, then I see the following:

  • field notes time stamp - display - log page date
  • 2010-01-20T07:58Z - 2010-01-19 - 2010-01-19
  • 2010-01-20T08:58Z - 2010-01-20 - 2010-01-19
  • 2010-01-20T09:58Z - 2010-01-20 - 2010-01-19
  • 2010-01-20T10:58Z - 2010-01-20 - 2010-01-19
  • 2010-01-20T11:58Z - 2010-01-20 - 2010-01-19
  • 2010-01-20T12:58Z - 2010-01-20 - 2010-01-19
  • 2010-01-20T13:58Z - 2010-01-20 - 2010-01-20

This is what I see on the first 6 entries:

 

769482425_6Zbrk-S.jpg

 

What appears to be happening is that:

 

1. UTC converted to PST for display

2. PST (mistakenly treated as UTC) converted to local time (based on TZ setting) in log.

 

*note* this is what appears to be happening and predicts the symptoms if you do not change the TZ after a field note is uploaded, but avernar has a better explanation in post #26 that explains what may actually be happening.

 

So, 12:58Z, for example, would be converted to 04:58 (same day) for display, then converted to 23:58 (previous day) when you click on "Compose Log".

 

To test my assumption, I nuked the field notes, changed my TZ to Pacific, then uploaded the same set of data. Anything with a field notes time stamp of 16:00Z would show up in the log page as the previous day. And I was correct:

  • field notes time stamp - display - log page date
  • 2010-01-20T07:58Z - 2010-01-19 - 2010-01-19
  • 2010-01-20T08:58Z - 2010-01-20 - 2010-01-19
  • 2010-01-20T09:58Z - 2010-01-20 - 2010-01-19
  • 2010-01-20T10:58Z - 2010-01-20 - 2010-01-19
  • 2010-01-20T11:58Z - 2010-01-20 - 2010-01-19
  • 2010-01-20T12:58Z - 2010-01-20 - 2010-01-19
  • 2010-01-20T13:58Z - 2010-01-20 - 2010-01-19
  • 2010-01-20T13:58Z - 2010-01-20 - 2010-01-19
  • 2010-01-20T13:58Z - 2010-01-20 - 2010-01-19
  • 2010-01-20T13:58Z - 2010-01-20 - 2010-01-20

In order to get the display and the log to match, I need to set my time zone to UTC.

 

In order to get the actual find time and the display time to match, I need to adjust the field notes time stamp if I'm not in PST (which, happily, I am).

Edited by Chrysalides
Link to comment
But my device records the fieldnotes and stamps it in UTC time. My computer is only involved in that it uploads the file to Groundspeak.

Yes. It is Groundspeak that interprets the time incorrectly.

 

I've uploaded some screen shots, it may help get my point across better.

 

What is your time zone in your profile set to? Is it set to Mountain time? Let's assume it is.

 

Let's say, you go out at night of Jan 19, and found a geocache at 12:30 a.m. (after midnight so local time is now Jan 20). You press "Found" on your GPSr.

 

The field notes record "Found it" with time stamp of 2010-01-20T07:30Z since MST is UTC-7.

 

You upload it, and it shows in the Field Notes page as 2010-01-19, because it converts it to PST. Time, not shown, is 23:30

 

You click "Compose Log", and the date it filled in automatically is 2010-01-19. Time, not shown, is 16:30 (23:30 - 7 hours).

 

Now, let's say you found a cache at 5 a.m. in the morning of Jan 20.

 

The field notes record "Found it" with time stamp of 2010-01-20T12:30Z since MST is UTC-7.

 

You upload it, and it shows in the Field Notes page as 2010-01-20, because it converts it to PST. Time, not shown, is 4:30Z

 

You click "Compose Log", and the date it filled in automatically is 2010-01-19. Time, not shown, is 21:30 (04:30 - 7 hours).

 

Give it a try.

Edited by Chrysalides
Link to comment

1. UTC converted to PST for display

2. PST (mistakenly treated as UTC) converted to local time (based on TZ setting) in log.

You almost have it correct:

 

1. UTC is converted to PST during the upload.

2. The value in 1 is mistakenly converted again during the upload from UTC to local time (based on TZ setting).

 

3a. In the filed notes screen the value in 2 is converted back using the TZ setting for display. This makes it look like PST again.

 

3b. In the log screen you see the value in 2.

 

This is why changing the TZ after upload makes things worse as it affects 3a.

Link to comment

But my device records the fieldnotes and stamps it in UTC time. My computer is only involved in that it uploads the file to Groundspeak.

You are correct. But Groundspeak is doing a double timezone conversion during the upload now.

 

You'll only notice it for finds between midnight and X:00am where X is your the negative of your TZ offset. i.e. For PST this would be between 12am and 8am, for EST this would be between 12am and 5am.

Link to comment
But my device records the fieldnotes and stamps it in UTC time. My computer is only involved in that it uploads the file to Groundspeak.

Yes. It is Groundspeak that interprets the time incorrectly.

 

I've uploaded some screen shots, it may help get my point across better.

 

What is your time zone in your profile set to? Is it set to Mountain time? Let's assume it is.

 

Let's say, you go out at night of Jan 19, and found a geocache at 12:30 a.m. (after midnight so local time is now Jan 20). You press "Found" on your GPSr.

 

The field notes record "Found it" with time stamp of 2010-01-20T07:30Z since MST is UTC-7.

 

You upload it, and it shows in the Field Notes page as 2010-01-19, because it converts it to PST. Time, not shown, is 23:30

 

You click "Compose Log", and the date it filled in automatically is 2010-01-19. Time, not shown, is 16:30 (23:30 - 7 hours).

 

Now, let's say you found a cache at 5 a.m. in the morning of Jan 20.

 

The field notes record "Found it" with time stamp of 2010-01-20T12:30Z since MST is UTC-7.

 

You upload it, and it shows in the Field Notes page as 2010-01-20, because it converts it to PST. Time, not shown, is 4:30Z

 

You click "Compose Log", and the date it filled in automatically is 2010-01-19. Time, not shown, is 21:30 (04:30 - 7 hours).

 

Give it a try.

That's exactly what I saw which led me to make my post (#2). I just didn't know why. It is the same issue - the caches that I had problems with, I had found early in the morning. Thanks for clearing that up. I'll have to watch future logs until this one has been fixed.

Link to comment

To be clear, this double-bumping of TZ offsets isn't new; it's been wrong this way since field notes were introduced and has been reported several times, e.g.

 

http://forums.Groundspeak.com/GC/index.php...t&p=3441375

 

Any program storing anything other than GMT and applying timezones at the edges where it displays things to users will eventually wrap itself around its own axle. 'set logtime = logtime + OFFSET where 1' on a large database is a scary thing to do, but storing data in either server-time or user-time will always come back to byte you.

 

"This has all happened before. It will all happen again." - Caprica Six

Link to comment

To be clear, this double-bumping of TZ offsets isn't new; it's been wrong this way since field notes were introduced and has been reported several times, e.g.

 

http://forums.Groundspeak.com/GC/index.php...t&p=3441375

 

Any program storing anything other than GMT and applying timezones at the edges where it displays things to users will eventually wrap itself around its own axle. 'set logtime = logtime + OFFSET where 1' on a large database is a scary thing to do, but storing data in either server-time or user-time will always come back to byte you.

It's been reported before, but I'm not sure if Groundspeak (specifically, the developer responsible for this) understands the problem exactly, even after this round of exercise. One can hope, however.

 

I do not quite understand the second part of your post, however. Getting date and time right is not impossible, merely tedious in terms of design and testing.

 

In the meantime, I won't go looking for geocaches between midnight and 8 a.m. to minimize confusion :P

Edited by Chrysalides
Link to comment

No, it's not impossible. Having multiple TZs in play merely makes things crazy complicated. If you're interfacing with external tools (libraries, parsers, language runtimes, etc.) they're almost sure to deal with GMT natively. My point was that bucking that trend and having GMT in some places and local time for others is a sure recipe for a disconnect.

Link to comment
Guest
This topic is now closed to further replies.
×
×
  • Create New...