Jump to content

Field notes file format


Chrysalides

Recommended Posts

Field notes look like it is in a very simple format. An example:

 

GC123,2010-01-31T17:45Z,Found it,"Comments"

 

GC123 is obviously the cache's GC code. The date / time is in Zulu time (UT / GMT) with one small caveat : there is a bug so that it is converted twice. The details is in a long post somewhere, but what it means is that time works very strangely. If you want the date to be correct and you're in Pacific time zone, your finds should be after 16:00Z.

 

However, creating a simple ASCII file will not work. I found out that the file generated by the Garmin is in UTF16, Little Endian. The first 2 bytes of the file is FF FE. The 2 bytes after that is "GC", encoded as 47 00 43 00. Each character is stored as 16 bits with the least significant byte first.

 

After I created the file in that format, everything worked fine. So I thought I'd share my findings here and hope someone finds it useful. If you have an idea of how field notes can be used to make your life easier, do share it here.

 

I generate a list of geocaches my daughter needs to log as field notes and upload it to her account. That way she can work through them at her leisure, without wondering what are the GC code and date of find.

Link to comment

Thanks for the information. I have an caching application for my Blackberry that generates the file so I have not looked into this before. I have tried to create a file by hand before and it would not work and I didn't know why. With the help from your discovery, I found that if I use Notepad++ (best free file editor there is IMHO) and set the encoding to UCS-2 Little Endian then the file works! Thanks again for the information.

Link to comment

Long story short - I like to make up my own things and have 'made' a sort of cache-management app for my netbook. This thread inspired me to try and create my own field notes from the database, so that when I reported a cache as 'found' or DNF I could create a fieldnote with my comments and just upload that to the GC site. Two things I found out:

 

The last comma-separated value in the text file is the comments - in the example above this is enclosed by speech marks ("..."). It IS possible to upload a fieldnote which does not enclose the string with speech marks, but then the comment better not have any commas in it! (Come to think of it, what happens when there's a speech mark in the comment? Perhaps it's gotta be escaped - I'll try it....)

 

Secondly, and more importantly - the PHP function to convert the string into that required by the GC site is:

 

mb_convert_encoding($string, "UCS-2LE")

 

Hopefully this should help those looking for similar functions in other languages! Thanks to the two previous posters for helping me determine that I was looking for "UCS-2LE" as the encoding format!

Edited by tiiiim
Link to comment

I've been editing the file by hand to insert the time into my log or sometimes to write a brief sentence or two before uploading. This saves me from having to paste in the info later when composing my log on the site.

 

Messing around with the file, I found that you can submit other log types too... for example if you log an event as "found" on your GPS, when composing your log on the site you need to select the "attended" log type (since you can't post a Found It to an event). Sure it's quicker to just select Attended from the menu, but there are other possibilities too. If you enter a log type into your field notes which a particular cache type can't accept, when you compose your log the dropdown box reverts to - select one -

 

GCGQ5G,2010-02-09T17:39Z,Needs Archived,""
GC1Z7DN,2010-02-09T18:24Z,Attended,""
GC1Z7DN,2010-02-09T28:44Z,Published,""

 

(The "publish" log type didn't work though... I thought it was at least worth a try :) )

 

487f2eeb-0f63-4d5d-8a8b-a8ea047e00fb.jpg

Edited by DavidMac
Link to comment

I've been editing the file by hand to insert the time into my log or sometimes to write a brief sentence or two before uploading. This saves me from having to paste in the info later when composing my log on the site.

 

Messing around with the file, I found that you can submit other log types too... for example if you log an event as "found" on your GPS, when composing your log on the site you need to select the "attended" log type (since you can't post a Found It to an event). Sure it's quicker to just select Attended from the menu, but there are other possibilities too. If you enter a log type into your field notes which a particular cache type can't accept, when you compose your log the dropdown box reverts to - select one -

 

GCGQ5G,2010-02-09T17:39Z,Needs Archived,""
GC1Z7DN,2010-02-09T18:24Z,Attended,""
GC1Z7DN,2010-02-09T28:44Z,Published,""

 

(The "publish" log type didn't work though... I thought it was at least worth a try :) )

 

487f2eeb-0f63-4d5d-8a8b-a8ea047e00fb.jpg

Let me know how that "Published" log type works for you.

Link to comment

Field notes look like it is in a very simple format. An example:

 

GC123,2010-01-31T17:45Z,Found it,"Comments"

 

GC123 is obviously the cache's GC code. The date / time is in Zulu time (UT / GMT) with one small caveat : there is a bug so that it is converted twice. The details is in a long post somewhere, but what it means is that time works very strangely. If you want the date to be correct and you're in Pacific time zone, your finds should be after 16:00Z.

 

However, creating a simple ASCII file will not work. I found out that the file generated by the Garmin is in UTF16, Little Endian. The first 2 bytes of the file is FF FE. The 2 bytes after that is "GC", encoded as 47 00 43 00. Each character is stored as 16 bits with the least significant byte first.

 

After I created the file in that format, everything worked fine. So I thought I'd share my findings here and hope someone finds it useful. If you have an idea of how field notes can be used to make your life easier, do share it here.

 

I generate a list of geocaches my daughter needs to log as field notes and upload it to her account. That way she can work through them at her leisure, without wondering what are the GC code and date of find.

[/quote

 

Is anyone able to explain this in simple terms that someone only a bit computer literate can understand. I edited my visit file and since then I cannot upload my field notes.

Link to comment
Is anyone able to explain this in simple terms that someone only a bit computer literate can understand. I edited my visit file and since then I cannot upload my field notes.

I can explain it in simpler terms, but I don't think you're really interested and just want it to work (that "insight" comes from trying to explain things to my wife - if I'm wrong, let me know, and I'll post something with more background).

 

If you're using Windows, download Notepad++ (free, available from http://notepad-plus.sourceforge.net/uk/download.php ). You want to download the binary installer. When editing, select menu "Encoding", choose "Encode in UCS-2 Little Endian".

 

If you're on a Mac, there should be Mac editors that can do that as well.

 

Also make sure that the fields are exactly as described. As noted:

 

GC123,2010-01-31T17:45Z,Found it,"Comments"

 

So, no space before / after the commas, GC in uppercase, time exactly in that format (2 digit months, 2 digit days, 24 hour time, Z to indicate that the time is UTC / GMT. Found it spelled exactly and in that case (not "Found It" - that may not work). If no comments are present, the last field should be two empty double quotes (i.e. "")

Link to comment

I'm not able to use this right now, but... one question comes to mind.

 

Is there a limit on the max length (characters) on the Comment field or any other for that matter?

 

Just wondering, since I haven't / can't use field notes, at least GPS generated ones.

 

Doug 7rxc

Link to comment

I'm glad that I came across this topic, because it helped me to solve a problem that I have been mulling over for quite some time. I have been wanting to bring my daughter's account up to date with the cache finds that she found with me, but I had never entered any finds for her for the past three years. I loaded "My Finds" into GSAK and exported a CSV file with just the GC code, date found, and cache name. I opened the CSV file in Excel and changed the date format form MM/DD/YYYY to YYYY-MM-DD. I deleted the lines for caches that she did not find, then I opened the file in TextPad (a fantastic text editor). I recorded a macro in TextPad to add the rest of the date field and to add a log comment. I uploaded the file to her field notes, then was able to create a log entry with two clicks on each field note. It took a while, with over 250 notes, but it was much easier than trying to create a log from scratch for each of her finds.

Link to comment
Field notes look like it is in a very simple format. An example:

 

GC123,2010-01-31T17:45Z,Found it,"Comments"

...

However, creating a simple ASCII file will not work. I found out that the file generated by the Garmin is in UTF16, Little Endian. The first 2 bytes of the file is FF FE. The 2 bytes after that is "GC", encoded as 47 00 43 00. Each character is stored as 16 bits with the least significant byte first.

Guess what? This is all so much easier now. Files no longer need to be in UTF format. As of a few site updates ago, they can now be straight ASCII. Yay!

 

I loaded "My Finds" into GSAK and exported a CSV file with just the GC code, date found, and cache name. I opened the CSV file in Excel and changed the date format form MM/DD/YYYY to YYYY-MM-DD. ... then I opened the file in TextPad (a fantastic text editor)... to add the rest of the date field and to add a log comment.

The file format is pretty flexible. It's just a CSV file.

"GC1HQA5",2009-12-11T13:05Z,"Found it","blah blah"
"GC1845R",2009-12-11T13:06Z,"Didn't find it","yadda yadda"

Note the quote marks around the GC Code and log type. This is the output of Open Office's CSV export. It will upload just fine to the website. I suspect Excel's CSV export will work as well.

 

Is there a limit on the max length (characters) on the Comment field or any other for that matter?

 

Just wondering, since I haven't / can't use field notes, at least GPS generated ones.

I haven't tried pushing the limits, but 200 characters seems to work well. My bet is it will work up to the maximum log size (2000 chars?). Go ahead and mock up a file in Excel or Open Office and let us know what you find out :)

Edited by Lil Devil
Link to comment
Guess what? This is all so much easier now. Files no longer need to be in UTF format. As of a few site updates ago, they can now be straight ASCII. Yay!

That is great news. I've been using Notepad++ only for those files because it was the only editor that would do UTF-16 properly.

 

I've also written a bunch of Python code to fix the time zone problem, among other things. It does UTF-16 nicely, but it will be easier if I can convert it to ASCII.

 

Minor rant: determining the time zone for coordinates is trivial. Why doesn't Groundspeak convert the times in field notes from UTC to local time when you upload them?

Link to comment

 

Guess what? This is all so much easier now. Files no longer need to be in UTF format. As of a few site updates ago, they can now be straight ASCII. Yay!

[snip!]

 

The file format is pretty flexible. It's just a CSV file.

"GC1HQA5",2009-12-11T13:05Z,"Found it","blah blah"
"GC1845R",2009-12-11T13:06Z,"Didn't find it","yadda yadda"

Note the quote marks around the GC Code and log type. This is the output of Open Office's CSV export. It will upload just fine to the website. I suspect Excel's CSV export will work as well.

 

Is there a limit on the max length (characters) on the Comment field or any other for that matter?

 

Just wondering, since I haven't / can't use field notes, at least GPS generated ones.

I haven't tried pushing the limits, but 200 characters seems to work well. My bet is it will work up to the maximum log size (2000 chars?). Go ahead and mock up a file in Excel or Open Office and let us know what you find out :)

 

Thanks for the extra information... As I said, I'm looking at it but don't think I can use it right now. I was guessing 256 characters or multiples similar to Excel cells... Are field notes in the realm of Premium Members? I don't remember ever seeing a place for them for us Basics, but that will change for me soon... just a few more bills to pay.

 

Do you know what it takes to create a POI file for Garmin?

 

Doug 7rxc

Link to comment

Just wanted to mention that I got this to work in vi/vim. If you have no idea what that or even what unix is just ignore this and continue to use something like notepad+ as suggested above.

 

  1. # vim FieldNotes.txt
  2. :set fileencoding=UCS-2LE
  3. :set fileencoding (To make sure it shows as UCS-2LE not utf-8, etc)
  4. i (To begin insert mode.)
  5. <GC code>,<date year-month-day>T<time>Z,<log type>,"<message>"
  6. esc (To exit insert mode.)
  7. :wq (Save and quit.)
  8. Go to http://www.geocaching.com/my/uploadfieldnotes.aspx
  9. Select your file and upload.

 

Now that I've read the rest of the thread I see that ascii files are accepted now, but strangely when I didn't have my encoding correct the site didn't accept the upload.

Edited by Krydos
Link to comment

I haven't seen conversion via iconv(1) mentioned here, so here we go, for all those Unix/NetBSD/MacOS or even Linux users:

% echo GC123 | iconv -f ascii -t UTF-16LE > /tmp/x
% hexdump -C /tmp/x
00000000  47 00 43 00 31 00 32 00  33 00 0a 00              |G.C.1.2.3...|
0000000c

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...
×
×
  • Create New...