GPX files, GSAK and Benchmarks

[Papa-Bear-NYC]

I have been using GSAK heavily lately for both tracking my finds and to generate the data for Google Maps on my new benchmarking web site. I have a few questions/issues:

 

1)Is there anyway to extend GSAKs capabilities? There are several important items I'd like to see: Vertical and Horizoontal order, Scaled vs. adjusted coordinates, marker type, maybe a few more. These are all on the datasheet which is in side GSAK but there doesn't seem to be a way to display the data except to dpouble click and see the whole thing.

 

2) I use "bmppx" to convert NGS county downloads (which are really just a pile of datasheets) to a GPX file. Is there an alternative which would parse more of these items from the datasheets?

 

3) I'm going to look into the shape files that monkeykat mentioned in another thread. What would really be nice would be if the data available from those files could be exposed by GSAK.

 

Thanks for any ideas.

Papa Bear

Edited April 20, 2007 by Papa-Bear-NYC

[+Black Dog Trackers]

Papa-Bear-NYC -

 

That is sort of a complicated question because, while it would probably be fairly easy for GSAK be changed to parse in more data elements, it may be beyond the current scope of Google Maps to show them. Filtering by them is surely something that GSAK does already, but I'm thinking you want them shown somehow on Google Maps. There would have to be something in the GPX data elements list and also something that Google Earth would show. So there's really 3 aspects that have to be compatible with each other on more data elements - GSAK, GPX, and Google Maps. I think the main stumbling block would be Google Maps at this point.

 

As you may know, I have written a script (not a self-contained program, but a script that relies on another program being present) to put locations on Google Earth. I cannot get it to provide more than one link in the little popup windows. I wanted to make 2 links - one to the NGS datasheet and one to the GC datasheet, but it will only do one link, so one has to resort to running the script with the 2 different options and load them both into Google Earth and then switch one or the other off with the control pane or whatever it's called. (I need to post a correction to that script in the other topic.) Anyway, I ran into this scope problem with Google Earth, and figure there are similar limitations to Google Maps. I tried to put 2 links in each PID's GPX output, but Google Earth just overlays the second one over the first one.

 

By the way, I believe the author of GSAK is in the Geocaching forums, but I don't know if he ever posted to the benchmark hunting section.

 

What is it that you want Google Maps to do with these other elements? Or is that what you're asking for?

[Papa-Bear-NYC]

Papa-Bear-NYC -

 

That is sort of a complicated question because, while it would probably be fairly easy for GSAK be changed to parse in more data elements, it may be beyond the current scope of Google Maps to show them. Filtering by them is surely something that GSAK does already, but I'm thinking you want them shown somehow on Google Maps. There would have to be something in the GPX data elements list and also something that Google Earth would show. So there's really 3 aspects that have to be compatible with each other on more data elements - GSAK, GPX, and Google Maps. I think the main stumbling block would be Google Maps at this point.

 

As you may know, I have written a script (not a self-contained program, but a script that relies on another program being present) to put locations on Google Earth. I cannot get it to provide more than one link in the little popup windows. I wanted to make 2 links - one to the NGS datasheet and one to the GC datasheet, but it will only do one link, so one has to resort to running the script with the 2 different options and load them both into Google Earth and then switch one or the other off with the control pane or whatever it's called. (I need to post a correction to that script in the other topic.) Anyway, I ran into this scope problem with Google Earth, and figure there are similar limitations to Google Maps. I tried to put 2 links in each PID's GPX output, but Google Earth just overlays the second one over the first one.

 

By the way, I believe the author of GSAK is in the Geocaching forums, but I don't know if he ever posted to the benchmark hunting section.

 

What is it that you want Google Maps to do with these other elements? Or is that what you're asking for?

Thanks for the reply.

 

Actually I have already done the Google maps part. I simply generate data elements from an Excel spreadsheet (generated from a GSAK export) and using the GMaps API, incorporate them in the text that goes with a marker. Obviously just putting a point up only requires a Lat and Long, but when you click on it you want more information displayed. The added information doesn't affect the way GMaps works, but it gives the user potentially interesting information. It's nice to know it's a drill hole without having to read through the data sheet.

 

As of now, I took what GSAK had to offer for data items (plus what I added such as my find date) and put that in. If you check my application (here) you will see that clicking on a marker on the county map brings up a whole page of information for the station including a close-up map, link to NGS, GC etc. It's this station page that I'd like to enhance. But obviously with a gazillion stations, these pages have to be generated automatically. That works but I would like a little bit more information added to my template.

 

Another thing I could do is populate the county map by color showing horizontal and vertical stations. I use color now for my finds and not-finds, but it would be a simple change to use another criterion.

 

Just now I downloaded a shape file for Manhattan, and got the "ADJUSTED/SCALED" information and merged it into what I had from GSAK. That was a one-time pain but as a proof of concept it worked fine.

 

I label my marks as adjusted, scaled or corrected. Until this morning's excercise I just put a few in by hand and corrected a few using the GMaps draggable markers feature. But to go live I need an automated way to get that. For now, with the shape files addition, let's call it semi-automated.

 

I guess another way to state the question is how can I use GSAK as the single resource for my benchmark logging and to build the data for my web site? If I need 2 or 3 sources, fine ( as long as it's automated) but one is better than several.

 

Regards

Papa Bear

Edited April 20, 2007 by Papa-Bear-NYC

[+Klemmer]

I have been using GSAK heavily lately for both tracking my finds and to generate the data for Google Maps on my new benchmarking web site. I have a few questions/issues:

 

1)Is there anyway to extend GSAKs capabilities? There are several important items I'd like to see: Vertical and Horizoontal order, Scaled vs. adjusted coordinates, marker type, maybe a few more. These are all on the datasheet which is in side GSAK but there doesn't seem to be a way to display the data except to dpouble click and see the whole thing.

 

2) I use "bmppx" to convert NGS county downloads (which are really just a pile of datasheets) to a GPX file. Is there an alternative which would parse more of these items from the datasheets?

Papa-Bear:

 

I've done some work on extending some of the GASK Benchmark data capabilities. I use it quite bit, in the same way you mentioned. It's been a year or so, but here's what I remember:

Using the macro capbility built-in, it is possible to pull in data from the full datasheet, which is loaded into GSAK as the "long description". Edit a record, and at the bottom of the edit screen you will find a tab for "long description". I think used the text search and parsing capabilities of the macro language. At the time, I was writing a macro ro fix the "UNK" placed date (related to last year's contest). Here is the macro, on the very nice GSAK forum, which I also recommend. If anythink I wrote is worthwhile, feel free to use it. I seem to remember that there were maybe two GSAK database variable you could stick data into, User Data and User Data 2. If I can help, just yell. Sounds like a good project!

 

Klemmer

[Papa-Bear-NYC]

Papa-Bear:

 

I've done some work on extending some of the GASK Benchmark data capabilities. I use it quite bit, in the same way you mentioned. It's been a year or so, but here's what I remember:

Using the macro capbility built-in, it is possible to pull in data from the full datasheet, which is loaded into GSAK as the "long description". Edit a record, and at the bottom of the edit screen you will find a tab for "long description". I think used the text search and parsing capabilities of the macro language. At the time, I was writing a macro ro fix the "UNK" placed date (related to last year's contest). Here is the macro, on the very nice GSAK forum, which I also recommend. If anythink I wrote is worthwhile, feel free to use it. I seem to remember that there were maybe two GSAK database variable you could stick data into, User Data and User Data 2. If I can help, just yell. Sounds like a good project!

 

Klemmer

Thanks. I'll look at it.

 

Unfortunately I am already using User Data and User Data 2, but I'll see what I can do.

 

Pb

[+Klemmer]

Papa-Bear:

I took a quick look at what I did last year, and you can modify (re-use) any database variable. I think you have to stay with the same variable type (numeric, text, date ...). So you are not limited to just the two User Data variable. I'll look at it more when I have a chance.

 

I think I better update to the newest version of GSAK while I'm at it.....

[Papa-Bear-NYC]

Thanks Klemmer and BDT and monkeykat

 

I also looked at which variables you could change and I decided I can use things like "State" or "Country" which are somewhat unneccessary for my own use (since my databases are each for just one county anyway). I also had a short diecussion with BDT and here is a summary of my thinking. Things look very hopeful.

 

The GSAK Macro language looks quite interesting and capable. It looks like I can do two things with it: 1) expose any of the data from the NGS datasheet I'm looking for in GSAK so I can use GSAK as my sole resource for maintaining my bench mark searches, and 2) it could actually be used to write the JavaScript arrays that I use to "feed" GoogleMaps and which I also use for the non-GoogleMaps part of my web application. If I only wanted to get the data in a form for Google Maps I might pursue another approach, but 1) using GSAK for my own benchmark tracking and 2) having more data items to use on the non-GoogleMaps part of my web site makes this approach more attractive.

 

JavaScript is the language the GoogleMaps API uses and although you can also import data in XML or KML format (which is sort of like what a GPX file is) using JavaScript arrays is the most straightforward method (or maybe I should say the easiest for me to understand method) that allows maximum access to individual data elements. In other words with an XML (or GPX) file you can just "drop" the file into Google Maps (or Google Earth), but with the JavaScript arrays you can pick apart the data and create specific links, text balloons, ets. or, as I do, whole web pages. So the point is it's not just GoogleMaps but it's the total web site design and implementation that is important for what I'm doing.

 

Right now my process is something like this:

 

1) download county archive from NGS

2) run it through bmgpx

3) Load it into GSAK

4) add in any FOUNDs or DNFs and any special notes (like "Found only the stem") for myself. For a virgin county skip this step

 

5) Export specific data elements to a CSV file

6) use Excel to turn the CSV file into a JavaSxript array used by my web app

 

Normally the first 4 steps would be part of my work flow anyway since I have decided that GSAK will be my primary means to track my benchmarking. Before I had GSAK, I used Excel spreadsheets, one for each county (cumbersome to say the least). GSAK is much nicer - as long as I can get at what I need. Now using the GSAK Macro language, I can skip steps 5) and 6) which were a bit manual and added more steps to break and more things to maintain.

 

[geek alert ON]

A typical JavaScript array, generated automatically (now by Excel, in the future hopefully by GSAK), might look like this (these are actually for my FOUND stations in Bergen county):

 

var County_Found_Markers = [

{"station":"KU1641", "lat":"40.8796540", "lng":"-73.9490578", "scaled":2, "status":"Found", "date":"9/30/2006",

"designation":"V 40", "condition":"Good", "type":"CGS", "file":true },

 

{"station":"KU1645", "lat":"40.8520931", "lng":"-73.9609294", "scaled":2, "status":"Found", "date":"9/30/2006",

"designation":"PALISADES RM 2", "condition":"Poor: disk cut from stem", "type":"CGS", "file":true },

 

{"station":"KU3879", "lat":"40.87502", "lng":"-73.947983", "status":"Found", "date":"12/3/2006",

"designation":"HAN", "condition":"Good", "type":"CGS", "file":true , "file":true },

 

{"station":"KU3881", "lat":"40.865258", "lng":"-73.952326", "status":"Found", "date":"9/26/2006",

"designation":"LIN 1930", "condition":"Poor: just drill hole", "type":"CGS", "file":true },

 

{"station":"KU3890", "lat":"40.852362", "lng":"-73.960799", "status":"Found", "date":"9/30/2006",

"designation":"PALISADES", "condition":"Good", "type":"CGS", "file":true },

 

{"station":"KU4042", "lat":"40.883762", "lng":"-73.943251", "status":"Found", "date":"9/26/2007",

"designation":"DYCK 1930", "condition":"Poor: just a drill hole", "type":"CGS", "file":true },

 

{"station":"KU4044", "lat":"40.903954", "lng":"-73.934555", "status":"Found", "date":"12/3/2006",

"designation":"LINDY", "condition":"Good", "type":"CGS", "file":true },

 

{"station":"KU4045", "lat":"40.902038", "lng":"-73.934065", "status":"Found", "date":"9/26/2006",

"designation":"WOOD", "condition":"Poor: just drill hole", "type":"CGS", "file":true },

 

{"station":"KU4047", "lat":"40.981683", "lng":"-73.90587", "status":"Found", "date":"12/16/2006",

"designation":"FOREST", "condition":"Good", "type":"CGS", "file":true },

 

{"station":"KU4050", "lat":"40.976619", "lng":"-73.909138", "status":"Found", "date":"12/9/2006",

"designation":"BURNETT", "condition":"Good", "type":"CGS", "file":true },

 

{"station":"KU4055", "lat":"40.943697", "lng":"-73.92087", "status":"Found", "date":"12/9/2006",

"designation":"ALPINE", "condition":"Good", "type":"CGS", "file":true },

 

{"station":"KU4057", "lat":"40.928364", "lng":"-73.926532", "status":"Found", "date":"12/9/2006",

"designation":"INDIAN HEAD 1921", "condition":"Good", "type":"NY6820", "file":true },

 

{"station":"KU4058", "lat":"40.922731", "lng":"-73.926033", "status":"Found", "date":"9/26/2006",

"designation":"EGG 2", "condition":"Poor: just drill hole", "type":"CGS", "file":true },

 

{"station":"KU4062", "lat":"40.997327", "lng":"-73.902715", "status":"Found", "date":"12/16/2006",

"designation":"BOUNDARY LINE NJ NY", "condition":"Good", "type":"CGS", "file":true },

 

{"station":"KU4063", "lat":"40.985988", "lng":"-73.906492", "status":"Found", "date":"12/9/2006",

"designation":"ROON", "condition":"Good", "type":"CGS", "file":true },

 

{"station":"KU5154", "lat":"40.881288", "lng":"-73.943895", "status":"Found", "date":"9/30/2006",

"designation":"2 S 1", "condition":"Good", "type":"NJGS", "file":true }

]

 

You can see the resemblance to other formats using tagged data items (GPX, XML, etc.) and I assume BDT's GAWK program produces something functionally similar. For the data I am looking for (SCALED/ADJUSTED, Marker Type, etc.) I can just add a few more tagged items.

[geek alert OFF]

 

Thanks again for BDT's monkeykat's and Klemmer's suggestions and ideas. They have been quite helpful.

 

Regards

Papa Bear

Edited April 21, 2007 by Papa-Bear-NYC

[+Klemmer]

[geek alert ON]

If you would like to experiment with GSAK, without getting too deep into it's macro language, here is a little macro that will identify benchmarks in GSAK as one of the following:

- Classic Adjusted

- GPS adjusted

- CORS related

- Scaled

Then add the notation to the User Data column, then sort by User Data and Waypoint name. You will need to have the User Data Coumn visible in your screen View to make it useful. You can copy & paste to the GSAK macro editor, then save it to your computer. If anyone would like an email of the files instead, email me thru GC.com, with your real email address (can't send attached files tru GC).

 

# For Benchmark Databases, this macro will:
# 1)  Identify records as scaled, classic adjusted, GPS adjusted, or CORS related records. 
# 2)  Place appropriate notation in the User Data column
# 3) Sort by User Data column, then Waypoimt name and go to the top of the file
VSUB Status=off
$x = 0
Goto Position=Top
$exp2 = "- 6 seconds"
$exp3 = "classical geodetic methods"
$exp4="horizontal coordinates were established by GPS observations"
$exp5="CORS"
While .not. $_EOL
  $x = $x + 1
  IF RegExData($exp2,$d_LongDescription,1)="- 6 seconds"
  $d_UserData="Scaled" 
  ENDIF
  IF RegExData($exp3,$d_LongDescription,1)="classical geodetic methods"
  $d_UserData="Classic Adjusted"
  ENDIF
  IF RegExData($exp4,$d_LongDescription,1)="horizontal coordinates were established by GPS observations"
  $d_UserData="GPS Adjusted"
  ENDIF
  IF RegExData($exp5,$d_LongDescription,1)="CORS"
  $d_UserData="CORS related"
  ENDIF
SHOWSTATUS Msg=  Record in process:  $x
Goto Position=Next
EndWhile
Sort by="UserData;SmartName"
GoTo Position=Top
Pause Msg=$x Records Processed

 

Here is a little "helper" application I made last year to clear both User Data columns. Useful during testing.

# This macro will clear User Data and Used Data2
$x = 0
Goto Position=Top
While .not. $_EOL
  $x = $x + 1
  $d_UserData =""
  $d_User2 = ""
  SHOWSTATUS Msg= $x
  Goto Position=Next
EndWhile
# Show total number records
Pause Msg=$x

[geek alert OFF]

[Papa-Bear-NYC]

[geek alert ON]

If you would like to experiment with GSAK, without getting too deep into it's macro language, here is a little macro that will identify benchmarks in GSAK as one of the following:

- Classic Adjusted

- GPS adjusted

- CORS related

- Scaled

Then add the notation to the User Data column, then sort by User Data and Waypoint name. You will need to have the User Data Coumn visible in your screen View to make it useful. You can copy & paste to the GSAK macro editor, then save it to your computer. If anyone would like an email of the files instead, email me thru GC.com, with your real email address (can't send attached files tru GC).

 

 

Thanks, I'll look at that. Meanwhile after spending some out of doors time, I managed a Macro to pull the "ADJUSTED/SCALED" item off the data sheet, and pull out the Marker Type. Those higher and higher levels of "adjustment" may be a bit over the top.

 

Then I ran it on a few counties. Here's a screen shot for Westchester county (cropped);

 

 

A good days work - unfortunately my out door work was a bunch of DNFs.

Edited April 21, 2007 by Papa-Bear-NYC

[+ClydeE]

I am not 100% sure it will help you but this macro by Markp99 does a lot of cool work with the GSAK macro language and using Javascript arrays for the generation of Google Maps. I guess it wouldn't hurt to take a look.

Edited April 22, 2007 by ClydeE

[Papa-Bear-NYC]

I am not 100% sure it will help you but this macro by Markp99 does a lot of cool work with the GSAK macro language and using Javascript arrays for the generation of Google Maps. I guess it wouldn't hurt to take a look.

Thanks Clyde, That looks extremely helpful. I'll be studying that.

[+Doc Geo]

You know the source code (written in C) is provided by the author, right?

So if you know C you can rewrite it to make sure and grab the datasheet

fields you want and put them in the GPX fields that are available to ensure

you can get to them in GSAK.

 

Or if you can wait for what might seem like forever, I plan to do this

someday myself...

 

Barry

aka Doc Geo