Jump to content
Sign in to follow this  
Followers 2
Black Dog Trackers

Google Earth and Benchmarks

Recommended Posts

There's a topic on Google Maps already, this topic is about different ways of using Google Earth to view benchmarks and their locations.

 

Google Earth reads benchmark location files that reside on your computer. It will read .loc files, .gpx files, and .kml files, among others.

 

I tried .gpx files before but .kml files have a lot more capability, so I switched my experiments to .kml files. This time, I wrote an awk script (given below) to read an NGS .dat file to produce a .kml file.

 

I wanted to get as much information as possible from an NGS .dat file into the popup balloons of Google Earth location icons. Here is an example of a nice popup ballon's contents with lots of RMs and an AZ mark in the box score as well as several history items:

 

HU1428

PID: HU1428

Designation: COLONY

Links: Geocaching NGS

State / County: DE / SUSSEX

USGS Quad: MILLSBORO

Horizontal position type: ADJUSTED

Elevation (ft.): 36.

3.780 METERS dddmm=13953 AF7887 COLONY RM 4

42.977 METERS dddmm=20017 DC7942 COLONY RM 2

5.709 METERS dddmm=22513 AF7889 COLONY RM 5

60.183 METERS dddmm=31356 DC7943 COLONY RM 3

441.692 METERS dddmmss.s=3362011.3 AF7888 COLONY AZ MK 2

MARKER: TRIANGULATION STATION DISK

SETTING: SET IN TOP OF CONCRETE MONUMENT

STAMPING: COLONY 1932

MARK LOGO: CGS

MAGNETIC: OTHER; SEE DESCRIPTION

1932: MONUMENTED CGS

1943: GOOD CGS

1961: GOOD CGS

1968: GOOD LOCENG

1969: GOOD CGS

1982: GOOD NGS

I also wanted to vary the labeling of the icons, so I made a command line switch for different labeling of icons. The default version uses the PID as the label, of course. Another version uses the designation to label the icons. This type of labeling is interesting in that the relationships between neighboring benchmarks is clearly seen with names like J 48, J 49, J 50, etc. interspersed with designations like GREEN, ALICE, 53 PRR, etc. Since some of the .dat files I was experimenting with were in mountains, I also made a switch for labeling by elevation. I also made a switch for labeling benchmark icons by the oldest date. Again, the relationships between groups of benchmarks was made visually clear while looking at a map. The last switch I made was to label the benchmarks by monumentation code. For both this one and the date one, it's interesting to search around visually on Google Earth to find the oddities - the oldest, the newest, and the unusual types in the county.

 

Google Earth shows county borders if you click that feature on in the sidebar, so it's pretty good for finding out-of-county errors.

 

In another dimension from the labeling switch, I color coded the icons by SCALED (yellow), HD_HELD (green), and other - ADJUSTED (red).

 

If you'd like to see how this all looks, I have uploaded the 5 versions of icon labeling for the benchmarks of Sussex county, Delaware here (click the link to the zipfile of kml files at the bottom of the page). After unzipping, you can tell Google Earth to open any one of these files to see how it looks.

 

The script below uses the free GAWK program available here or here.

 

Here is the script. You could call it dat2kml.awk and then run it in a DOS window like this:

gawk -f dat2kml.awk de005.dat

and that will produce de005.kml which is ready to open in Google Earth.

# Usage:
# (in each case, the program makes datfile.kml from datfile.dat)

# gawk -f thisfile.awk datfile.dat
# This default version uses the PID to label each icon

# gawk -f label=des thisfile.awk datfile.dat
# This version uses the designation, instead of the PID, to label each icon

# gawk -f label=date thisfile.awk datfile.dat
# This version uses the oldest (monumentation) date to label each icon

# gawk -f label=elev thisfile.awk datfile.dat
# This version uses the elevation value to label each icon

# gawk -f label=code thisfile.awk datfile.dat
# This version uses the monumentation code to label each icon

{
if (NR==1) {
if (label=="DATE") label="date"
if (label=="DES") label="des"
if (label=="ELEV") label="elev"
if (label=="CODE") label="code"
if (label!="") {
	if ((label!="date")&&(label!="des")&&(label!="elev")&&(label!="code")) {
		print "Incorrect label= value.  Using PIDs as labels."
		print "Valid label= values are: des date elev code"
		}
	}
gcu="http://www.geocaching.com/mark/details.aspx?PID="
ngs="http://www.ngs.noaa.gov/cgi-bin/ds_mark.prl?PidBox="
name= substr(FILENAME,1,(length(FILENAME)-4)) ".kml"
print "<kml>" > name
printf("<Document>\n") > name  # newlines just to make the kml file easier to look at
print "<Style id=\"adj\">" > name
print "<IconStyle><color>#ff4444ff</color></IconStyle>" > name
print "</Style>" > name
print "<Style id=\"nonadj\">" > name
print "<IconStyle><color>#ffeeeeff</color></IconStyle>" > name
print "</Style>" > name
print "<Style id=\"hand\">" > name
print "<IconStyle><color>#ff44ff44</color></IconStyle>" > name
print "</Style>" > name
}
else {
while (substr($0,1,1)==" ") {
	dcount++
	if (substr($0,8,13)=="  DESIGNATION") {
		# set up a new placemark
		printf("<Placemark><description>\n") > name
		printf("<![CDATA[\n") > name  # start html coding enabling
		pid=substr($0,2,6)
		printf("<u>PID</u>: %s\n",pid) > name
		oldest=""
		# get designation
		des=substr($0,25,length($0)-25+1)
		printf("<br><u>Designation:</u> %s\n",des) > name
		# print the links now
		printf("<br><u>Links</u>: <a href=\"%s%s\">Geocaching</a>\n",gcu,pid) > name
		printf(" <a href=\"%s%s\">NGS</a>\n",ngs,pid) > name
		}
	else if (substr($0,8,14)=="  STATE/COUNTY") {  # get state/county
		state=substr($0,25,length($0)-25+1)
		sub(/\//, " / ", state)  # add spaces
		printf("<br><u>State / County</u>: %s\n",state) > name
		}
	else if (substr($0,8,6)=="  USGS") {  # get quad
		qd=substr($0,25,length($0)-25+1-7)
		printf("<br><u>USGS Quad</u>: %s\n",qd) > name
		if ((quad!="")&&(quad!=qd)) prt=0
		}
	else if (substr($0,8,1)=="_") {  # detailed specifications
		# ignore some items that don't help in finding
		if (substr($0,9,13)=="U.S. NATIONAL") {}
		else if (substr($0,9,9)=="STABILITY") {}
		else if (substr($0,9,9)=="SATELLITE") {}
		else {
			stuff=substr($0,9,length($0)-9+1)
			colon=index(stuff,":")
			equals=index(stuff,"=")
			kind=substr(stuff,1,colon-1)
			start= equals > colon ? equals : colon
			value=substr(stuff,start+2,length(stuff)-start)
			printf("<br><u>%s</u>: %s\n",kind,value) > name
			if (substr($0,9,6)=="MARKER") hippo=substr($0,17,2)
			}
		}
	else if (substr($0,8,9)=="  HISTORY") {  # print history items
		date=substr($0,24,4)
		if (date=="Date") next
		if (oldest=="") oldest=date
		condition=substr($0,33,15)
		reporter=substr($0,50,length($0)-50+1)
		printf("<br><u>%s</u>: %s %s\n",date,condition,reporter) > name
		}
	else if (substr($0,8,1)=="|") {  # a box score
		if (start=index($0," METERS ")>0) {  # a nearby box score item
			sub(/  *\|/, "", $0)
			split(substr($0,10,length($0)-10+1),xxx,/   */)
			split(xxx[2],yyy," ")
			if (length(yyy[3])==5) { precision="dddmm=" }
			else { precision="dddmmss.s=" }
			printf("<br>%9s %s %s%s %s\n",yyy[1],yyy[2],precision,yyy[3],xxx[1]) > name
			}
		}
	else if (substr($0,33,8)=="*CURRENT") {
		dcount=0  # start a count to the coordinates line
		}
	else if (dcount==2) {  # get pid,lat,lon,pos
		split(substr($0,25,14),deg," ")
		lat=deg[1]+deg[2]/60+deg[3]/60/60
		lon=substr($0,46,15)
		split(substr($0,46,15),deg," ")
		lon=(deg[1]+deg[2]/60+deg[3]/60/60)*(-1)
		pos=substr($0,69,length($0)-69+1)
		printf("<br><u>Horizontal position type</u>: %s\n",pos) > name
		}
	else if (dcount==3) {  # get the elevation if there is one
		elev=substr($0,50,8)
		if ((elev=="") || (elev=="		")) next
		printf("<br><u>Elevation (ft.)</u>: %s\n",elev) > name
		}
	else if (substr($0,1,2)==" *") {  # direct downloads have this at the end
		done=1
		exit
		}
	else {}
	next
	}
	endit(name,pid,des,lon,lat,label,pos,oldest,elev,hippo)
}
}
END {
# finish off the last PID in the .dat file
if (done!=1) endit(name,pid,des,lon,lat,label,pos,oldest,elev,hippo)
# finish off the .kml file
printf("</Document>\n") > name
print "</kml>" > name
}
function endit(name,pid,des,lon,lat,label,pos,oldest,elev,hippo) {
# end description
printf("]]>\n") > name  # end html coding enabling
printf("</description>\n") > name
if (pos=="SCALED	") { print "<styleUrl>#nonadj</styleUrl>" > name }
else if ((pos=="HD_HELD1  ") || (pos=="HD_HELD2  ")) {
	print "<styleUrl>#hand</styleUrl>" > name
	}
else { print "<styleUrl>#adj</styleUrl>" > name }
if (label=="des") { printf("<name>%s</name>\n",des) > name }
else if (label=="date") { printf("<name>%s</name>\n",oldest) > name }
else if (label=="elev") {
	elevv = elev=="		" ? "blank" : elev
	printf("<name>%s</name>\n",elevv) > name
	}
else if (label=="code") { printf("<name>%s</name>\n",hippo) > name }
else { printf("<name>%s</name>\n",pid) > name }
# output the coordinates
printf("<Point>\n<coordinates>\n") > name
printf("%.8f, %.8f\n",lon,lat) > name
printf("</coordinates></Point>\n") > name
# finish off this placemark
printf("</Placemark>\n") > name
}

Share this post


Link to post

There's a topic on Google Maps already, this topic is about different ways of using Google Earth to view benchmarks and their locations.

 

Google Earth reads benchmark location files that reside on your computer. It will read .loc files, .gpx files, and .kml files, among others.

 

I tried .gpx files before but .kml files have a lot more capability, so I switched my experiments to .kml files. This time, I wrote an awk script (given below) to read an NGS .dat file to produce a .kml file.

 

I wanted to get as much information as possible from an NGS .dat file into the popup balloons of Google Earth location icons. Here is an example of a nice popup ballon's contents with lots of RMs and an AZ mark in the box score as well as several history items:

 

HU1428

PID: HU1428

Designation: COLONY

Links: Geocaching NGS

State / County: DE / SUSSEX

USGS Quad: MILLSBORO

Horizontal position type: ADJUSTED

Elevation (ft.): 36.

3.780 METERS dddmm=13953 AF7887 COLONY RM 4

42.977 METERS dddmm=20017 DC7942 COLONY RM 2

5.709 METERS dddmm=22513 AF7889 COLONY RM 5

60.183 METERS dddmm=31356 DC7943 COLONY RM 3

441.692 METERS dddmmss.s=3362011.3 AF7888 COLONY AZ MK 2

MARKER: TRIANGULATION STATION DISK

SETTING: SET IN TOP OF CONCRETE MONUMENT

STAMPING: COLONY 1932

MARK LOGO: CGS

MAGNETIC: OTHER; SEE DESCRIPTION

1932: MONUMENTED CGS

1943: GOOD CGS

1961: GOOD CGS

1968: GOOD LOCENG

1969: GOOD CGS

1982: GOOD NGS

I also wanted to vary the labeling of the icons, so I made a command line switch for different labeling of icons. The default version uses the PID as the label, of course. Another version uses the designation to label the icons. This type of labeling is interesting in that the relationships between neighboring benchmarks is clearly seen with names like J 48, J 49, J 50, etc. interspersed with designations like GREEN, ALICE, 53 PRR, etc. Since some of the .dat files I was experimenting with were in mountains, I also made a switch for labeling by elevation. I also made a switch for labeling benchmark icons by the oldest date. Again, the relationships between groups of benchmarks was made visually clear while looking at a map. The last switch I made was to label the benchmarks by monumentation code. For both this one and the date one, it's interesting to search around visually on Google Earth to find the oddities - the oldest, the newest, and the unusual types in the county.

 

Google Earth shows county borders if you click that feature on in the sidebar, so it's pretty good for finding out-of-county errors.

 

In another dimension from the labeling switch, I color coded the icons by SCALED (yellow), HD_HELD (green), and other - ADJUSTED (red).

 

If you'd like to see how this all looks, I have uploaded the 5 versions of icon labeling for the benchmarks of Sussex county, Delaware here (click the link to the zipfile of kml files at the bottom of the page). After unzipping, you can tell Google Earth to open any one of these files to see how it looks.

 

The script below uses the free GAWK program available here or here.

 

Here is the script. You could call it dat2kml.awk and then run it in a DOS window like this:

gawk -f dat2kml.awk de005.dat

and that will produce de005.kml which is ready to open in Google Earth.

# Usage:
# (in each case, the program makes datfile.kml from datfile.dat)

# gawk -f thisfile.awk datfile.dat
# This default version uses the PID to label each icon

# gawk -f label=des thisfile.awk datfile.dat
# This version uses the designation, instead of the PID, to label each icon

# gawk -f label=date thisfile.awk datfile.dat
# This version uses the oldest (monumentation) date to label each icon

# gawk -f label=elev thisfile.awk datfile.dat
# This version uses the elevation value to label each icon

# gawk -f label=code thisfile.awk datfile.dat
# This version uses the monumentation code to label each icon

{
if (NR==1) {
if (label=="DATE") label="date"
if (label=="DES") label="des"
if (label=="ELEV") label="elev"
if (label=="CODE") label="code"
if (label!="") {
	if ((label!="date")&&(label!="des")&&(label!="elev")&&(label!="code")) {
		print "Incorrect label= value.  Using PIDs as labels."
		print "Valid label= values are: des date elev code"
		}
	}
gcu="http://www.geocaching.com/mark/details.aspx?PID="
ngs="http://www.ngs.noaa.gov/cgi-bin/ds_mark.prl?PidBox="
name= substr(FILENAME,1,(length(FILENAME)-4)) ".kml"
print "<kml>" > name
printf("<Document>\n") > name  # newlines just to make the kml file easier to look at
print "<Style id=\"adj\">" > name
print "<IconStyle><color>#ff4444ff</color></IconStyle>" > name
print "</Style>" > name
print "<Style id=\"nonadj\">" > name
print "<IconStyle><color>#ffeeeeff</color></IconStyle>" > name
print "</Style>" > name
print "<Style id=\"hand\">" > name
print "<IconStyle><color>#ff44ff44</color></IconStyle>" > name
print "</Style>" > name
}
else {
while (substr($0,1,1)==" ") {
	dcount++
	if (substr($0,8,13)=="  DESIGNATION") {
		# set up a new placemark
		printf("<Placemark><description>\n") > name
		printf("<![CDATA[\n") > name  # start html coding enabling
		pid=substr($0,2,6)
		printf("<u>PID</u>: %s\n",pid) > name
		oldest=""
		# get designation
		des=substr($0,25,length($0)-25+1)
		printf("<br><u>Designation:</u> %s\n",des) > name
		# print the links now
		printf("<br><u>Links</u>: <a href=\"%s%s\">Geocaching</a>\n",gcu,pid) > name
		printf(" <a href=\"%s%s\">NGS</a>\n",ngs,pid) > name
		}
	else if (substr($0,8,14)=="  STATE/COUNTY") {  # get state/county
		state=substr($0,25,length($0)-25+1)
		sub(/\//, " / ", state)  # add spaces
		printf("<br><u>State / County</u>: %s\n",state) > name
		}
	else if (substr($0,8,6)=="  USGS") {  # get quad
		qd=substr($0,25,length($0)-25+1-7)
		printf("<br><u>USGS Quad</u>: %s\n",qd) > name
		if ((quad!="")&&(quad!=qd)) prt=0
		}
	else if (substr($0,8,1)=="_") {  # detailed specifications
		# ignore some items that don't help in finding
		if (substr($0,9,13)=="U.S. NATIONAL") {}
		else if (substr($0,9,9)=="STABILITY") {}
		else if (substr($0,9,9)=="SATELLITE") {}
		else {
			stuff=substr($0,9,length($0)-9+1)
			colon=index(stuff,":")
			equals=index(stuff,"=")
			kind=substr(stuff,1,colon-1)
			start= equals > colon ? equals : colon
			value=substr(stuff,start+2,length(stuff)-start)
			printf("<br><u>%s</u>: %s\n",kind,value) > name
			if (substr($0,9,6)=="MARKER") hippo=substr($0,17,2)
			}
		}
	else if (substr($0,8,9)=="  HISTORY") {  # print history items
		date=substr($0,24,4)
		if (date=="Date") next
		if (oldest=="") oldest=date
		condition=substr($0,33,15)
		reporter=substr($0,50,length($0)-50+1)
		printf("<br><u>%s</u>: %s %s\n",date,condition,reporter) > name
		}
	else if (substr($0,8,1)=="|") {  # a box score
		if (start=index($0," METERS ")>0) {  # a nearby box score item
			sub(/  *\|/, "", $0)
			split(substr($0,10,length($0)-10+1),xxx,/   */)
			split(xxx[2],yyy," ")
			if (length(yyy[3])==5) { precision="dddmm=" }
			else { precision="dddmmss.s=" }
			printf("<br>%9s %s %s%s %s\n",yyy[1],yyy[2],precision,yyy[3],xxx[1]) > name
			}
		}
	else if (substr($0,33,8)=="*CURRENT") {
		dcount=0  # start a count to the coordinates line
		}
	else if (dcount==2) {  # get pid,lat,lon,pos
		split(substr($0,25,14),deg," ")
		lat=deg[1]+deg[2]/60+deg[3]/60/60
		lon=substr($0,46,15)
		split(substr($0,46,15),deg," ")
		lon=(deg[1]+deg[2]/60+deg[3]/60/60)*(-1)
		pos=substr($0,69,length($0)-69+1)
		printf("<br><u>Horizontal position type</u>: %s\n",pos) > name
		}
	else if (dcount==3) {  # get the elevation if there is one
		elev=substr($0,50,8)
		if ((elev=="") || (elev=="		")) next
		printf("<br><u>Elevation (ft.)</u>: %s\n",elev) > name
		}
	else if (substr($0,1,2)==" *") {  # direct downloads have this at the end
		done=1
		exit
		}
	else {}
	next
	}
	endit(name,pid,des,lon,lat,label,pos,oldest,elev,hippo)
}
}
END {
# finish off the last PID in the .dat file
if (done!=1) endit(name,pid,des,lon,lat,label,pos,oldest,elev,hippo)
# finish off the .kml file
printf("</Document>\n") > name
print "</kml>" > name
}
function endit(name,pid,des,lon,lat,label,pos,oldest,elev,hippo) {
# end description
printf("]]>\n") > name  # end html coding enabling
printf("</description>\n") > name
if (pos=="SCALED	") { print "<styleUrl>#nonadj</styleUrl>" > name }
else if ((pos=="HD_HELD1  ") || (pos=="HD_HELD2  ")) {
	print "<styleUrl>#hand</styleUrl>" > name
	}
else { print "<styleUrl>#adj</styleUrl>" > name }
if (label=="des") { printf("<name>%s</name>\n",des) > name }
else if (label=="date") { printf("<name>%s</name>\n",oldest) > name }
else if (label=="elev") {
	elevv = elev=="		" ? "blank" : elev
	printf("<name>%s</name>\n",elevv) > name
	}
else if (label=="code") { printf("<name>%s</name>\n",hippo) > name }
else { printf("<name>%s</name>\n",pid) > name }
# output the coordinates
printf("<Point>\n<coordinates>\n") > name
printf("%.8f, %.8f\n",lon,lat) > name
printf("</coordinates></Point>\n") > name
# finish off this placemark
printf("</Placemark>\n") > name
}

 

BlackDog,

 

I am very interested in your script, but I am not used to these types of programming items. I went to your links, but did not even understand what to download. Is this or can this be as easy as using BMGPX.exe? I use that all the time, but as you said there are much more capabilities in the kml files.

 

Thanks for any further help,

CallawayMT

Share this post


Link to post

CallawayMT -

 

Sorry about that. I'm used to getting programs and files out of .zip files and running stuff in a DOS window so I tend to forget how that's not a usual thing to be doing.

 

Yes, it could be as easy as using BMGPX but I misplaced my C compiler long ago. My old hard drive died and the C compiler was on 5" floppies! I just use scripts these days. :blink: Hopefully someone will take some of this kind of thing and compile it. An awk script is pretty much the same as C or many other similar programming languages.

 

For now, the script will work OK and if anyone wants a step-by-step for getting gawk.exe from one of the links I posted, I could provide it here, but it sounds like you'd rather wait for a compiled version. :D

Share this post


Link to post

Anyone here know whether gawk should be compatible with Mac OS X awk? I've only used awk a couple of times in the past 20 years, so I don't know the syntax at all. I tried running the above script, but just wound up with a shell prompt again a couple of seconds later.

 

Patty

Share this post


Link to post

Callaway & BDT:

 

On BDT's previous GAWK project (the county chopper upper), after doing it the old "DOS" way (which I still remember), I did try it from inside Windows (using the "run..." function), and it worked fine. Let me set up the latest BDT "GAWK-ER", and I'll see if I can post the step by step without leaving the comfort of gates-ville (Windows). If have a MAC, good luck, you're on your own.

 

Klemmer

Share this post


Link to post

BDT:

Works well! I like it. I have soooooo many neat software tools now, I feel like I could spend all my time playing with BM maps, and forget to go out and find any! One comment: for my taste, a smaller icon would be better. If you try to change that in GE, you have to make all the icon & label properties the same first, which ruins the color coding. Is there a constant you can point me to in the script that sets the icon size? Looked, but.....

 

Callaway:

PM or GC me with your email address, and I'll send you back a couple files (gawk.exe and dat2kml1.awk)

When you get the files

1) save them in a new directory called "gawk" under your C drive (without the quotes)

2) copy your NGS dat file to the same directory (for this example, it will be ca059.dat, which is Orange County, CA - use your own NGS dat file)

3) click the start menu button. slide up and right some to the "run..." button (assumes Windows XP, should be similar in other vesions). Click the run... button.

4) type the following in the "open" box in the run command:

c:\gawk\gawk -f dat2kml1.awk ca059.dat

5) you will see a black DOS window for a few seconds, then your resulting ca059.kml file will appear in the same gawk directory

6) drag and drop the ca059file onto Google Earth. There you go!

 

You can modify the run... command line with BDT's other options (e.g. add "label=des" to the command line like he shows. no quotes, no spaces).

Edited by Klemmer & TeddyBearMama

Share this post


Link to post

I Googled some info on awk and gawk and it sounded like they weren't very different, and then I thought of something. I looked at my .awk file in vi and discovered that when I had saved it as a text-only file from MS Word, some carriage-return characters were added. So I deleted that file, created a new one by copying and pasting BDT's script directly into vi, and it worked great! Even the color coding for the adjusted and scaled coordinates showed up. I'll try playing with the other switches some other time.

 

So Mac users, have at it!

 

Thanks very much for this, BDT!

 

Patty

Share this post


Link to post

Wintertime -

 

Thanks very much for checking this out with a MAC ! :blink:

Good ol' vi - that's what I'm using too.

 

========================================

 

Klemmer & TeddyBearMama -

 

Thank you very much for checking this one out and assisting in getting it going for others!

 

(As you know, I fixed the chopper's precision problem but got into so many more additions it got a lot bigger - not done playing with that one yet.)

 

Anyway, the scaling thing goes in the 3 lines near the top of the script that look like this:

print "<IconStyle><color>#ff4444ff</color></IconStyle>" > name

to make them look like this:

print "<IconStyle><color>#ff4444ff</color><scale>.6</scale></IconStyle>" > name

That puts a .6 scale for that series (in this example, the ADJUSTED marks).

The default scale is 1.

Using .2 is waaaay too small.

Using 5 is rather shocking. :D

 

There is also <LabelStyle> that you can use to scale or color the labels for the icons.

You could add:

print "<LabelStyle><scale>.8</scale></LabelStyle>" > name

under each of the 3 <IconStyle>....</IconStyle> lines.

This scaling is very sensitive and I find that using .6 or less is nearly invisible.

The colors of the labels can be changed too.

 

This stuff is explained in the Google kml reference here.

 

I kinda crammed stuff together and it may be easier if you want to change this:

print "<IconStyle><color>#ff4444ff</color></IconStyle>" > name

to this:

print "<IconStyle>" > name

print "<color>#ff4444ff</color>" > name

print "</IconStyle>" > name

It doesn't matter either way.

Share this post


Link to post

BDT:

Got it. Thanks. Very flexible. Guess I'm learning more about scripting. I find the icon scale of .3 best for most things. As you said, very sensitive. Looking forward to playing with it more.

 

Patty:

Way to go! You sure know that Mac stuff.

 

Klemmer

Share this post


Link to post

Patty:

Way to go! You sure know that Mac stuff.

Well, as I mentioned, I don't really know awk. But I've been using vi every day for more than 20 years, so it was easy for me to check BDT's script with that and find the spurious newlines in my MS Word document, then use it to create a fresh, UNIX-friendly file. I don't often play directly (at the command-line level) with the UNIX commands in Mac OS X; I prefer to save those experiments for the host machine at my ISP, which I know is well protected! But it is handy sometimes to be able to run UNIX programs directly.

 

Patty

Share this post


Link to post

BDT or anyone, can you point me in the direction of what changes would be needed to the script in order to color code by last report date? Something like:

 

REPORT YEAR <= 1989 (green)

1990 <= REPORT YEAR <= 1999 (yellow)

2000 <= REPORT YEAR (red)

 

It would help me zero in on marks that haven't been recovered in a while. (Of course, I'd have to read the datasheet to find out whether the last report was FOUND or NOT FOUND, but this would be a start.)

 

Patty

Share this post


Link to post

I was playing with the program to see if I could use it to display what marks I have found so I decided to use this as an exercise in AWK programming first. Here is what I came up with.

 

In place of BDT's code to set the style for the type of mark (scaled, adjusted and handheld) I put code to set the icons to the three colors:

 

  print "<Style id=\"RedIcon\">" > name
print  "<LabelStyle><color>ff0000ff</color><scale>.8</scale></LabelStyle>" >name
print "<IconStyle><color>ff0000ff</color><scale>.6</scale>" > name
print "<Icon><href>http://maps.google.com/mapfiles/kml/pal3/icon61.png</href></Icon>" > name
print "</IconStyle>" > name
print "</Style>" > name

print "<Style id=\"YellowIcon\">" > name
print  "<LabelStyle><color>ff00ffff</color><scale>.8</scale></LabelStyle>" >name
print "<IconStyle><color>ff00ffff</color><scale>.6</scale>" > name
print "<Icon><href>http://maps.google.com/mapfiles/kml/pal3/icon61.png</href></Icon>" > name
print "</IconStyle>" > name
print "</Style>" > name

print "<Style id=\"GreenIcon\">" > name
print  "<LabelStyle><color>ff00ff00</color><scale>.8</scale></LabelStyle>" >name
print "<IconStyle><color>ff00ff00</color><scale>.6</scale>" > name
print "<Icon><href>http://maps.google.com/mapfiles/kml/pal3/icon61.png</href></Icon>" > name
print "</IconStyle>" > name
print "</Style>" > name

 

Near the bottom, where the style gets applied, I commented out the code for the type of mark and substituted the code for date coloring:

#	if (pos=="SCALED	") { print "<styleUrl>#nonadj</styleUrl>" > name }
#	else if ((pos=="HD_HELD1  ") || (pos=="HD_HELD2  ")) {
#		print "<styleUrl>#hand</styleUrl>" > name
#		}
#	else { print "<styleUrl>#adj</styleUrl>" > name }
  if (date > 1999) { print "<styleUrl>#RedIcon</styleUrl>" > name }
   else 
	if (date > 1989) { print "<styleUrl>#YellowIcon</styleUrl>" > name }
	 else { print "<styleUrl>#GreenIcon</styleUrl>" > name }   

 

Edited to fix a color error. Note, if you want the labels (PIDs) to be white, replace the color value in LableStyle with ffffffff (that's 8 "f"s), e.g. <color>ffffffff</color>

Edited by mloser

Share this post


Link to post

mloser -

 

Excellent !

 

I did pretty much the same thing, even re-naming the 3 styles as red, green, and yellow.

I also made a new option called label=last.

I sent the whole thing to Patty.

 

I wish I could get blue. No blue though.

Share this post


Link to post

BDT,

I will never understand why they did it, but they swapped the standard HTML colors from RGB to BGR, so get blue you put in xxFF0000, where xx is the opacity. For 100% opacity the value is FFFF0000. Other blues can be gotten by looking at any of the HTML color guides and reversing the Red and Blue values.

Share this post


Link to post

mloser -

 

Yeah I knew about the unusual ordering of colors. But try as I might, I never get blue; only black. (I think the needle part is blue, but the pinhead is black.) If I move the other colorss up from zero of reduce opacity, I get sickly colors that have no blue in them. I would think that ffffeeee should give light blue.

Share this post


Link to post

Try picking the blue pin icon:

 

print "<Icon><href>http://maps.google.com/mapfiles/kml/pushpin/blue-pushpin.png</href></Icon>" > name

Share this post


Link to post

mloser -

 

Excellent ! The blue icon works great. It'll certainly be better for those that are red-green colorblind.

=========================

 

Here is a script to read a whole-state shapefile. The shapefiles don't have all the fields that datasheets have, but I collected as many as seemed interesting and put them the popup balloons. To make up for the smaller data set, I included a whole bunch of amusing variables:

first: label by the first (monumentation) date

des: label by the designation

elev: label by elevation

last: label by the latest report year

county: label by the county the mark is in

who: label by who did the last report. (Check out GEOCAC land vs. USPSQD land in your state. :laughing: )

cond: label by the mark's last reported condition and date, i.e. GOOD1964, POOR1998, NOTFOUND1981

dates: label by both first and last dates, i.e. 1931-1984

 

As before, the default is:

gawk -f thisfile.awk VA_dbf.txt

and to use one of the options above,

gawk -f thisfile.awk label=elev VA_dbf.txt

(The top of the script file has all the examples listed.)

 

I put in a bunch of extra lines, currently commented out, that can be uncommented (remove the # character) to do more experiments with sizing and labeling of the icons and their labels.

 

=============================

# Usage:
# (in each case, the program makes datfile.kml from datfile.dat)

# gawk -f thisfile.awk datfile.dat
# This default version uses the PID to label each icon

# gawk -f label=des thisfile.awk datfile.dat
# This version uses the designation, instead of the PID, to label each icon

# gawk -f label=date thisfile.awk datfile.dat
# This version uses the oldest (monumentation) date to label each icon

# gawk -f label=elev thisfile.awk datfile.dat
# This version uses the elevation value to label each icon

# gawk -f label=who thisfile.awk datfile.dat
# This version uses the monumentation code to label each icon

# gawk -f label=cond thisfile.awk datfile.dat
# This version uses the reported condition and date to label each icon

# gawk -f label=dates thisfile.awk datfile.dat
# This version uses the first and last dates to label each icon

BEGIN { FS="," }
{
if (NR==1) {
# do some setups
if (label=="FIRST") label="first"
if (label=="DES") label="des"
if (label=="ELEV") label="elev"
if (label=="LAST") label="last"
if (label=="COUNTY") label="county"
if (label=="WHO") label="who"
if (label=="COND") label="cond"
if (label=="DATES") label="dates"
if (label!="") {
	if ((label!="first")&&(label!="des")&&(label!="elev")&&
		(label!="last")&&(label!="county")&&(label!="who")&&
		(label!="cond")&&(label!="dates")) {
		print "Incorrect label= value.  Using PIDs as labels."
		print "Valid label= values are: first des elev last county who cond dates"
		}
	}
gcu="http://www.geocaching.com/mark/details.aspx?PID="
ngs="http://www.ngs.noaa.gov/cgi-bin/ds_mark.prl?PidBox="
name= substr(FILENAME,1,(length(FILENAME)-4)) ".kml"
print "<kml>" > name
printf("<Document>\n") > name  # newlines just to make the kml file easier to look at
#print "<Style id=\"red\">" > name
print "<Style id=\"blue\">" > name
print "<IconStyle>" > name
#print "<color>#ff4444ff</color>" > name
# grab the blue pin from the net
print "<Icon><href>http://maps.google.com/mapfiles/kml/pushpin/blue-pushpin.png</href></Icon>" > name
# optional scaling and label coloring commented out
#print "<scale>.6</scale>" > name
print "</IconStyle>" > name
#print "<LabelStyle>" > name
#print "<color>ffff00cc</color>" > name
#print "<scale>.6</scale>" > name
#print "</LabelStyle>" > name
print "</Style>" > name
print "<Style id=\"yellow\">" > name
print "<IconStyle>" > name
print "<color>#ffeeeeff</color>" > name
print "</IconStyle>" > name
print "</Style>" > name
print "<Style id=\"green\">" > name
print "<IconStyle>" > name
print "<color>#ff44ff44</color>" > name
print "</IconStyle>" > name
print "</Style>" > name
}
else {
lon=$4; lat=$5; pid=$6; des=$7; cty=$9	
quad=$10; pos=$15; elev=$16; firstdate=$29
lastdate=$30; condition=$31; reporter=$32

printf("<Placemark><description>\n") > name
printf("<![CDATA[\n") > name  # start html coding enabling
printf("<u>PID</u>: %s\n",pid) > name
printf("<br><u>Designation:</u> %s\n",des) > name
printf("<br><u>Links</u>: <a href=\"%s%s\">Geocaching</a>\n",gcu,pid) > name
printf(" <a href=\"%s%s\">NGS</a>\n",ngs,pid) > name
printf("<br><u>County</u>: %s\n",cty) > name
printf("<br><u>USGS Quad</u>: %s\n",quad) > name
# commented-out optional year, month, and day versions below
yearf=substr(firstdate,1,4); monthf=substr(firstdate,5,2); dayf=substr(firstdate,7,2)
#printf("<br><u>First report</u>: %s %s %s\n",yearf,monthf,dayf) > name
printf("<br><u>First report</u>: %s\n",yearf) > name
yearl=substr(lastdate,1,4); monthl=substr(lastdate,5,2); dayl=substr(lastdate,7,2)
#printf("<br><u>Last report</u>: %s %s %s %s %s\n",yearl,monthl,dayl,condition,reporter) > name
printf("<br><u>Last report</u>: %s %s %s\n",yearl,condition,reporter) > name
printf("<br><u>Horizontal position type</u>: %s\n",pos) > name
#if (elev!="") printf("<br><u>Elevation (meters)</u>: %s\n",elev) > name
if (elev!="") printf("<br><u>Elevation (ft.)</u>: %s\n",elev*39.37/12) > name
printf("]]>\n") > name  # end html coding enabling
printf("</description>\n") > name
# arrange color coding
if (pos=="SCALED") { 
	print "<styleUrl>#yellow</styleUrl>" > name 
	}
else if ((pos=="HD_HELD1") || (pos=="HD_HELD2")) {
	print "<styleUrl>#green</styleUrl>" > name
	}
else { 
	print "<styleUrl>#blue</styleUrl>" > name 
	}
# handle icon label options
if (label=="des") { 
	printf("<name>%s</name>\n",des) > name 
	}
else if (label=="first") { 
	printf("<name>%s</name>\n",yearf) > name 
	}
else if (label=="last") { 
	printf("<name>%s</name>\n",yearl) > name 
	}
else if (label=="elev") {
	elevv = elev=="" ? "blank" : elev
	printf("<name>%s</name>\n",elevv) > name
	}
else if (label=="county") { 
	printf("<name>%s</name>\n",cty) > name 
	}
else if (label=="who") { 
	printf("<name>%s</name>\n",reporter) > name 
	}
else if (label=="dates") { 
	printf("<name>%s-%s</name>\n",yearf,yearl) > name 
	}
else if (label=="cond") { 
	bit=substr(condition,1,2)
	if (bit=="FI") {condl="observed" yearl}
	else if (bit=="GO") {condl="GOOD" yearl}
	else if (bit=="MA") {condl="NOTFOUND" yearl}
	else if (bit=="MO") {condl="monumented" yearl}
	else if (bit=="PO") {condl="POOR" yearl}
	else if (bit=="SE") {condl="SEEDESC." yearl}
	else {condl=condition yearl}
	printf("<name>%s</name>\n",condl) > name 
	}
else { 
	printf("<name>%s</name>\n",pid) > name  # default label
	}
# output the coordinates
printf("<Point>\n<coordinates>\n") > name
printf("%.8f, %.8f\n",lon,lat) > name
printf("</coordinates></Point>\n") > name
# finish off this placemark
printf("</Placemark>\n") > name
}
}
END {
# finish off the kml file
printf("</Document>\n") > name
printf("</kml>\n") > name
}

Share this post


Link to post

I've been trying to get a blue icon, too, but keep ending up with muddy other colors. I'm talking about selecting a color myself for a specific pushpin or set of pushpins, not running a script to make a KML file.

 

I got BDT's revised script and am now able to label the pushpins by their last report date. That combined with color coding for age of the last report will really help me zero in on the marks that need new reports.

 

BTW, those of you using awk (rather than gawk) in conjunction with non-default label options will want to put the name of the script file immediately after the "-f" argument. E.g., "-f filename label=foo". It doesn't work if you say "-f label=foo filename". At least, that's what I needed to do in the Mac OS X version of awk.

 

Many thanks to BDT for this very helpful script!

 

Patty

Edited by Wintertime

Share this post


Link to post

In both awk and gawk, the name of the script file must go next after the -f because the -f flag is telling the program that the next thing on the command line will be the name of the script file that it is to use.

 

If anyone is interested, a reference for gawk is given (among similar places) here.

Share this post


Link to post

Oops!

 

Wintertime pointed out that I had messed up the examples I listed in both of the scripts!

The order of label= and the scriptfile.awk was wrong. No wonder she had trouble with it !

The scripts work OK, just not the examples at the beginnings.

Here are the corrected versions:

 

For the shapefile one (the latest one):

# Usage:
# (in each case, the program makes shapefile.kml from shapefile.txt)

# gawk -f thisfile.awk shapefile.txt
# This default version uses the PID to label each icon

# gawk -f thisfile.awk label=des shapefile.dat
# This version uses the designation, instead of the PID, to label each icon

# gawk -f thisfile.awk label=date shapefile.txt
# This version uses the oldest (monumentation) date to label each icon

# gawk -f thisfile.awk label=elev shapefile.txt
# This version uses the elevation value to label each icon

# gawk -f thisfile.awk label=who shapefile.txt
# This version uses the monumentation code to label each icon

# gawk -f thisfile.awk label=cond shapefile.txt
# This version uses the reported condition and date to label each icon

# gawk -f thisfile.awk label=dates shapefile.txt
# This version uses the first and last dates to label each icon

 

For the datfile one (the earlier one I posted here):

# Usage:
# (in each case, the program makes datfile.kml from datfile.dat)

# gawk -f thisfile.awk datfile.dat
# This default version uses the PID to label each icon

# gawk -f thisfile.awk label=des datfile.dat
# This version uses the designation, instead of the PID, to label each icon

# gawk -f thisfile.awk label=date datfile.dat
# This version uses the oldest (monumentation) date to label each icon

# gawk -f thisfile.awk label=elev datfile.dat
# This version uses the elevation value to label each icon

# gawk -f thisfile.awk label=code datfile.dat
# This version uses the monumentation code to label each icon

Share this post


Link to post

BDT, Klemmer et al,

 

Thanks for the great information and new way to build kml files, I have it working and it really is a nice way to get the files built.

 

CallawayMT

Share this post


Link to post

Instead of starting a new thread (maybe I should be, but.....) can anyone tell me in the most simple of terms if you can get benchmarks on google earth (like you can geocaches) and how.

 

I have to be honest - I am new to this whole thing, and especially benchmarks - and my head is spinning as it regards benchmarks (ie: you found this - but you didn't find IT, etc etc etc). I want to look for benchmarks as a way of complimenting my geocaching hobby, and that is why I would like to see benchmarks on google earth.

 

Any help would be awesome - thanks!

Share this post


Link to post

wvnewbie:

 

The process to get the data from NGS, and covert it to a .gpx file is described pretty well in the sticky topic"Me First" topic at the top of this forum:

http://forums.Groundspeak.com/GC/index.php...p;#entry1811704

Then, you can just drag and drop the .gpx file onto Google Earth.

 

The process discussed above is a different way to do it, resulting in a more usable display on Google Earth, using a .kml file instead. You can try either way. The .gpx route might be a bit simpler for you.

 

Klemmer

Share this post


Link to post

Thanks klemmer - I think I may be out of my comfort zone alittle on this benchmarking thing- as I see people take it very seriously. I just thought it would be cool to find them and report them if necessary. I really like the ones that take me to old, now unused train bridges etc.

 

I will try your suggestion - thanks.

Share this post


Link to post

wvnewbie -

 

I guess we take benchark hunting seriously, but not all that seriously that we can't have a lot of fun with it! :P

 

Benchmark hunting does bring one to interesting places sometimes, it's true. But then, it also takes you to the most mundane places that there are. One I did was described as hidden under a garbage can. Indeed it was!

 

Back to the subject at hand. Actually using Google Earth isn't necessary for benchmark hunting at all. The map links right on a benchmark's page do rather well, especially the Geocaching.com map and the Google Maps map. Not only can you drop a .gpx file onto Google Earth as Klemmer says, but also, Google Earth will take a .loc file you get from downloading a batch of benchmarks from a geocaching benchmark search (via this page).

 

So anyway, enjoy your benchmark hunting and mapping at any level you want. :anicute:

 

Welcome to the hobby!

Share this post


Link to post

BDT is right — this is (or should be) mainly about the fun of finding stuff that's often hidden in plain sight. Reading old reports on these benchmarks reminds you of how much things have changed. In 1971 a country store on a two-lane road has become by 2007 a six-lane highway and a regional mall. Water towers and air navigation beacons disappear. Elementary schools become arts centers. Post offices become most anything at all.

 

Many of us who take the time to post in the forums are in the top five percent of seriousness, but don't let that dissuade you. Still, even the casual benchmarker will find it rewarding to take a few minutes to read through the information on the main benchmark page. In particular, note the sections about coordinates being way off, reference marks and azimuth marks, and marks that are not in the database.

 

One other mapping tool I find useful is USAPhotoMaps, which accepts imports of .gpx files (which can be created in one easy drop-and-drag step by taking the .dat file you download from the NGS and run it through bmgpx). USAPhotoMaps will then display the benchmarks overlaid on either USGS topo maps or aerial photos. Except in some urban areas, the aerial photos are older, black-and-white images, lower-resolution than available elsewhere. But having images 10 or 20 years old can be very useful, as for example when I was able to see a water tower on an old picture that was not there in current images. I had been to the site, but the old aerial photo was confirmation that I was in the right place.

 

-ArtMan-

Share this post


Link to post

You guys are far more ambitious than I am. I simply did a find by my zip code on the “Benchmark Hunting” web site and then saved the “loc” file. Launched Google earth, did a file Open and Opened the loc file. Got all the bench marks in Atlanta that way. I saved it as a “kmz” (well the full name is Atlanta Waypoints.kmz). Now after one of my BM hunting trips, I open this file and color code the BM I found as green and the ones I could not locate as red. I have found this as an easy method of keeping track of the BM I have searched for and the results. Of course, when I start added other states, this will not work as well as it does for just a localized area.

Share this post


Link to post

Hi, everyone.

 

I'm resurrecting this subject because I'm trying to divide my benchmark pushpins into more categories. The FFFF0000 gives me kind of a blue-black pin; I guess I could use the HTML code to grab the really blue pin from Google's website. But are there any other colors available?

 

I thought these were standard HTML colors, or a simple variation thereon. But I've tried substituting other values and the best I've gotten is a redder red. I can't get magenta or any other really different color.

 

I've tried searching for other options, but no one (including Google) seems to have a list of valid Google Earth pushpin colors. Anyone here know of others than the green, yellow, and red/orange that BDT already has in his program?

 

Thanks!

Patty

Share this post


Link to post

Patty,

 

Make you a folder for custom icons in my pictures.

Then search for the icons you want to add to your list,add to that folder.

 

Then in google earth you can add custom icons.

Choose the one you want to add,from that folder.

 

I have 100's of icons I use for waymarkingm,mapping.

I also use the benchmark icon for my benchmarks,you can choose a custom color for those as well.

Share this post


Link to post

Patty -

 

Yeah there's some sort of bug in the Google Earth pushpins' RGB color specification command. The blue simply does not work - it goes pretty much from zero to black in one shot. All I could get is the 'fall colors' scenario; i.e. no blue. Mloser found that a blue pushpin could be called from some website, but I forget which. I think it's earlier in this topic.

 

I figured that G.E. would fix this bug right away, but it lingers on and on and on and on........

Share this post


Link to post
Make you a folder for custom icons in my pictures.

Then search for the icons you want to add to your list,add to that folder.

Hi, GT. Thank you for offering to make a folder for me with your icon collection. (Or maybe you're saying that you already have such a folder; I'm not quite sure.) However, I don't need different icons. The pushpins are fine. I just need the hexadecimal codes for additional colors.

 

Yeah there's some sort of bug in the Google Earth pushpins' RGB color specification command. The blue simply does not work - it goes pretty much from zero to black in one shot. All I could get is the 'fall colors' scenario; i.e. no blue. Mloser found that a blue pushpin could be called from some website, but I forget which. I think it's earlier in this topic.

Yep, as I mentioned, I have the HTML code that Mloser provided for a truly blue pin, but the blue-black one generated by the hex code isn't bad. I'm just wondering whether there are any other pushpin colors available. From your remark about "fall colors," I'm guessing that green, yellow, and red/orange are it? As I mentioned, I managed to get a red that was redder than the orangeish one. But most of the hex I've tried has just given me the same few colors.

 

Is there something weird in the GE code that looks at the input hex and says, "Hmm, I don't have that color, I'll just do red again"? Are there truly only three or four colors in the GE pushpin palette? I notice that when you create a placemark, you can choose a color for it. But the only ones that "took" were yellow, orange, red, green, blue/back, and brown. I wouldn't mind adding brown to my pushpin repertoire, but the color charts in the placemark customization window don't provide hex code, so I don't know how to duplicate it in my .kml file.

 

Patty

Share this post


Link to post

Well to get specific, the Google Earth color commands are all in the format aabbggrr the first 2, aa, being opacity, the bb being the blue intensity of color, the gg is the green intensity of color, and the rr being the red intensity of color. The amounts are in hexidecimal. Therefore ff0000ff is full opacity, full red with 0 blue and 0 green. Almost any mount in the bb part will go black. Something like ff1100ff will be gray, as I recall. Go much higher than 11 in the blue part of the color spec. and you'll get black. It's about as if the color spec was: opacity, black, green, red. There's even a random color mode but all you'll get is the 'fall colors'.

 

All this problem is just with the pushpins. With labels, you can get nice blues. :unsure:

Edited by Black Dog Trackers

Share this post


Link to post
Well to get specific, the Google Earth color commands are all in the format aabbggrr the first 2, aa, being opacity, the bb being the blue intensity of color, the gg is the green intensity of color, and the rr being the red intensity of color.

Yeah, I saw your discussion with Mloser about that earlier in this thread. Confusing! I used that advice and reversed the hex numbers I found elsewhere. Nonetheless, I kept ending up with red, yellow, and green...

 

Something like ff1100ff will be gray, as I recall.

Oh, thanks, maybe I'll try that one.

 

I managed to get brown working! I used "ff006699". I also tweaked the DAT2KML script you gave me a few months ago to add one more category, and to redefine the divisions among categories. So I now have my benchmark pushpins color-coded to display the last NGS report date according to the following criteria:

 

Before 1958 (more than 50 years ago; high-value targets!)

1959-1987 (worth checking)

1988-2002 (not as interesting, but if I'm in the vicinity...)

2003-present (no use filing reports on these unless something really interesting has happened)

 

This will really help me zero in on the marks I most want to look for!

 

Patty

Share this post


Link to post

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...
Sign in to follow this  
Followers 2

×
×
  • Create New...