Mapping Benchmarks

April 4, 2007

I'm sure there is, but can someone tell me if, and how, to map multiple benchmarks into any of the map programs (googlemaps, google earth, mapquest, etc.) I downloaded the county listing as instructed, but it would sure be nice to have mapping capabilities, so I can see multiple BMs as I drive through town, etc.

Thanks all!

7.9k · April 4, 2007

You can use a little program called BMGPX to convert the files received from the NGS into .gpx files.

Then you can use GSAK to merge the files, and do the usual conversions to your mapping software(s) and upload them to your GPSr(s).

Here's a little tutorial on the subject.

April 4, 2007

I'm sure there is, but can someone tell me if, and how, to map multiple benchmarks into any of the map programs (googlemaps, google earth, mapquest, etc.) I downloaded the county listing as instructed, but it would sure be nice to have mapping capabilities, so I can see multiple BMs as I drive through town, etc.

Thanks all!

There's a thread in progress that talks about this:

"Google Maps API and Benchmarks" thread

Although the subject started with Google Maps, the discussion encompassed other programs. My own understanding is that dropping a .gps file onto Google Earth (read the thread to see how) is straightforward and works with no "massaging" of the data.

For myself, I wanted something web oriented as part of my web site, so I went with Google Maps. I have something working:

My Gooogle Maps prototype

Note: that when the map comes up, you must click on "Add unvisited marks" to bring up the bulk of the stations in the county. I suggest you zoom in a little first before doing this, or the marks will overwhelm the map (as well as taking some time). If you put the mouse over a mark, a little baloon comes up and you can click on it and get a hi-res map of that station.

I'm working to improve the speed. The steps to get this going are more than the above (read the thread) and involves getting into the Google Maps API, but for me it was worth the effort since I wanted it up on my site and accessible to folks who know nothing about GSAK, county downloads (or even Google Earth).

Stuff is out there. Have fun.

Papa Bear

Edited April 4, 2007 by Papa-Bear-NYC

April 5, 2007

Anyone have any ideas on doing a partial county? My neighboring county, Owyhee county, Idaho covers 7666 square miles. That makes for a very large map. It would be nice to confine mapping to longitude and latitude boundaries. BTW, that county is mostly empty of people. Nice. :laughing:

4.4k · April 5, 2007

GPSTrackmaker.

It does everything you need to interface with all the different programs.

I won't go into details.

GPSTrackmaker

DGTV is really nice too.

DGTV

Edited April 5, 2007 by GEO*Trailblazer 1

1.6k · April 5, 2007

Tater:

Drop a .gpx file into Google Earth, then you can zoom down to whatever ara you want to see.

or

If you want to chop up a county into smaller pieces (e.g. to get 500 mark chunks into your GPS), GSAK has nice tools to do that. You can setup filters that describe round or rectangular areas. I wish it was easier, though. It would be nice to have it automatically section LA county (8189 marks) into 500 mark chunks. But at least you can do it.

K

April 5, 2007

Yes, GSAK is great. I set a filter to exclude those waypoints greater than a specific distance (16 miles today) and then send the resulting subset of waypoints to MapSource to see where they all are. If the location isn't exactly what I want, I can change the location of the center in GSAK and it automatically recomputes the waypoints in the set distance. I resend them to MapSource to recheck their whereabouts... etc... I also filter out the ones I've marked Found, Not Found or Destroyed.

Barry

aka Doc Geo

2.7k · April 5, 2007

automatically section LA county (8189 marks) into 500 mark chunks

Hmm, interesting problem. One approach could be to:

Determine how many marks you want per section.
Find the nearest divisor that is a square number. For instance if you want 9 marks per section, and there are 81 marks total, then the divisor is 3^2 =9.
Sort a file or marks by latitude. Divide into 3 parts, giving you 27 marks per horizontal band.
Sort each band separately by longitude into 3 parts, giving you 9 marks in each part of a band. You get a bunch of funny-shaped rectangles.
Print out each list of benchmarks (either just mapping gpx files, or whole-datasheet files), perhaps in filenames like a1 through a9.

Well, I guess for all this to be automatic it would have to be in a program, like GSAK. :laughing: (Maybe it already is.)

(Of course the number doesn't really have to be a square; you could use 3 horizontal bands and then split each band into 4 parts instead of 3, giving 12 total sections.)

Maybe there's a fancy smoother way to do that using circles of various radii, I don't know.

1.6k · April 6, 2007

BDT:

Yeah, I'm working on ways in GSAK. You ideas are roughly mine also, but the %^&* county is so irregularly shaped, and BM distibution is anything but uniform, so it gets messy. I need less than 500 mark chunks to put into my Meridian. Lots of files is not trouble. All sorts of room on the SD card.

At the moment, I'm defining a polygon in Google Earth. Save it as a .kml, load it into GSAK in a filter as a polygon (yep, GSAK takes the .kml for that purpose - thanks again, Clyde!). Not too bad. I did Orange county a long time ago before the tools were as good. Riverside County almost rectangular, so it was not too bad. LA county lots bigger, so I waited for better tools. Glad I did. Still time consuming. Life is like that sometimes!

4.1k · April 6, 2007

The simplest way to cut-up a county in to usable chunks (I prefer 475 waypoints per file) is to sort the GSAK database by either Latitude or Longitude (I usually use Latitude) first. Then clear all "user flags" . Now set the first XX user flags, Where XX is a multiple of the number of waypoints you want in your GPSr files. For instance, I use 475 waypoints to allow some waypoints to be available for field use, so I would set use flag for the first 950 waypoints. Filter on User Flag "Set" and save them to a temp database. Open the Temp database and sort on the opposite coordinates - if you filtered on Latitude, then sort on Longitude. Clear all 'user flags' and then set them for the first 475 waypoints. Filter on User Flag = set and save that group to your mapping program & GPSr. Reverse the current filter and Save that set of waypoints to your Mapping program & GPSr. You can now clear the temp database.

Go back to the main county database, making sure you are still sorted on Lat or Long whichever you did the first time through. Highlight the first waypoint that does NOT have a user flag set. Clear all user flags and then set the user flag from the current position for the same number of waypoints that you used the first time through. Again filter on User Flag = set and save to the temp database and filter as described above. Send those two files to the mapping program & GPSr. Clear the temp database.

Go back to the main county database, making sure you are still sorted correctly and highlight the next waypoint in line and clear all user flags. Set the user flags from the highlighted waypoint for the proper number of waypoints and repeat the above procedure.

When you get to the last set of waypoints you can just split it in half if there is less than the number of waypoints you have been using. In other words, if the are only 730 waypoints in the last set you can filter for 365 waypoints or you can filter for 475 waypoints. I would use 365 to avoid 1 large file and 1 small file for this last group.

It sounds more complicated than it really is.

Perhaps Klemmer can come up with a macro to do this?

John

1.6k · April 7, 2007

Hmmm....

I like your idea, John. Thanks.

As for a Macro --- I dunno... I'll look at the possibilities.

Klem

2.7k · April 8, 2007

Here is a script file to use with free gawk (download unxutils.zip and extract gawk.exe, put gawk in your path or in the directory where you would use it).

This script will read either an NGS .DAT file or an NGS shapefile. It will do the latitude and longitude sorting, split according to a desired maximum number of benchmarks per file, and produce either .txt files or .gpx files. The .txt files are just lists of PIDs to use at the NGS file reading page, and the .gpx files are for Google Earth mapping. It will also provide either NGS URLs or geocaching benchmark URLs for datasheets. It will also filter by county or quadrangle (without the date part of the quad name).

gawk -f ngsread.awk inputfile

is the normal mode, where inputfile is a .dat file such as va001.dat

With the Virginia shapefile, the program outputs 42 files of 484 marks per file.

In Google Earth, loaded files can be easily turned off or on by clicking in their boxes in the Google Earth sidebar.

Here is the contents of the scriptfile ngsread.awk

# Ordinary usage:
# gawk -f thisfile.awk datfile.dat			Default outputs .txt files of just lists of PIDs

# The following examples show options can be used alone or together
# gawk -f thisfile.awk marks=800 datfile.dat		Limit pieces to 800 marks (default=500)
# gawk -f thisfile.awk out=gpx datfile.dat		Output .gpx files just for mapping
# gawk -f thisfile.awk county=SHARP datfile.dat		Only include benchmarks in SHARP county
# gawk -f thisfile.awk quad="GREEN MT" datfile.dat	Only include benchmarks in the GREEN MT quad
# gawk -f thisfile.awk fileinit=XX datfile.dat		Use XX as the first part of output names
# gawk -f thisfile.awk input=shape shapefile.txt	Read a shapefile instead of the default (a .dat file)
# gawk -f thisfile.awk increase=bands shapefile.txt	Artificially increase the number of horizontal bands

BEGIN {FS=","}
{
if (NR==1) {
gcu="http://www.geocaching.com/mark/details.aspx?PID="
ngs="http://www.ngs.noaa.gov/cgi-bin/ds_mark.prl?PidBox="
if ((dbase=="ngs")||(dbase=="NGS")) gcu=ngs
filen="afile"
if (fileinit!="") filen=fileinit
ext=".txt"
if ((out=="gpx")||(out=="GPX")) ext=".gpx"
if (marks=="") marks=500
}
else {
if ((input=="shape")||(input=="SHAPE")) {
	# read a shapefile
	lon=$4; lat=$5; pid=$6; cty=$9
	qd=substr($10,1,length($10)-7)
	}
else {
		while (substr($0,1,1)==" ") {
		# default is a .dat file
		dcount++
		if (substr($0,8,13)=="  DESIGNATION") {
			# get designation
			des=substr($0,25,length($0)-25+1)
			}
		else if (substr($0,8,6)=="  USGS") {
			# get quad
			qd=substr($0,25,length($0)-25+1-7)
			if ((quad!="")&&(quad!=qd)) prt=0
			}
		else if (substr($0,33,8)=="*CURRENT") {
			dcount=0
			}
		else if (dcount==2) {
			# get pid,lat,lon,pos
			pid=substr($0,2,6)
			split(substr($0,25,14),deg," ")
			lat=deg[1]+deg[2]/60+deg[3]/60/60
			lon=substr($0,46,15)
			split(substr($0,46,15),deg," ")
			lon=(deg[1]+deg[2]/60+deg[3]/60/60)*(-1)
			pos=substr($0,69,length($0)-69+1)
			}
		else {
			}
		next
		}
		#if (NR>1) print lat,lon,pid,des,quad
	}
if ((quad!="")&&(quad!=qd)) next
if ((county!="")&&(county!=cty)) next
id++
#print "id =		 ",id
latlonpid=sprintf("%s|%s|%s",lat,lon,pid)
#print latlonpid
latarr[id]=latlonpid
}
}
END {
n=asort(latarr)

# determine the number of bands, sections within a band, benchmarks per file
sqrt(n/marks) - int(sqrt(n/marks)) >= 0.5 ? bands=int(sqrt(n/marks))+1 : bands=int(sqrt(n/marks))
if (increase=="bands") bands++
inband=int(n/bands)
if (bands*inband < n) inband++
sections=int(inband/marks)
if (bands*sections*marks < n) sections++
bms=int(inband/sections)
if (bms*sections*bands < n) bms++
print n,"marks;",bands,"horizontal bands;",sections,"sections per band;",bms,"marks per file;",bands*sections,"files"

# make each band
for (j=1;j<=bands;j++) {
	fileno++
	if ((out=="gpx")||(out=="GPX")) if (fileno>1) print "</gpx>" > name
	name=sprintf("%s%d%s",filen,fileno,ext)
	if ((out=="gpx")||(out=="GPX")) print "<gpx>" > name
	first=(j-1)*inband+1
	last=j*inband
	# make an array of the horizontal band
	l=0
	for (i=first;i<=last;i++) {
		l++
		lonarr[l]=""
		split(latarr[i],partat,"|")
		lonlatpid=sprintf("%s|%s|%s",partat[2],partat[1],partat[3])
		lonarr[l]=lonlatpid
		}

	# sort and print out the band in longitude order
	m=asort(lonarr)
	count=0
	for (k=1;k<=m;k++) {
		count++
		if (count>bms) {
			count=1
			fileno++
			if ((out=="gpx")||(out=="GPX")) print "</gpx>" > name
			name=sprintf("%s%d%s",filen,fileno,ext)
			if ((out=="gpx")||(out=="GPX")) print "<gpx>" > name
			}
		split(lonarr[k],parton,"|")
		fpid=parton[3]; flat=parton[2]; flon=parton[1]
		url=gcu fpid
		if (fpid=="") break
		if ((out=="gpx")||(out=="GPX")) {
			printf("<wpt lat=\"%s\" lon=\"%s\">\n",flat,flon) > name
			printf("<url>%s</url>\n",url) > name
			printf("<urlname>Datasheet</urlname>\n") > name
			printf("<name>%s</name>",fpid) > name
			printf("</wpt>\n") > name
			}
		else {
			print fpid > name
			}
		}
	}
	if ((out=="gpx")||(out=="GPX")) print "</gpx>" > name
print "Wrote ",fileno," files"
}

This should be pretty easy to convert to C or one of the other similar languages.

1.6k · April 8, 2007

While BDT was writing the above gawk script, I was slogging through manually chopping up LA county (using John's method). Worked OK, but a little tedious. Better than other ways I tried. Thanks, John! I set up a pretty map from Google Earth of all the sections of LA county. I started working on a macro in GSAK, but ran into some problems. Not sure it can be done in GSAK. At least not by me.

Now, I will have to gawk myself and give BDT's program a try. Should be handy! Thanks, BDT!

BTW: I left the three southernmost Channel Islands off the map (Catalina, Santa Barbara & San Clemente), which are part of LA county. I extracted those together as one 400 mark file. If anyone wants the files, just ask. The .zip file of the 18 separate .gpx files is just over 3.8MB (including the islands). 8189 Benchmarks. Source NGS data file is about 6 months or so old. I also have the fies for Orange County available (6 files).

Boy, I have a lot of benchmarking potential out here! I bet only a few percent have recent NGS or GC logs. Of course, some will be inaccessible, and some will be in neighborhoods you wouldn't want to be in anytime, without an armed Law Enforcement escort, if then.

The blue dot and "Klemmer" southeast of LA down in the OC is me.

1.4k · April 8, 2007

I set up a pretty map from Google Earth of all the sections of LA county.

That is a pretty map!

Are the marks mostly aligned along major thoroughfares?

The blue dot and "Klemmer" southeast of LA down in the OC is me.

You folks really do call it "the OC"?? I assumed that was just silly TV marketing. Any idea how "the" ever got prepended to the county initials?

Patty

2.7k · April 8, 2007

Here's a view of most of Virginia from the NGS state shapefile with labels removed and just the benchmark locations showing in colors. Each section has 484 benchmarks.

Near the lower right is the Chesapeake Bay bridge-tunnel with benchmarks all along it. Near the top right are the denser areas near Washington DC. Near the center right in yellow is the Richmond area.

1.6k · April 9, 2007

Patty:

Naw, we don't really call it "the OC". Gimme a break! (or at least I don't!). Maybe just OC sometimes.... Sort of like "LA", but upscale...? Just trying to lighten up the discussion..... :unsure: Maybe the "the" is like "The Donald" (Trump)?

We sure do make pretty pictures, huh? Thanks. Yes, the "corridors" of marks are somewhat along CERTAIN roads, but not necessarily the major ones (by today's standards). Maybe they were earlier in the century... ???

ALSO, interestingly (actually logically), there are huge numbers of marks along the major aquaducts (from the Colorado, and from NorCal, and branches of them). I see some possibilities there for serious numbers, if access is not too much of a problem....

1.6k · April 9, 2007

BDT:

Your ngsread.awk program works GREAT! Thanks a million! Owe you one. Glad I still remember how to run DOS. It did a very nice job a chopping up Riverside County into 9 nice sized pieces, gpx files work great. Now I need to get them into my Meridian... Hmmm... GSAK will work fine for that.

Thanks again!

Klemmer

April 11, 2007

not sure if this will help you? but i found it quite interesting in finding benchmarks in my area. I dont have a high speed conection like some do so i diddent explor it to its full potential but its a start. the first tells you how to use it and the second is the page. ngs ngs2

Mapping Benchmarks

Recommended Posts

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Join the conversation