+dukeofurl01 Posted January 30, 2011 Share Posted January 30, 2011 In: ftp://ftp.ngs.noaa.gov/pub/DS_ARCHIVE/ There are 3 folders, Archive, datasheets, and Shapefiles I know what Shapefiles are, they can only be opened in ArcGIS. But what do they contain? Current, active marks, or destroyed/archived ones, or both? As for the folders Archive and Datasheets, what is the difference in these? Is Archive for Archived datasheets that are no longer active or have been destroyed, and /datasheets for current active marks? Quote Link to comment
kayakbird Posted January 30, 2011 Share Posted January 30, 2011 In: ftp://ftp.ngs.noaa.gov/pub/DS_ARCHIVE/ ---Shapefiles are, they can only be opened in ArcGIS. But what do they contain? Duke, The below unzipped _dbf file (header line and three MT-113-YELLOWSTONE NAT PARK points) was acquired by following the procedures at the NGS site. It can be pasted into an Excel file and arranged into columns using the Data, Text to Columns feature. #FeatureId,DATA_DATE,DATA_SRCE,DEC_LONG,DEC_LAT,PID,NAME,STATE,COUNTY,QUAD,LATITUDE,LONGITUDE,POS_DATUM,DATUM_TAG,POS_SRCE,ELEVATION,ELEV_DATUM,ELEV_SRCE,ELLIP_HT,ELLIP_SRCE,POS_ORDER,POS_CHECK,ELEV_ORDER,ELEV_CLASS,ELEV_CHECK,DIST_RATE,ELLP_ORDER,ELLP_CLASS,FIRST_RECV,LAST_RECV,LAST_COND,LAST_RECBY,SAT_USE,SAT_DATE,STABILITY 1,20110130,http://www.ngs.noaa.gov/cgi-bin/ds_mark.prl?PidBox=QX0684,-110.24656, 45.00715,QX0684,BUFFALO,MT,YELLOWSTONE NAT PARK,ROUNDHEAD BUTTE (1987),45 00 25.73045(N),110 14 47.61969(W),NAD 83,(1993),ADJUSTED, 2806.7 ,NAVD 88 ,VERTCON,,,3,Y,,,,,,,1958 ,1958 ,MONUMENTED ,USGS,,,C 2,20110130,http://www.ngs.noaa.gov/cgi-bin/ds_mark.prl?PidBox=QX0706,-110.83757, 45.00534,QX0706,ELECTRIC PEAK,MT,YELLOWSTONE NAT PARK,ELECTRIC PEAK (1986),45 00 19.23924(N),110 50 15.24631(W),NAD 83,(1992),ADJUSTED, ,NAVD 88 ,,,,3,Y,,,,,,,1950 ,1950 ,FIRST OBSERVED ,CGS,,, 3,20110130,http://www.ngs.noaa.gov/cgi-bin/ds_mark.prl?PidBox=QX0707,-110.83755, 45.00532,QX0707,ELECTRIC RM 2,MT,YELLOWSTONE NAT PARK,ELECTRIC PEAK (1986),45 00 19.14289(N),110 50 15.19780(W),NAD 83,(1992),ADJUSTED, 3351.8 ,NAVD 88 ,VERTCON,,,3,Y,,,,,,,1951 ,1951 ,MONUMENTED ,USGS,,, There are 35 columns. The good stuff - recovery information - is in AC to AF. I end up deleting a whole bunch of the technical information columns. Wish it had a 'Set By' column, but I will generally go through the DS's for the UNK's and add the agency and my best guess at the year. L/L is given in two formats. DeLorme uses DD.ddddd -DDD.ddddd WAYPOINT NAME, so the L/L columns have to be swapped. I combine PID NAME FIRST_RECV (or NFYY) columns for the waypoint name. This all takes a bit of time, but I end up fairly familiar with the county and generally tab a few unique ones as targets. A good Montana winter project. kayakbird Quote Link to comment
foxtrot_xray Posted January 31, 2011 Share Posted January 31, 2011 I know what Shapefiles are, they can only be opened in ArcGIS. Oh, heck no - MANY GIS programs can open them, including GDAL (open source projection tools), DeLorme Xmap, and the free MapWindow GIS. Quote Link to comment
+dukeofurl01 Posted January 31, 2011 Author Share Posted January 31, 2011 Ok, but that's secondary. What I'm really after is... what is the difference between Archive and Datatsheets? Is one inactive and one active? Or is it something else? Quote Link to comment
tosborn Posted January 31, 2011 Share Posted January 31, 2011 (edited) The NGS database is constantly changing....new recoveries, new marks, marks classified as destroyed, etc. Once a month on a particular day for each State, NGS takes a snapshot of all non-destroyed PIDs. They call this an archive since, on the very next day, one or more datasheets may be different due to new recoveries, new marks, etc. The archive can be either in textual datasheet format (Datasheets folder), or they can be in shapefile format (Shapefiles folder). After a month, they create new archive files for the State, and save the previous month's textual datasheet archive in the ARCHIVE folder. The previous month's shapefile archive is saved in an Archive folder under the Shapefiles folder. So there are really two meanings of archive involved. But archive doesn't mean destroyed or inactive. Edited January 31, 2011 by tosborn Quote Link to comment
foxtrot_xray Posted January 31, 2011 Share Posted January 31, 2011 Ok, but that's secondary. What I'm really after is... what is the difference between Archive and Datatsheets? Is one inactive and one active? Or is it something else? Ah - I think someone else mentioned this - but a shapefile only contains a small sub0-set of data - The Lat/Lon, Designation, State, uhm.. URL. Those fields I know for sure, possibly other fields I can't remember at the moment. The datasheet has everything, including those fields above. So other than data differences, SHP files can be imported without modification into a handful of apps. The datasheets really can't, unless processed first. Quote Link to comment
tosborn Posted January 31, 2011 Share Posted January 31, 2011 (edited) That's correct. Important data items for benchmarking that are not in the shapefiles are monumentation type (e.g. benchmark, triangulation station, tower, etc), stamping, box scores, setting, and "to reach" info. The following fields are what the shapefiles include: DATA_DATE - date the archive was prepared DATA_SRCE - URL of the datasheet DEC_LONG - decimal longitude DEC_LAT - decimal latitude PID NAME STATE COUNTY QUAD - USGS quadrangle containing mark LATITUDE - degrees, minutes and seconds LONGITUDE - ditto POS_DATUM - NAD83 DATUM_TAG POS_SRCE - scaled, adjusted, HH ELEVATION ELEV_DATUM ELEV_SRCE ELLIP_HT ELLIP_SRCE POS_ORDER POS_CHECK ELEV_ORDER ELEV_CLASS ELEV_CHECK DIST_RATE ELLP_ORDER ELLP_CLASS FIRST_RECV - first recovery date LAST_RECV - last recovery date LAST_COND - good, not found, etc LAST_RECBY - agency for last recovery SAT_USE - satellite visibility SAT_DATE STABILITY Edited January 31, 2011 by tosborn Quote Link to comment
kayakbird Posted January 31, 2011 Share Posted January 31, 2011 (edited) For me the utility of the shape file is that once I get it in an Excel file I can do a variety of data sorts. Generally the first sort will be on FIRST_RECV, which is almost always the year set. Out West UNK's with a numerical NAME will be mostly USGS marks. Some UNK's might fit in a bench mark string that does have a known year. Odd NAME(s) can be used to spot State Survey or Bureau of Public Road series. Along the Medicine Line look for mid-20's dates that may have been set by a Canadian crew. Still hoping to find one of their disks. As previously mentioned, I will then combine PID NAME FIRST_RECV into one column for the waypoint name when I pin my DeLorme laptop map. Someday I may break down and get a GPS that will take more than the 10 characters available in a Legend. More or less current full text County by County DATASHEETS are kept in a State by State Notebook file. kayakbird Edited February 1, 2011 by kayakbird Quote Link to comment
foxtrot_xray Posted January 31, 2011 Share Posted January 31, 2011 Someday I may break down and get a GPS that will take more than the 10 characters available in a Legend. I've been able to do 'paperless benchmarking' with DeLorme's PN-series devices, taking the info straight from a GPX (using my converter program OR bmgpx) and importing complete datasheets. Thanks tosborn for the full list. I knew there was other data in there. The most important bit - especially for scaled marks - the description is missing, which makes it useful in only rare circumstances. Quote Link to comment
+dukeofurl01 Posted January 31, 2011 Author Share Posted January 31, 2011 The NGS database is constantly changing....new recoveries, new marks, marks classified as destroyed, etc. Once a month on a particular day for each State, NGS takes a snapshot of all non-destroyed PIDs. They call this an archive since, on the very next day, one or more datasheets may be different due to new recoveries, new marks, etc. The archive can be either in textual datasheet format (Datasheets folder), or they can be in shapefile format (Shapefiles folder). After a month, they create new archive files for the State, and save the previous month's textual datasheet archive in the ARCHIVE folder. The previous month's shapefile archive is saved in an Archive folder under the Shapefiles folder. So there are really two meanings of archive involved. But archive doesn't mean destroyed or inactive. That's the best explanaition to fit what I was looking for yet. Now that the /Archive folder is known, (a monthly snapshot) what is the difference between this and the /Datasheet folder? Is the /Datasheet folder "live" as in changed every day? Quote Link to comment
tosborn Posted February 1, 2011 Share Posted February 1, 2011 (edited) No. The Datasheet folder contains the most recent monthly State Archive files in textual format. The Archive folder contains the archived Archive files....in other words, last month's textual format Archive files. That's why I said the word "archive" has two meanings.....1) a statewide snapshot and 2) last month's files. Edited February 1, 2011 by tosborn Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.