TUPPERHUNTER Posted June 18, 2004 Share Posted June 18, 2004 Has any one saw a program that will remove certain lines from a datasheet to make the file smaller. Quote Link to comment
+Black Dog Trackers Posted June 18, 2004 Share Posted June 18, 2004 TUPPERHUNTER - I did it with a unix sed (freeware) program and a sed script file. YeOldeImposter did it with a perl script. Quote Link to comment
TUPPERHUNTER Posted June 18, 2004 Author Share Posted June 18, 2004 can you explane a little more about this. Quote Link to comment
+Black Dog Trackers Posted June 18, 2004 Share Posted June 18, 2004 I use my sed script in a batch file that invokes it in DOS. It is for changing a file from the NGS into a shorter file. Here is an example: For a one-PID file example, this: The NGS Data Sheet See file dsdata.txt <http://www.ngs.noaa.gov/cgi-bin/ds_lookup.prl?Item=DSDATA.TXT> for more information about the datasheet. DATABASE = Sybase ,PROGRAM = datasheet, VERSION = 7.03 1 National Geodetic Survey, Retrieval Date = JUNE 5, 2004 HV2008 *********************************************************************** HV2008 DESIGNATION - SPEED RM 1 HV2008 PID - HV2008 HV2008 STATE/COUNTY- DC/DISTRICT OF COLUMBIA HV2008 USGS QUAD - WASHINGTON WEST (1983) HV2008 HV2008 *CURRENT SURVEY CONTROL HV2008 ___________________________________________________________________ HV2008* NAD 83(1986)- 38 53 04. (N) 077 02 52. (W) SCALED HV2008* NAVD 88 - 1.017 (meters) 3.34 (feet) ADJUSTED HV2008 ___________________________________________________________________ HV2008 GEOID HEIGHT- -32.05 (meters) GEOID03 HV2008 DYNAMIC HT - 1.016 (meters) 3.33 (feet) COMP HV2008 MODELED GRAV- 980,096.0 (mgal) NAVD 88 HV2008 HV2008 VERT ORDER - FIRST CLASS II HV2008 HV2008.The horizontal coordinates were scaled from a topographic map and have HV2008.an estimated accuracy of +/- 6 seconds. HV2008 HV2008.The orthometric height was determined by differential leveling HV2008.and adjusted by the National Geodetic Survey in June 1991. HV2008 HV2008.The geoid height was determined by GEOID03. HV2008 HV2008.The dynamic height is computed by dividing the NAVD 88 HV2008.geopotential number by the normal gravity value computed on the HV2008.Geodetic Reference System of 1980 (GRS 80) ellipsoid at 45 HV2008.degrees latitude (g = 980.6199 gals.). HV2008 HV2008.The modeled gravity was interpolated from observed gravity values. HV2008 HV2008 SUPERSEDED SURVEY CONTROL HV2008 HV2008 NGVD 29 (??/??/??) 1.253 (m) 4.11 (f) ADJUSTED 1 2 HV2008 HV2008.Superseded values are not recommended for survey control. HV2008.NGS no longer adjusts projects to the NAD 27 or NGVD 29 datums. HV2008.See file dsdata.txt <http://www.ngs.noaa.gov/cgi-bin/ds_lookup.prl?Item=HOW_SUP_DET>to determine how the superseded data were derived. HV2008 HV2008_U.S. NATIONAL GRID SPATIAL ADDRESS: 18STV836836(NAD 83) HV2008_MARKER: DR = REFERENCE MARK DISK HV2008_SETTING: 30 = WALL HV2008_STAMPING: NO 1 1935 HV2008_STABILITY: D = MARK OF QUESTIONABLE OR UNKNOWN STABILITY HV2008 HV2008 HISTORY - Date Condition Report By HV2008 HISTORY - 1935 MONUMENTED CGS HV2008 HV2008 STATION DESCRIPTION HV2008 HV2008'DESCRIBED BY COAST AND GEODETIC SURVEY 1935 HV2008'AT WASHINGTON. HV2008'AT STANDARD BRONZE REFERENCE DISK STAMPED NO. 1 1935 SET IN THE HV2008'CONCRETE TOP OF THE GRANITE SEA WALL ALONG WEST POTOMAC PARK HV2008'1/2 MILE NW OF THE INLET GATE BRIDGE OF THE TIDAL BASIN, AND S HV2008'OF THE COMFORT STATION AT THE GOLF PRACTICE COURSE IN WEST HV2008'POTOMAC PARK. MARK IS 1.2 FEET IN FROM THE WATER FACE OF THE SEA HV2008'WALL, AND 65 FEET SW OF SPEED. *** retrieval complete. Elapsed Time = 00:00:01 becomes this: HV2008 *********************************************************************** HV2008 DESIGNATION - SPEED RM 1 HV2008 PID - HV2008 HV2008 STATE/COUNTY- DC/DISTRICT OF COLUMBIA HV2008 *CURRENT SURVEY CONTROL HV2008 NAD 83(1986)- 38 53 04. (N) 077 02 52. (W) SCALED HV2008 NAVD 88 - 1.017 (meters) 3.34 (feet) ADJUSTED HV2008 MARKER: DR = REFERENCE MARK DISK HV2008 SETTING: 30 = WALL HV2008 STAMPING: NO 1 1935 HV2008 MONUMENTATION: 1935 by CGS HV2008 STATION DESCRIPTION HV2008 DESCRIBED BY COAST AND GEODETIC SURVEY 1935 HV2008 AT WASHINGTON. HV2008 AT STANDARD BRONZE REFERENCE DISK STAMPED NO. 1 1935 SET IN THE HV2008 CONCRETE TOP OF THE GRANITE SEA WALL ALONG WEST POTOMAC PARK HV2008 1/2 MILE NW OF THE INLET GATE BRIDGE OF THE TIDAL BASIN, AND S HV2008 OF THE COMFORT STATION AT THE GOLF PRACTICE COURSE IN WEST HV2008 POTOMAC PARK. MARK IS 1.2 FEET IN FROM THE WATER FACE OF THE SEA HV2008 WALL, AND 65 FEET SW OF SPEED. (Note: this forum software removes some spaces.) If you want my sed script, email me. If you want to get a sed.exe, put "sed" in an internet search engine and you can find a freeware sed for DOS. Quote Link to comment
+embra Posted June 18, 2004 Share Posted June 18, 2004 mloser directed me to TextHarvest; that seemed to do the job nicely once I read the brief help file thoroughly. Quote Link to comment
+YeOleImposter Posted June 18, 2004 Share Posted June 18, 2004 Yep, and my perl script does somethign real similar. But if you are not familiar with perl (ie don't know what it is) then TextHarvest that Embra mentioned probably will do what you want. I play with perl once a year to manipulate some file somewhere. This year it is the benchmark datasheets. Next year I will have to google myself through the basics again to remember how to do everything. I have to sell insurance during the day to feed my 7 kids (no joke) - and so I really don't have alot of time to keep up to date with this stuff. I usually go through spurts like creating the perl script to clean up the datasheets and the GSAK html output. Then I live with it until I just have to do something different. Quote Link to comment
+rusty_tlc Posted June 18, 2004 Share Posted June 18, 2004 There is a nice little program called Frogbit that is great for manipulating text files. It is pretty easy to use and a great way to get started doing simple programs. Quote Link to comment
mloser Posted June 18, 2004 Share Posted June 18, 2004 TextHarvest is available for Windows systems. It does a decent job of cleaning out lines. You can find it on just about any download site, like Download.com or Tucows. Quote Link to comment
+Black Dog Trackers Posted June 18, 2004 Share Posted June 18, 2004 I downloaded and tried TextHarvest. It's a nice tool. I got pretty much the same result with it as I did with sed. The only minor difference is that sed will also make changes within a line, like removing the _ characters, and re-fiddling such as I did with the HISTORY - MONUMENTED line. Sed can also be made to trim off the first 8 characters of each line, which I tried after the previous post. It saves a significant amount of toner or ink by not having the PID repeated on the beginning of each line. I'm not sure yet whether that will be a feature or bug in the field. If you're thinking of trying either perl or sed for the first time, I suggest sed. Sed is essentially a subset of perl, but perl requires some weird stuff at the beginning of a script and sed requires nothing. So, it's much easier to jump into sed. If you download perl5 you can always use the s2p utility to convert your sed script to a perl script. Quote Link to comment
+YeOleImposter Posted June 19, 2004 Share Posted June 19, 2004 I downloaded and tried TextHarvest. It's a nice tool. I got pretty much the same result with it as I did with sed. The only minor difference is that sed will also make changes within a line, like removing the _ characters, and re-fiddling such as I did with the HISTORY - MONUMENTED line. Sed can also be made to trim off the first 8 characters of each line, which I tried after the previous post. It saves a significant amount of toner or ink by not having the PID repeated on the beginning of each line. I'm not sure yet whether that will be a feature or bug in the field. If you're thinking of trying either perl or sed for the first time, I suggest sed. Sed is essentially a subset of perl, but perl requires some weird stuff at the beginning of a script and sed requires nothing. So, it's much easier to jump into sed. If you download perl5 you can always use the s2p utility to convert your sed script to a perl script. Only reason I leave the PID on each line is because I process the output through bmgpx and so it (probably) expects that on each line to do its conversion from the datasheet to a gpx file. I will have to look at sed - the only reason I use perl is because it can do what I need - and I don't need much Sed sounds like I won't have to relearn quite so much each time. Will go back and find out your recomendation for a sed program. Thanks Quote Link to comment
+Colorado Papa Posted June 19, 2004 Share Posted June 19, 2004 I use my sed script in a batch file that invokes it in DOS. It is for changing a file from the NGS into a shorter file. Hey, That looks great!! Will have to try it. Thanks for the tip! Quote Link to comment
+GeckoGeek Posted June 27, 2004 Share Posted June 27, 2004 Well, I wrote a (please don't laugh) QBasic program to do the trimming. Like YeOld I also feed the output to BMGPX. Quote Link to comment
+gnbrotz Posted August 16, 2006 Share Posted August 16, 2006 I'm wondering if anyone uses TextHarvest in combination with GSAK and/or pda? I'm running into some descriptions that are truncated when GSAK exports for CacheMate (due to CacheMate's imposed limits) and was wondering if I could possible shorten things up before bringing them in to CacheMate, so that the info I need (at the end) isn't pushed out by all of the 'extras' I don't need at the beginning of the description. Quote Link to comment
Team Fawlty Posted August 20, 2006 Share Posted August 20, 2006 (edited) I have created a rudimentary perl script as well to clean up the files. Mine outputs html. sample code I load these on my PDA when I go out. Brendan Edited August 20, 2006 by Team Fawlty Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.