Enlighten Me On Adjusted Marks...

[ckhd]

I know that the coordinates for "adjusted" marks are more accurate than my GPS. My question is: how do they "adjust" a mark? Do they already have accurate coordinates from when the mark was set and they are just changing it from NAD27 to WGS82, or do they figure it out somehow?

 

Just curious...

[Bill93]

Rob just posted an explanation under the thread on "More Recently Set Marks".

[Z15]

fyi

 

From NGS..

 

DATA PROCESSING IN THE NATIONAL GEODETIC SURVEY

by Maralyn Vorhauer

Since its inception in 1807, the National Geodetic Survey (NGS) and its predecessors, have always strived to produce the most accurate, highest quality geodetic data for its customers. Initially, and for well over 150 years, this meant painstaking, meticulous manual computation of coordinates and other geodetic quantities. It also meant that the quantity of data was limited and the time it took to produce results lengthy, since this oldest scientific agency in the U.S. Government was never large nor generously funded.

 

The past 10 years have seen what can only be described as an explosion in technology and a sharp increase in the amount of data flowing into the agency both for processing and dissemination. NGS has had to work ever more efficiently and take extensive advantage of new technology to even begin to meet these demands on its resources. This has never, however, lessened the commitment of NGS employees to produce products which users can depend on to meet their needs whether for land surveys, airplane navigation or Geographic Information Systems (GIS) applications. This commitment to excellence has required the development of extensive computer software and a set of procedures for use in-house and, increasingly, outside NGS, to assure the accuracy and reliability of its published data. It has also been necessary to limit the types of data which we can include in our databases and the formats which we can process and load into these databases.

 

This article deals with the general format, computations and checking of control data which is submitted for inclusion in the National Spatial Reference System as defined by the NGS data base (from which it is distributed to the public in one form or another). The data may come from NGS field parties, NGS contractors, the numerous federal, state, and county agencies or the private sector, all of whom submit field observations to NGS headquarters. But for all, the procedures and products are the same and rather stringently adhered to. This assures the consistently high quality of the data as it is published regardless of the source. All software programs mentioned in this article are available from NGS, either by calling the Information Center at 301-713-3242 or from the NGS home page on the World Wide Web - http://www.ngs.noaa.gov. The established standard for the submittal of data to NGS is found in the document entitled "Input Formats and Specifications of the National Geodetic Survey Data Base". This document, available from the Information Center, also details how and to whom to submit data or call for specific information.

 

This extensive document, usually referred to as the "Bluebook" because it was originally distributed in blue binders, defines every piece of information required for submittal of data to NGS and the format of this information. It defines formats for classically observed data (usually called the t-file), Global Positioning System (GPS) data (b-files and g-files), and descriptive data (d-files). Programs have been developed which expedite the production of the digital files in this format. Most manufacturers of GPS equipment have available software to produce the b-files and g-files necessary to meet NGS format requirements. NGS programs, MTEN- used to produce t-files- and DDPROC- used to produce descriptive data files- are available. Programs have also been written at NGS to check that the range and format of the data in the files are correct. Submittors may or may not use the programs to produce the files but all files must be checked using COMPGB (GPS data checking), CHKOBS (used to check classical t-files), OBSCHK (used to check b-files and g-files and to compare the information in each), CHKDESC (used to check d-files), and OBSDES (used to compare the b-file and d-file) and outputs of these programs are required in the data submittal package. Usually if the available software is used to produce the bluebook files, the programs will give few, if any, errors and expedite the submittal process. These programs have been written not just to check the bluebook files but to work in conjunction with the data base update software so that these routines will be free from time consuming errors when the project is completed and loaded.

 

Of course, the observational data are only the means to the end products. Geodetic coordinates, including latitude, longitude, orthometric height and where appropriate, ellipsoidal height, which result from these observations are the quantities which become the products used by consumers. Computation of these values formerly were handled by office staff with extensive geodetic backgrounds and years of experience. Today, facilitated by computers, sophisticated software, and ever increasing educational backgrounds, almost everyone involved with the collection of geodetic data have the means and resources to compute highly accurate and reliable values for geodetic quantities. These quantities are usually computed using least squares statistical methods-called adjustments'.

 

Many least squares adjustment software programs exist which can be used to accurately compute the coordinates. However, files of data submitted to NGS must have coordinates which are the output of the software program ADJUST. The question often arises as to why NGS requires that its own adjustment software, ADJUST, be used when results from others are quite similar or identical. The answer is that, to the best of our knowledge, ADJUST is the only program that uses the b-files and g-files. Given ever dwindling personnel resources in NGS, files in other formats cannot be accepted and the quality checking provided by using ADJUST with these files cannot be provided. Prudence dictates that if NGS is to provide the seal of approval' implied by publishing coordinates under its auspices, that we be confident that these coordinates are correct and, further, that supporting observations are available if future changes to these coordinates are required. ADJUST outputs can be easily reviewed for correct values and procedures. Submittal of these adjustments has been required for data submittal for a couple of years and has been the major reason that NGS can continue to provide the data base storage, distribution and quality assurance of its published data services currently available free of charge.

 

Before giving some details on NGS procedures for adjustments, especially GPS adjustments, it is probably important to outline what type of data is routinely accepted by NGS. None of the above discussion applies to leveling data. Although in principle, the same rules' apply and a bluebook is available defining the formats for submitting leveling data, portable software which can be used for level data digitizing, format checking (for conformance to the leveling formats and specifications similar to the horizontal data bluebook'), and adjustments has not been developed and is presently confined to NGS in-house machines. Users wishing more information on submitting leveled observations should contact NGS for more specific information. All horizontal data submitted must be observed to current first-order standards. Although not limited to GPS data, in fact no first-order data observed by classical methods have been collected by NGS for many years (since 1985, in fact) nor has any been submitted. This effectively results in all data submitted being GPS data. Therefore the following discussion will apply to GPS data; while the steps used also apply to classically observed data, the order of processing would be different or modified.

 

The first step in the adjustment process is to run a minimally constrained (free) adjustment holding fixed one position and one height. The ADJUST input and output is controlled by a file (A-file) which includes the parameters for the adjustment and the specific constraints to be used (CC records). If geoid heights have not been added to the b-file, then the height held should be an ellipsoidal height; if they have been added, it should be an orthometric height. ADJUST software is designed to expect orthometric heights to be provided in the A-file, and so in order for the program to produce valid results, the height used must match the presence or absence of geoid heights in the b-file. Without geoid heights, the ellipsoidal height equals the orthometric height and will be treated correctly in the software. The adjustment is run in three dimensions (DD3 record), without the option of scaling the results (MM record). It is usually helpful to include the options to print the full input b-file and g-file (PP record in the A-file) as a record of what information was included in the adjustment. Accuracies may be included (QQ records) to help determine that the internal consistency of the project has met the intended accuracy. At this point in the adjustment process, external (or network) accuracies are meaningless. (The internal accuracy of a line is the first accuracy given in the output; the external accuracy is the second.) The output residuals can be analyzed for blunders, redundant vectors that differ by more than about 5 centimeters (possibly pointing to the need for additional observations because of poor centering on the mark or atmospheric interference), and height residuals that are above about 8-9 centimeters ( possibly indicating a problem with antenna heights). Output positions can be compared with the published values to determine if the control is adequate. Most reduction software produces standard errors that are overly optimistic to assign to the vector components so a high variance is not always a good indicator of problems with the survey. High residuals throughout the free adjustment should be a cause for concern and a look at the reductions and possibly the field observations.

 

After a satisfactory free adjustment is run, program MODGEE is run to scale the standard errors assigned to the vectors by the standard deviation of the free adjustment (given near the end of the ADJUST output, before the listing of the adjusted positions). This results in all first-order projects having essentially equivalent standard error assignments when they may be combined in the future. Higher order projects are not scaled. A new free adjustment is usually then run to verify that the resulting variance is very close to 1.00 indicating proper scaling.

 

Two adjustments are run on the b-file to determine the final coordinates. What NGS usually refers to as a horizontally' constrained adjustment is run to determine latitude, longitude, and ellipsoidal height. A vertically' constrained adjustment is run to determine orthometric heights. This dual adjustment procedure has been the practice since NGS began to process GPS observations. It is necessary to ensure that both the orthometric and ellipsoidal heights are computed correctly. They are independent quantities and separate adjustments provide the mechanism to compute each one as such.

 

The horizontally constrained adjustment is run holding fixed NGS published North American Datum of 1983 (NAD83) positions. Marks which have published positions less than first order or positions not determined by GPS are usually allowed to float - such constraints are seldom as accurate as the GPS positions and observations and exert undue stress on the network. If shifts are found to be very small (less than 4-5 centimeters) for these points and the accuracies produced when holding them fixed meet project specifications, then the adjustor has the option of holding fixed the position simply to reduce the number of unnecessary positional changes at a point over time. This, however, is rarely the case for such marks. On the other hand, points which are B-order and above and first-order positions determined from these high accuracy positions can almost always be held fixed and produce the needed accuracies. These determinations are made by examining residuals in the adjustment, the external accuracies resulting from a free adjustment using with the final output positions from the horizontally constrained adjustment, and shifts of positions which are not held. Several adjustment runs may be necessary with various constraints before the adjustor is satisfied with the results. Not to be overlooked are the ellipsoidal heights. Their fit' with previous heights are not reflected in the accuracy listing and residuals must be examined manually to determine whether previously published values should be held. The older the HARN or the previous first-order survey, the more likely it is that the heights will result in higher (i.e. decimeter or more) residuals. Until the completion of the effort to reobserve the HARNs throughout the country using stringent methods developed to upgrade the height component of the vectors, this situation is likely to occur. The adjustor must determine the importance of the ellipsoidal heights to the survey to make a decision whether to extensively or selectively readjust these heights or leave them as currently published.

 

The orthometric height adjustment is run and analyzed similarly. The software program, GEOID96, should be used to add geoid heights to the bluebook. Usually only previously published, leveled elevations are held fixed. One horizontal position (latitude and longitude) is held fixed and the 3-D option used for the adjustment. If the adjustor or client is interested in using orthometric heights previously determined by GPS and since most GPS determined orthometric heights are only published to the nearest decimeter, NGS can be contacted to obtain the full two decimal place value stored in the database. Seldom are previously determined trigonometric values held. Studies are underway and criteria being formulated for the conditions under which a GPS survey can be expected to yield GPS derived orthometric elevations at the centimeter level. Until then, if such accuracies are desired for the orthometric heights, contact NGS or the local state NGS advisor for help in planning the survey to meet such specifications. Currently the decision regarding whether the heights meet the more stringent criteria is made on a case by case basis. The analysis of this vertical adjustment, however, is similar to the horizontal adjustment analysis, a review of the control, the residuals and the shift in elevation when it is not held fixed. The extent of this analysis often depends on the specifications for the survey but as the heights become more and more reliable, it can be expected that this phase of the project will take on more importance.

 

Since two files have now been produced, one with the final adjusted orthometric heights and one with the final latitudes, longitudes and ellipsoidal heights, and only one file can be used to update the data base, the files must be combined. Program ELEVUP provides a very easy means of accomplishing this task. Note that the output b-file from the vertical adjustment is used with the positions and ellipsoidal heights from the horizontal adjustment output b-file replacing those values. No format checking is performed and no other quantities are replaced or combined.

 

Final values for the accuracies can now be computed using this file. A minimally constrained adjustment is executed including the QQ records in the A-file. Once again as noted above, and, since geoid heights are present in the final file, an orthometric height must be held fixed. Holding a substantially different height (as would be the case with an ellipsoidal height) results in significantly lower accuracies and possibly incorrect conclusions about the results of the survey. The output of this adjustment is also used as input to the program ELLACC which gives an estimate of the order and class of the ellipsoidal height. This information must be entered in the final b-file; generally, one order and class is given to the entire survey based on which one most of the lines fall into. (This information is entered in the *86* record in the bfile.)

 

The final step, and probably one which is best implemented as the project is processed and analyzed, is an extensive writeup in the field report of the results of the adjustments, the analyses, and the conclusions reached. List published positions or heights not held fixed and why, error messages remaining in the format checking software outputs, unusual situations (an example might be an underground mark used for observations or a station which appears from the results to have been moved but no other evidence exists), vertical control which came from other than NGS sources (allowed but must be documented), etc. Often these details provide valuable documentation when the project is initially reviewed by NGS as well as in the years to come when questions arise about specific situations.

 

While this article could only deal with the general outline of the procedures used, the employees at NGS are more than anxious to provide answers to specific questions. NGS is committed to accepting the processing and decisions made by the submitter when satisfactory results are achieved and documented. In fact, it has been our experience that if these steps are followed with attention to detail, little additional effort is needed when the project is received by NGS to review it and update the database. The user can also be assured that the results will not change and can be used immediately and NGS can be assured that no better results could be achieved by in-house processing. Following the basic outline above provides both NGS and the data user with the quality of data they want and have come to expect from NGS.

 

--------------------------------------------------------------------------------

Edited October 31, 2004 by elcamino

[+Black Dog Trackers]

For those who don't want to really get into the math:

 

The key phrase here is "least squares adjustment".

 

What is that?

 

Imagine your job is to survey some telephone poles' positions on a curvy stretch of road. You start at pole X17 and survey from pole to pole, 5 poles on one side of the road, across the road at the end, and back along some other poles on the other side of the road and finally back across the road to x17. You plug all the data in to a computer and find that you have 2 positions for x17, even though you made zero blunders. This is due to natural (statistical) survey error - the measurements cannot be totally exact.

 

Obviously, not only do you have 2 values for the position of x17, you realize that wereever you begin in your loop you surveyed, the last pole will be seen to have 2 different calculated positions. What to do? The answer begins when you arrange all the measurements of all the poles in a system of equations, each depending on the others' values. You get a list of x equations with x unknowns - a matrix. You get a computer to solve the matrix with a matrix algebra program. The solved matrix distributes the survey error to each telephone pole in a mathematical way so that each pole has only one set of coordinates (position). These are the adjusted coordinates.

 

The least squares terminology refers to the type of mathematics used in the algebra to solve the matrix. Basically, in statistics (and survey errors are a statistical phenomenon), one must use the squares of differences in positions, not the differences themselves.

[evenfall]

This is a great thread! Excellent info!

 

It should Also be noted that the NGS only adjusts, or converts to NAD 83 and later derivatives which are updates to NAD 83, such as NAD 83 (1991) and the like as an example.

 

NAD 83 is the North American Datum of 1983 and it is owned by the National Geodetic Survey. They created it and they work to improve it's accuracy.

 

We should make sure we are setting our GPS to NAD 83 Datum when we hunt for Benchmarks in Geocaching, because it is the correct Datum to use. While it is true that we will likely be exceeding the accuracy of a consumer grade GPS device, it doesn't make sense to further degrade it either. There is a difference between WGS 84 and NAD 83 which is equal to approximately +/- 1 meter at earth center, which is something, depending on where you are on the earth, which can add or subtract 1 meter of error to the accuracy of your GPS both vertically and horizontally, all at once, if you are using the wrong Datum for what you are trying to do. The amount of error you get will vary from locale to locale. The amount and direction the error which is present when the wrong datum is selected will be hard to determine with a consumer device, but it will be present none the less.

 

The WGS 84 Datum, Which superseded the WGS 72 DATUM is known as the World Geodetic System Datum. It is owned by the US Department of Defense and maintained by the Defense Mapping Agency.

 

For those who would like to familiarize themselves with definitions for many of the items listed on NGS Datasheets, a good place to learn is here. There is more to the quality of a Survey Monument than just Adjusted.

 

Rob

[Z15]

Least Square - a method of deducing from a number of carefully made yet slightly discordant observations of a phenomenon the most probable values of the unknown quantities.

 

Note: It takes as its fundamental principle that the most probable values are those which make the sum of the squares of the residual errors of the observation a minimum.

 

Source: Websters Dictionary

 

THIS explains it is simple terms

Edited October 31, 2004 by elcamino

[evenfall]

Mike,

 

I went to that link and was blazing along until I came to the The Jacobian Matrix, where I was met by a man named Morpheus, who offered me the choice of either a Red, or Blue Pill. Then my pencil broke.

 

Now if we climb into the time machine and go back to say, the year 1900. The Geodetic Surveyors have a 90 foot wooden tower built over the top of a station which is to become a First Order Triangulation Station. They are not near a town, not really, and there is a heavy theodolite at the top of the tower they built from lumber they logged and cut near the site. They have 12-16 separate triangulation observations they have to observe to qualify this Station for first order quality. They are living in tents and use horses and wagons. They cook and eat camp style, and in fact they are likely living off provisions they brought. They have to feed the horses as well. Maybe it is a crew of 10 - 15 guys and some have to ride to the other locations out there somewhere to other towers built over other stations to help make observations. It is a wet October going into November, and it is not pleasant. Yet there are a few people on the crew who with a pencil and paper are doing, and doing all day long, some derivative of these least squares equations to bring the various triangulation's they have done to the one Latitude and Longitude they determine this Station to be at.

 

And then they do it again.

 

Today, we set up a GPS for a few hours, let the data collector have it all, pack up, drive back to the office and hook the data collector to the computer where we can parse the data any number of different ways before we drive home to our warm homes. Sure there is hard stuff but comparatively?

 

It kinda makes me feel glad I have it this easy.

 

Can someone loan me a pencil? :-) Oh, and um, a lot of paper too? NOT!

 

Rob.

[Z15]

How could they do it without computers?

 

Until they came up with an "early out" in 2002, I had 3 top-of-line Leica GPS 500 receivers, a Geodimeter Robotic total station, a Wild Digital level and still thought it was to much work for 1 man. Then my ORG installed a Co-op CORS network statewide and NGS launched OPUS. So I would set my 3 receivers to turn on a at certain time (noonish), collect data for 5.5 hrs and they shut off. All I had to do was to remember to record the HI at the start and end, convert the raw data to Rinex and send it to OPUS and wait the few minutes to get back state plane, dump that back into the Leica software, adjust my network with the OPUS positions and go out the next day and RTK all the required control for the highway work. It would take a week or 2 to set control, do the observations and set intermediate control for the consultants to do the mapping work.

 

Here is a photo of the Leica GPS during the HARN re-observations in 2001. I had to spend a total of 16.5 hrs at this one point over 2 days, two 5.5 hrs sessions on day 1 and one on day 2. IMAGES HERE

 

Edited November 1, 2004 by elcamino

[DaveD]

Just an update to evenfall's post about WGS 84 "belonging" to the Defense Mapping Agency (DMA). On Oct 1, 1996 DMA was renamed the National Imagery and Mapping Agency (NIMA) which was again renamed on November 24, 2003 to be the National Geospatial-Intelligence Agency (NGA). It's hard to keep up with these guys. If you want to know more about WGS 84 they have a publication – TR8350.2 that has more than anyone ever wanted to know. -- ftp://164.214.2.65/pub/gig/tr8350.2/wgs84fin.pdf

[+GEO*Trailblazer 1]

I tried the Link twice and got no where then I looked at the url should it not be http instead of ftp?

 

What DaveD said

 

I tried with the http and can not get it to link here.

 

second edit

And now the link is working

Edited November 2, 2004 by GEO*Trailblazer 1