Followers 0

Nominal Precision of WGS-84

Recommended Posts

Here is a musing I got started on and then didn't know what to do with it, so I'm posting it here.

WGS-84's precision is limited to its significant numbers, namely XXX° XX.XXX'. In other words how far apart are two points when the reading changes from XXX° XX.XX3' to XXX° XX.XX4'?

Given that there are 360° and 60' per degree and 1000 decimal minutes and the circumference of the earth is about 12,765 kilomters, then the answer is 59 centimeters. About one pace.

For one centimeter precision the reading would have to be XXX° XX.XXXXX' or the addition of two more significant figures.

fyi: I distinguish between precision and accuracy.

Not quite, it is around 6ft (in latitude), longitude lines get closer together as you go towards the poles, so the distance is COS(lat)*6ft.

An easier way to calculate this is to know that 1 minute of latitude = 1 nauticale mile = 6076.12 ft (in WGS84). So 0.001 minutes is 6 ft. One more digit gets you 0.6ft.

Rick.

Oops! My source for the circumference for the world was wrong! They said that the circumference was 12,765 kilometers, but they meant to say diameter. Not being very metric I didn't catch the discrepancy.

This 'pi' error is exactly how much my ~2' pace differs from your ~6' distance. The nominal precision of WGS-84 is 2 yards or 3 paces. Thanks!

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

×   Pasted as rich text.   Paste as plain text instead

Only 75 emoji are allowed.