# Wanted: Formulae for calculating distance between coordinates

## Recommended Posts

I'm trying to create my own distance calculator, but I don't know the formulae for calculating the distance between two sets of coordinates. IIRC, the distance that corresponds to a degree of longitude depends upon the latitude with this being greatest at the equator. I suspect that (because the Earth is flattened at the poles) that distance per degree of latitude also depends on the latitude. So it's not the simple Pythagorean calculation that would apply to a plane - i.e. distance = sqrt((x2-x1)^2 + (y2-y1)^2)

Help!?!

TIA,

Geoff

I'm trying to create my own distance calculator, but I don't know the formulae for calculating the distance between two sets of coordinates. IIRC, the distance that corresponds to a degree of longitude depends upon the latitude with this being greatest at the equator. I suspect that (because the Earth is flattened at the poles) that distance per degree of latitude also depends on the latitude. So it's not the simple Pythagorean calculation that would apply to a plane - i.e. distance = sqrt((x2-x1)^2 + (y2-y1)^2)

Help!?!

TIA,

Geoff

The distance between two waypoints is calculated using the great circle formula.

1. Distance = acos(sin(lat1)*sin(lat2)+cos(lat1)*cos(lat2)*cos(lon1-lon2))

where all lat and longs are expressed in radians

2. Angle between two waypoints = mod(atan2(sin(lon1-lon2)*cos(lat2),cos(lat1)*sin(lat2)-sin(lat1)*cos(lat2)*cos(lon1-lon2)), 2*pi)

where all lat and longs are expressed in radians

3. A Nautical Mile is 1/60th of a degree or one minute of latitude. Converted to feet by multipling by 6076.11549

or at least that's what google says

I'm trying to create my own distance calculator, but I don't know the formulae for calculating the distance between two sets of coordinates. IIRC, the distance that corresponds to a degree of longitude depends upon the latitude with this being greatest at the equator. I suspect that (because the Earth is flattened at the poles) that distance per degree of latitude also depends on the latitude. So it's not the simple Pythagorean calculation that would apply to a plane - i.e. distance = sqrt((x2-x1)^2 + (y2-y1)^2)

Most handheld GPSr devices can calculate these distances for you automatically, so don't make extra work for yourself

But the formula that clan_barron posted does work. I work for a retail company in Canada and actually used that formula in a batch program I wrote to calculate the distance to the nearest store for every postal code across the country. We have ~2500 locations and about 800,000 postal codes.. & it could do the whole country in about 4 hours

Thanks guys, much appreciated. I guess that I've now got to brush up on my maths (it's amazing what you forget if you don't use it for over a decade!)

Most handheld GPSr devices can calculate these distances for you automatically, so don't make extra work for yourself

Unfortunately, mine doesn't! Besides which, it's something I want to do out of personal interest and as a programming exercise.

Again, many thanks.

Geoff

I'm trying to create my own distance calculator, but I don't know the formulae for calculating the distance between two sets of coordinates. IIRC, the distance that corresponds to a degree of longitude depends upon the latitude with this being greatest at the equator. I suspect that (because the Earth is flattened at the poles) that distance per degree of latitude also depends on the latitude. So it's not the simple Pythagorean calculation that would apply to a plane - i.e. distance = sqrt((x2-x1)^2 + (y2-y1)^2)

Help!?!

TIA,

Geoff

The distance between two waypoints is calculated using the great circle formula.

1. Distance = acos(sin(lat1)*sin(lat2)+cos(lat1)*cos(lat2)*cos(lon1-lon2))

where all lat and longs are expressed in radians

2. Angle between two waypoints = mod(atan2(sin(lon1-lon2)*cos(lat2),cos(lat1)*sin(lat2)-sin(lat1)*cos(lat2)*cos(lon1-lon2)), 2*pi)

where all lat and longs are expressed in radians

3. A Nautical Mile is 1/60th of a degree or one minute of latitude. Converted to feet by multipling by 6076.11549

or at least that's what google says

Alternatively, to could you could use UTM coordinates and calculate the distance using the Pythagorean Theorem.

I'm trying to create my own distance calculator, but I don't know the formulae for calculating the distance between two sets of coordinates. IIRC, the distance that corresponds to a degree of longitude depends upon the latitude with this being greatest at the equator. I suspect that (because the Earth is flattened at the poles) that distance per degree of latitude also depends on the latitude. So it's not the simple Pythagorean calculation that would apply to a plane - i.e. distance = sqrt((x2-x1)^2 + (y2-y1)^2)

Help!?!

TIA,

Geoff

The distance between two waypoints is calculated using the great circle formula.

1. Distance = acos(sin(lat1)*sin(lat2)+cos(lat1)*cos(lat2)*cos(lon1-lon2))

where all lat and longs are expressed in radians

2. Angle between two waypoints = mod(atan2(sin(lon1-lon2)*cos(lat2),cos(lat1)*sin(lat2)-sin(lat1)*cos(lat2)*cos(lon1-lon2)), 2*pi)

where all lat and longs are expressed in radians

3. A Nautical Mile is 1/60th of a degree or one minute of latitude. Converted to feet by multipling by 6076.11549

or at least that's what google says

Alternatively, to could you could use UTM coordinates and calculate the distance using the Pythagorean Theorem.

Indeed use UTM, metric based, easy ! Chinese and Indian peasants should teach American how to !

Alternatively, to could you could use UTM coordinates and calculate the distance using the Pythagorean Theorem.

You got my interest. However, I suspect that UTM has even more issues than OSGB with respect to zone boundaries. I suspect that the same zone letter means the same latitude wherever you are in the World, but how do you deal with a change in numerical portion? For example, how do you calculate the distance between 30U 0632481, 5680051 and 31U 0385893, 5757484?

Also how do you deal with the problem of the overlap that arises from projecting the curved ground to a flat mapping? For example, you could have two sets of coordinates, with one from 30U being further east than one in 31U even though zone 31 is east of zone 30! It all seems instinctively very messy to me!

Geoff

Alternatively, to could you could use UTM coordinates and calculate the distance using the Pythagorean Theorem.

You got my interest. However, I suspect that UTM has even more issues than OSGB with respect to zone boundaries. I suspect that the same zone letter means the same latitude wherever you are in the World, but how do you deal with a change in numerical portion? For example, how do you calculate the distance between 30U 0632481, 5680051 and 31U 0385893, 5757484?

Also how do you deal with the problem of the overlap that arises from projecting the curved ground to a flat mapping? For example, you could have two sets of coordinates, with one from 30U being further east than one in 31U even though zone 31 is east of zone 30! It all seems instinctively very messy to me!

Geoff

Geoff - the solution that I suggested would work within a zone but probably would not work if the waypoints in question were in different zones.

Thanks, jmundinger. Your posts got me thinking that since I'm only interested in UK locations I might be able to use ten-figure OSGB grid references. For those who might not know, these are in the form of two letters followed by ten digits. For example, the OS grid ref of the middle of Tower Bridge, London (WGS84 51.50564, -0.07538) is TQ3367080291. The first five digits give the easting and the last five digits the northing in metres from the bottom-left corner of the 100km square that the letters identify. What I didn't know until today is that each letter of that identifier represents a position in a 5 x 5 grid with the letter "I" not being used, viz:

```	  A  B  C  D  E
F  G  H  J  K
L  M  N  O  P
Q  R  S  T  U
V  W  X  Y  Z
```

Each larger square is sub-divided in to a similar 5 x 5 grid. FWIW, square AA is somewhere in the North Atlantic and the OS actually use a false origin at the bottom-left corner of major square S. However, I'd prefer to use my own origin, which would be the bottom-left of major square V as this makes calculations a little easier. So it seems straightforward to calculate distances on this grid using just Pythagoras and a lookup function to convert grid letters to distances. That method's probably the best way forward for a non-geocaching application I have in mind, but I still need to be able to calculate distances directly from WGS-84 coordinates (whether DMS, decimal minutes, or decimal degrees). Thankfully, I've found a few examples of code in various languages that I can use to check my own efforts based on formulae this thread has given.

Thanks again to all,

Geoff

I'm trying to create my own distance calculator, but I don't know the formulae for calculating the distance between two sets of coordinates. IIRC, the distance that corresponds to a degree of longitude depends upon the latitude with this being greatest at the equator.

None of the replies given so far will give you accurate results. Spherical Trig (Haversine) formulas are adequate for very short distances, but since the Earth is not a sphere...

Using UTM coordinates and Mr. Pythagoras will give you accurate distances but not across zone boundaries and not with good bearings.

The good formulas with all the fudge factors are out there in the Googlesphere, or you can reverse engineer from this Javascript calculator. I just keep a copy of the page on my hard drive.

There is also Compsys21 from the FAA, which is very powerful and a Royal PITA (the pain not the bread) to use. Its value is in being able to calculate intersections and allowing you to save sets of coordinates and calculations as text files which can be re-used by the program.

I'm trying to create my own distance calculator, but I don't know the formulae for calculating the distance between two sets of coordinates. IIRC, the distance that corresponds to a degree of longitude depends upon the latitude with this being greatest at the equator.

None of the replies given so far will give you accurate results. Spherical Trig (Haversine) formulas are adequate for very short distances, but since the Earth is not a sphere...

Using UTM coordinates and Mr. Pythagoras will give you accurate distances but not across zone boundaries and not with good bearings.

The good formulas with all the fudge factors are out there in the Googlesphere, or you can reverse engineer from this Javascript calculator. I just keep a copy of the page on my hard drive.

There is also Compsys21 from the FAA, which is very powerful and a Royal PITA (the pain not the bread) to use. Its value is in being able to calculate intersections and allowing you to save sets of coordinates and calculations as text files which can be re-used by the program.

That page looks to me like it uses Vincenty's algorithm to calculate distances. If you click on the link provided by Redwoods Mtn Biker, I posted a link in that thread to http://www.movable-type.co.uk/scripts/latlong.html. That page actually displays the Javascript source for calculating distances using Haversine or Spherical Law of Cosines. Dig deeper, and it has a link to another page: http://www.movable-type.co.uk/scripts/latlong-vincenty.html <== the guy has implemented and shown the source code for the Vicenty formula.

I have some scripts which calculate distances between lat/lon points. For my purposes, the Vincenty formula is WAY overkill, as at most I care about distances about 3km away. But as is said in the quoted text, if you're planning on performing calcs of lat/lon pairs over much greater distances, then the Vicenty formula is probably the best (apparently, up to 1mm accuracy!). Don't be put off by the do/while loop, unless speed is what you need....

Again, thanks. FWIW those scripts on movable-type.co.uk are some I'd already found (or rather, been pointed to), but I'd planned to use those as-is on the movable-type site to check how good my approximations are!

For my purposes, Vincenty is also way overkill! I'm not interested in the bearing from one coordinate set to the other - just the distances; and to the nearest ten metres over a 2,500 km square patch is probably as accurate as I actually need. Converting to 10-figure OS grid refs gives accuracy to the nearest metre - so that should work well for UK. However, I'm intrigued by the entire subject - so I guess that I'm not going to stop when I've got what I really need!

Thanks again,

Geoff

## Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

×   Pasted as rich text.   Paste as plain text instead

Only 75 emoji are allowed.