Spherical Earth and Great-Circle Distance

For routing purposes, Earth is treated as a sphere. Although it bulges slightly at the equator, using the mean radius

$$R \approx 6371 \text{ km}$$

is accurate to well under 1% — more than good enough for flight planning.

A surface point is specified by two angles:

• Latitude $\varphi$: angle north (+) or south (−) of the equator, with $\varphi \in [-90°,\, 90°]$.
• Longitude $\lambda$: angle east (+) or west (−) of the prime meridian, with $\lambda \in [-180°,\, 180°]$.

A great circle is the intersection of the sphere with a plane through its center. Every great circle has the same radius $R$ as the sphere. The shortest path on the surface between two points lies along the unique great circle through them, so we call its arc length the great-circle distance.

If two points subtend a central angle $\Delta\sigma$ at Earth's center, then the great-circle distance is

$$d = R \cdot \Delta\sigma,$$

with $\Delta\sigma$ expressed in radians. To convert from degrees to radians, multiply by $\dfrac{\pi}{180}{:}$

$$\Delta\sigma_{\text{rad}} = \Delta\sigma_{\text{deg}} \cdot \dfrac{\pi}{180}.$$