I'm trying to decode ADS-B messages for a school project, and I cannot reproduce the results of the NL() function as described in this document. I've seen the same formula in several other documents. Also in this document is the reverse of this function - the NL is input, and the latitude "boundary" is the result. I cannot solve that either.

I've tried radians and degrees for the trig functions and several different calculation tools, and my result is never close to correct.

The "known" good inputs and result are from this document. I've also seen these inputs and results elsewhere, but no details about the equation itself.

The summary of the "known" good data set is:

$rLat = 10.2157745361328$

$NZ = 60$

$NL = 59$

The equation is:

$$ NL = int \left( 2 \pi \left[ arccos \left( 1 - { 1 - cos \left( {\pi \over 2NZ} \right) \over cos^2 \left( {2 \pi \over 180} \mid lat \mid \right) } \right) \right] ^{-1} \right) $$

Can anyone help me understand this calculation?

  • $\begingroup$ I'm pretty sure latex is supported here, at least I've seen questions and answers with that format before... $\endgroup$
    – Jae Carr
    Commented May 4, 2014 at 15:41
  • 2
    $\begingroup$ @JayCarr; well almost, we have MathJax, it's supports a lot of LaTeX's equation-stuff, so just paste it, it might very well render just fine. $\endgroup$
    – falstro
    Commented May 4, 2014 at 16:30
  • 2
    $\begingroup$ Hi Kevin welcome on Aviation.SE. I will try and help you, but from your description there is little to start from. What doesn't work, what language do you use? What is your background, what kind of school work is this for? $\endgroup$
    – DeltaLima
    Commented May 4, 2014 at 17:56
  • 1
    $\begingroup$ I added the equation - you might want to double check my work to make sure it's correct. $\endgroup$
    – Steve V.
    Commented May 5, 2014 at 1:33
  • $\begingroup$ NZ should be 15,see this code code.google.com/p/adsb-pgr/source/browse/trunk/… $\endgroup$ Commented Jun 12, 2015 at 22:51

1 Answer 1


Compact Position Reporting (CPR) is a way of reducing the number of bits needed to transmit position whilst maintaining high position resolution (~5.1 meters for airborne encoding). Ordinary encoding would require 45 bits, but CPR uses 35 bits thus saving 10 bits. To do so, the world is divided up into a number of zones, both for latitude and for longitude and the position within the zone is encoded. Since transmitting the zone numbers would cost as many bits as that would have been saved a clever trick is used. Two types of encoding are used, each with a different number of zones. From the difference in zone coordinates between the two types of encoding one can determine which zone the aircraft is in. Therefore one needs to receive an even and an odd encoded message within 10 seconds from each other. Once the initial position decoding is successful every subsequent position message will give the updated position.

In CPR decoding, you first determine the latitude of the aircraft based on a pair of odd and even position messages received within 10 seconds from each other.

Latitude is always encoded in the same number of zones between the equator and the poles (15 for even encoding, 14.75 for odd encoding). Due to the roundness of the earth the numbers of zones used for longitude encoding depends on the latitude. Longitude even encoding uses 59 zones at the equator and only 1 at the poles. Odd encoding uses always 1 zone less, except near the pole where 1 zone is used like in even encoding. The number of longitude zones used for even encoding can be found by the $NL()$ function.

The transition latitudes, i.e. where the number of zones for longitude encoding change, can be calculated by the inverse $NL()$ function. This function can be used to create a lookup table of transition latitudes which is more efficient to use than the original $NL()$ function.

Without knowing anything about your implementation it is difficult to assist further, the formula is depicted correctly in your question. Note that input latitude is given in degrees, but the trigonometric function are assumed to work on radians hence the $\frac{\pi}{180}$ factor.

Edit: on further inspection I noticed a mistake in the formula: it should be :

$$ NL = int \left( 2 \pi \left[ arccos \left( 1 - { 1 - cos \left( {\pi \over 2NZ} \right) \over cos^2 \left( { \pi \over 180} \mid lat \mid \right) } \right) \right] ^{-1} \right) $$

(note that the conversion from degrees to radians is corrected)

For those that really want to understand CPR coding and decoding in detail, you'll have to obtain a copy of RTCA document DO-260B or Eurocae document ED-102A (unfortunately they're expensive, I think about $700) and read Appendix A.1.7 and Appendix T.

  • $\begingroup$ where did you get that information from: Latitude is always encoded in the same number of zones between the equator and the poles (15 for even encoding, 14.5 for odd encoding). Especially the 14.5 for odd encoding instead of 15? Couldn't find it on the internet.... $\endgroup$
    – Florian
    Commented Aug 2, 2018 at 11:05
  • 1
    $\begingroup$ Hello Florian, welcome to aviation.stackexchange.com. I got the numbers from ED-102A or DO-260B. In fact the latitude coding is 360 degrees from the equator, over the north pole, then the backside of the earth and from the southpole up to the equator. Only the 0-90 degrees and 270-360 degrees are used. There are either 60 or 59 zones in the full circle, so 15 or 14.75 from equator to pole. I know realise that the 14.5 I wrote is a mistake, it should be 14.75. Thank you for notifying! $\endgroup$
    – DeltaLima
    Commented Aug 2, 2018 at 11:33

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .