I'm trying to decode ADS-B messages for a school project, and I cannot reproduce the results of the NL() function as described in this document. I've seen the same formula in several other documents. Also in this document is the reverse of this function - the NL is input, and the latitude "boundary" is the result. I cannot solve that either.

I've tried radians and degrees for the trig functions and several different calculation tools, and my result is never close to correct.

The "known" good inputs and result are from this document. I've also seen these inputs and results elsewhere, but no details about the equation itself.

The summary of the "known" good data set is:

$rLat = 10.2157745361328$

$NZ = 60$

$NL = 59$

The equation is:

$$ NL = int \left( 2 \pi \left[ arccos \left( 1 - { 1 - cos \left( {\pi \over 2NZ} \right) \over cos^2 \left( {2 \pi \over 180} \mid lat \mid \right) } \right) \right] ^{-1} \right) $$

Can anyone help me understand this calculation?

  • $\begingroup$ I'm pretty sure latex is supported here, at least I've seen questions and answers with that format before... $\endgroup$
    – Jay Carr
    May 4 '14 at 15:41
  • 2
    $\begingroup$ @JayCarr; well almost, we have MathJax, it's supports a lot of LaTeX's equation-stuff, so just paste it, it might very well render just fine. $\endgroup$
    – falstro
    May 4 '14 at 16:30
  • 2
    $\begingroup$ Hi Kevin welcome on Aviation.SE. I will try and help you, but from your description there is little to start from. What doesn't work, what language do you use? What is your background, what kind of school work is this for? $\endgroup$
    – DeltaLima
    May 4 '14 at 17:56
  • 1
    $\begingroup$ I added the equation - you might want to double check my work to make sure it's correct. $\endgroup$
    – Steve V.
    May 5 '14 at 1:33
  • $\begingroup$ NZ should be 15,see this code code.google.com/p/adsb-pgr/source/browse/trunk/… $\endgroup$ Jun 12 '15 at 22:51

Compact Position Reporting (CPR) is a way of reducing the number of bits needed to transmit position whilst maintaining high position resolution (~5.1 meters for airborne encoding). Ordinary encoding would require 45 bits, but CPR uses 35 bits thus saving 10 bits. To do so, the world is divided up into a number of zones, both for latitude and for longitude and the position within the zone is encoded. Since transmitting the zone numbers would cost as many bits as that would have been saved a clever trick is used. Two types of encoding are used, each with a different number of zones. From the difference in zone coordinates between the two types of encoding one can determine which zone the aircraft is in. Therefore one needs to receive an even and an odd encoded message within 10 seconds from each other. Once the initial position decoding is successful every subsequent position message will give the updated position.

In CPR decoding, you first determine the latitude of the aircraft based on a pair of odd and even position messages received within 10 seconds from each other.

Latitude is always encoded in the same number of zones between the equator and the poles (15 for even encoding, 14.75 for odd encoding). Due to the roundness of the earth the numbers of zones used for longitude encoding depends on the latitude. Longitude even encoding uses 59 zones at the equator and only 1 at the poles. Odd encoding uses always 1 zone less, except near the pole where 1 zone is used like in even encoding. The number of longitude zones used for even encoding can be found by the $NL()$ function.

The transition latitudes, i.e. where the number of zones for longitude encoding change, can be calculated by the inverse $NL()$ function. This function can be used to create a lookup table of transition latitudes which is more efficient to use than the original $NL()$ function.

Without knowing anything about your implementation it is difficult to assist further, the formula is depicted correctly in your question. Note that input latitude is given in degrees, but the trigonometric function are assumed to work on radians hence the $\frac{\pi}{180}$ factor.

Edit: on further inspection I noticed a mistake in the formula: it should be :

$$ NL = int \left( 2 \pi \left[ arccos \left( 1 - { 1 - cos \left( {\pi \over 2NZ} \right) \over cos^2 \left( { \pi \over 180} \mid lat \mid \right) } \right) \right] ^{-1} \right) $$

(note that the conversion from degrees to radians is corrected)

For those that really want to understand CPR coding and decoding in detail, you'll have to obtain a copy of RTCA document DO-260B or Eurocae document ED-102A (unfortunately they're expensive, I think about $700) and read Appendix A.1.7 and Appendix T.

  • $\begingroup$ where did you get that information from: Latitude is always encoded in the same number of zones between the equator and the poles (15 for even encoding, 14.5 for odd encoding). Especially the 14.5 for odd encoding instead of 15? Couldn't find it on the internet.... $\endgroup$
    – Florian
    Aug 2 '18 at 11:05
  • 1
    $\begingroup$ Hello Florian, welcome to aviation.stackexchange.com. I got the numbers from ED-102A or DO-260B. In fact the latitude coding is 360 degrees from the equator, over the north pole, then the backside of the earth and from the southpole up to the equator. Only the 0-90 degrees and 270-360 degrees are used. There are either 60 or 59 zones in the full circle, so 15 or 14.75 from equator to pole. I know realise that the 14.5 I wrote is a mistake, it should be 14.75. Thank you for notifying! $\endgroup$
    – DeltaLima
    Aug 2 '18 at 11:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.