In what situation will an aircraft show you "LOC*" in FMA? What signal does the aircraft received when LOC*?(how does the aircraft know it is on LOC*) And which heading will the aircraft choose,is there any formula for the heading?
The Localizer signal is the horizontal (lateral) component of the full ILS signal (top portion of the image). It consists of two "lobes" which are amplitude modulated from each other, allowing the receiver to distinguish between left and right. The FMA reporting LOC has identified what it believes to be the ILS localizer signal, but NOT the glideslope signal (which is the same technique, but rotated 90-degrees so that the system is able to distinguish up from down, instead). You have side-to-side guidance along the runway centerline, but no guidance as to what altitude you should be flying at.
LOC* (as opposed to LOC) specifically refers to the "capture" phase. It's mechanically no different from LOC mode, but having a specific mode for the capture phase allows for a different set of logic to control the capture vs. the hold phase of the approach. The capture phase essentially exists between the moment you pick up the ILS signal until you've stabilized on the centerline the first time. After that I would expect the FMA to say "LOC" instead of "LOC*".
As to how the aircraft will maneuver itself in response to that signal, that will vary by aircraft model, possibly operator, and possibly even specific configurations of a given aircraft. Generally the heading for the capture maneuver is whatever vector you're given by Approach, until you've intercepted the beam itself. At that point the aircraft is turning to match the approach heading for the given runway but will shift as needed to stay centered on the beam. The maths for that are fairly complex. But if you're curious, here's an example method using H-infinity synthesis, if you want to dig into it.