The priority button is not normally used when taking over controls from the other pilot. Even in Airbus the control hand over is done verbally. The pilot who wants the control says, 'I have controls' and the pilot who gives the control says, 'You have controls.' Or if the flying pilot wants to give the controls to the pilot monitoring, he/ she says, 'You have controls' and the pilot monitoring confirms it by saying, 'I have controls.'
As mentioned in other answers, the priority button when pressed gives side stick control to the pilot who presses it the last. If you keep holding the button and pass 40 seconds, the other side stick deactivates. It can only be activated by pressing the priority button of the other stick. The main problem with the side stick control of Airbus aircraft is not that it is mechanically not linked. The problem is how the aircraft behaves when two inputs are given at the same time. In Airbus fly by wire, the side stick inputs are algebraically added. This means, if two pilots give an opposite and an equal demand, the aircraft may not even react to the controls. Or worse, if the two pilots were to give an identical input (in the same direction) the aircraft may overreact to the controls. Below is a picture which shows this behavior. As you can see, the left pilot gives a 20% left side stick demand, while the right pilot gives a 30% left demand. The aircraft adds up the inputs and the net input becomes a 50% left roll demand.

What are the risks associated with dual inputs? The main risk is that the aircraft might behave inappropriately to what the pilots want. For example, in the flare to the landing, if two pilots put in inputs to the side stick, the aircraft may under flare (opposite inputs) and the result can be a hard landing. Or if the inputs are given in the same direction (nose up), the resulting over flare may cause a tail strike. This is where the priority button becomes extremely crucial. If you for instance see the other pilot not flaring during the landing, you say I have controls, press the priority button and do the flare yourself. This ensures the other pilot can no longer control the aircraft. This is particularly important in training flights with a new pilot who for example fails to flare at the right time. Many a times new pilots tend to be nervous (natural human behavior) and even when the instructor says he has controls (after seeing what is happening), because of nervousness the trainee can unknowingly manipulate the controls. This can develop into a very dangerous situation where neither of the pilots know who has the controls. The priority button really helps in these type of dynamic situations. To ensure that the pilots are aware that there is a dual input, if the inputs meet a certain threshold, a 'dual input' aural alert comes on.
In case of Capt. Sully, we cannot be entirely sure why he pressed the priority button after saying he has controls. It might be because they were in a dire situation (dual engine flame out) and he did not want his first officer to unknowingly add any inputs to his inputs. Usually, in normal flights as said above changing over controls do not require the use of priority button.