There are many examples where the PF will place the thrust levers at idle and the "retard" command will continue sounding. Here are some examples: 1 and 2. Note that I am not referring to the singular "retard" command that will sound between 20 and 10 ft regardless of the A/THR status or thrust lever position (here is an example of that).
Simply because the callout always occurs at 20 ft radio height, regardless of the position of the levers. Therefore, idling the levers also does not interrupt the callout. The callout is only a reminder to the pilots, not an order according to the A320 Flight Crew Training Manual: