If the answer is 'NO', advise of the consequences of refusal.
"If you refuse to remain/accompany me to a police station/place for the purpose of a drug impairment assessment you may be charged with this offence and if found guilty you may be fined or imprisoned for 18 months, and you will lose your licence for a minimum of 2 years.
Are you prepared to accompany me?"
If still 'NO' - Offence of 'Refuse to Remain/Accompany' is complete and a charge should be laid for that offence.