To delete a recurrency, the user needs to be the assignee of the task or ‘ProjectPlanning’ permissions in general or in the project the task is related to.