Hello,
You can replace the ex-officio with co-opted trustees if your governing document allows this. If this is not the case, you will need to change your governing document allowing your board to elect trustees instead. It could be done fairly quick and easy depending on your legal structure.
The Charity Commission can give you a free advice on the phone on how to proceed correctly. I am happy to share my experience as we had similar issues.
Good luck.
Lana
robertthompsoncharities@gmail.com