I remove mine when all sides of the hexagon are attached to another one. If I am working on a small project and do not need to reuse the hexies then I take them out all at once.
HEXIE TIP - When I want to reuse the paper insert I give them a squirt with my water bottle and press with a not iron. It get then to that stiff point again. I usually can use the same paper piece 3 -4 times.