You can leave the paper in until you sew the surrounding blocks and then remove it. This will prevent any bias edges from stretching. You don't have to leave it in beyond that. I, too, use the wrong edge of a seam ripper. It's easy to take out and perfect for when you are watching tv or just sitting down to have a break.