I normally leave it in to protect the edges of the blocks which often are bias edges. I take mine off when the piece is bordered by a non paper pieced edge of fabric.
Note: some authors suggest stay stitching in the seam allowance to stabalize but if you do this when the paper is on the blocks, taking off the paper is much harder on the fabric of the edge. If you do it, test a block first. In fact, with this project, I would test all on a sample block so you know how it works.