It is a pain, I usually give them to my adult daughter to remove. Some things in the future to make it easier: always use a larger needle(#14, it makes bigger holes), shorten your stitch length and use flat head pins. You can use tweezers to rip out those difficult corner pieces of paper. I have found it better to rip out the paper as I make them, I hate to have to tackle a huge stack at once.