I don’t think you will be able to remove it without messing up the blocks. If you use something like Goo Gone, if it removes the glue, it will probably leave an oily stain and that may not come out with washing. Your best option is to trim the blocks in my opinion.