I don't know if you've solved your problem yet but, if not, here are a couple of suggestions.

IMPORTANT NOTE: Both of the things I am suggesting below should be used in a VERY well ventilated area and preferably outside. Also, I would put a small amount into a small contain and use a piece of clean white fabric to dab it onto the outside edge of the blocks to see if works. If so, I would continue in that manner.

Suggestions:
I have used Goof Off and had very good luck removing residue and tags off many items. I do use Goo Be Gone but had that leave an oily residue on a couple of items.

Another thing I would try is Acetone. I had some upholstery and drapery fabric sample books given to me and there was some type of stiff paper or cardboard glued to the fabric. I couldn't find anything to remove the paper until I tried acetone.