i've never had to do this but i've always had Citrasolv in the back of my mind if this problem ever happened to me. its a degreaser that isn't oil based.
i use this product all the time on fabric to transfer ink jet prints onto fabric so i know it won't stain or change the hand of the fabric in any way.
i just don't know how the fabric will stand up to the rubbing you will have to do to remove the wonder under adhesive.
i would soak a cotton ball with the Citrasolv and let it sit on the area for some time, then i would use the edge of a spoon to remove the softened adhesive, repeat as needed, and finally when it was all gone wash the fabric.
you can buy citrasolv at amazon.