How frustrating!
I don't think I would wash it. I also think you would damage the fusible part.
Try sprinkling baking soda on the area and putting it out in direct sunlight for a few hours then shake the baking soda off. This may help reduce the odor enough so you could work with it.
Although I am sure the smell would come back out (and possibly be set in) when you ironed the fusible fleece.
Good luck.