What colour is the fabric it went onto? If it is a highly coloured fabric be careful you don't remove colour from the fabric trying to remove the ink. If you have scraps of the background fabric I would mark it with the sharpie and try different things.
Sharpie is difficult as it is meant to be permanent. I have had good luck removing REGULAR ink from fabric with hairspray. Some people have had good results with either a tide spot remover or a magic eraser but I don't know what will work with sharpie.