I agree with Joe, try heat on the epoxy and if that doesn't work you probably will have to carefully sand/grind it off.
I've removed a couple of the broken brush caps before. First you score the remaining piece so that it has another slot to fit a screwdriver. It's takes awhile and you need to find an object that does a good job of it. Work carefully and be sure to get the slot deep enough or you will just strip it out again.