I did this once and that was enough. I ended up soaking my hands in a bowl of milk for some time because it felt good. I didn't know at the time that it helps to cut the capsacin.
I hope something works for you because I know how painful it is. Feel better soon.
My sons girlfriend says to try baking soda. It absorbs the oil out of your skin. New one on me but at this point if it was me, I would try anything.
Husband says at work they use handcleaner like they use in an auto garage to help remove poison oak when they get into it.
I keep a box laxtex gloves in the house just for this kind of thing. In the long run it is worth the investment.