I don't know about rubbing off on fabric, but for very dry hands I've used Hoofmaker by Mane & Tail for years. I also take it to
mobile blood drives where we have to wash hands before and after wearing gloves, and hands get so dry and cracked with harsh cleansers. Everyone grabs at my bottle of it sitting on my table, and when it's gone someone else will bring in another one.
I've used it on horse's hooves, as makeup remover, body lotion, etc. Can't see not having it around. SaveMart has it, so do some drug stores, farm stores carry it.