I don't know if you still can, but you used to be able to buy the plug insulators at hard ware stores. I've seen them made from cardboard, plastic and something that looks like a mixture of cardboard and plastic. Maybe that's the dielectric paper board mentioned in the text Christy mentioned.
However for many years I've been making them out of cardboard stock like what is on the back of legal pads. So far I've had zero problems with that. Antique / vintage plugs come in lots of sizes and shapes so sometimes it's easier to make one than find one.
Joe