I buy bars of soap, wrap with a layer of batting, then wrap with strips of fabric the width of the wide side of the bar -- kind of like putting a ribbon on a package. Tie the strip on top with a knot and trim it. It leaves a bump on the bottom, but is pretty stable. Any kind of soap will do. I usually use Ivory. I don't remove the paper wrapping before I wrap with batting. When you first put your pins into the whole bar, be careful, because the soap is somewhat hard and you can bend your pins. Once the soap starts to breakdown, it becomes much easier.
I also make 4-1/2" 4-patches and sew them together, turn and fill. I have used sandbox sand (it is heat treated to sterilize it), but it comes in large bags, so isn't practical unless you are either making a lot of pincushions or have a sandbox you can dump the rest into. I have also used the crushed walnut shells and haven't had any problem with them. I haven't tried the kitty litter, because I don't have a kitty. I don't use anything scented, in case someone has a sensitivity to some scent.