I buy Organ brand needles from Amazon in boxes of 100 for about $10-$12. It's 10 packs of 10 needles. Great deal and my machine loves 'em.
I like the painter's tape idea. I do something similar for Xacto blades when I need to switch to a different type of blade and the blade coming out of the handle is still good; I painter-tape them to the underside of my glass drawing table. The blade is in clear sight so I don't lose it, and safe where I won't jab myself with it.