Right after I use Roxanne's Baste-It bottle with the long needle tip, I run hot water over it/through it (might need to let it lay in hot water for a few minutes). Then I put my mouth against the cap end and blow through it to clear the glue and water. It is then stored off the bottle until the next use. Roxanne's comes with a lid to close it. I can also insert a long pin into the needle.
The bottles I bought at Michael's have the needle which screws directly onto the plastic nozzle of the bottle. I haven't taken it off, because I don't know if it would stay on the second time it was attached. A little wire comes with it, so I leave it in the needle. However, I think the wire will rust over time. Soooo....in my opinion, the best option is to buy Roxanne's glue in order to refill the bottle when empty.