visit Paperpieces.com
they have all of the supplies you need for all kinds of english paper piecing- hex's, diamonds, applecores; many to choose from.
they also have good free content- tutorials, and free design sheets you can print off then color with colored pencils to see exactly how your quilt will look.
and to answer the question- when do you remove the templates? when you will no longer add anything more to it...that is when the papers come out.
i use both plastic and paper pieces- the plastic ones are very expensive- the paper ones work just as well for a fraction of the cost.