I do this all the time. Took a class on Quilt University using teas, spices, onion skins to dye with.
LOVED IT! I have used other plant material as well. You just have to remember that it will NOT be permanent if you do not use a Mordant! Alum is the most widely used with tea! Though even salt and vinegar can do it!