You can do a burn test on a tiny scrap (need only about 1/4 x 1 inch) to see if it's cotton or not.
Fabrics with polyester, nylon, or dacron burn with black smoke and leave either a hard or 'crunchy' ash.
Cotton leaves a powdery ash and does not put out black smoke.
Rayon is the 'iffy' one for me - I understand it's made of cellulose from wood - and that also leaves a soft ash.