American Folk Art Museum -
http://www.folkartmuseum.org has an exhibit thru 7/8 of quilt masterpieces from their collection
The City Quilter - nice shop, steep prices but wonderful NYC related quilt fabric and other items - worth a visit. Say hello for me to Diane Rode Schneck, Judy Doenias or either of the the twins if any of them are there.
http://www.cityquilter.com
Wander around in the Garment District for a variety of shops selling trim, buttons etc.
Try Googling "fabric shops NYC" - there are several on the Lower East side and around 14th Street that have great, different fabrics that you don't find in LQSs.
Haven't lived in the City since '96 and can't remember the names or precise locations
It's a fabulous city; have a wonderful time