I have used silk, cotton velvet, corduroy, cotton/poly, rayon, T-shirt knit material (stabilized, and not just in T-shirt quilts!), fleece (for applique, and for texture in paper-pieced critters), tulle, satin, and upholstery fabrics of many kinds.
The only problems I've had came from not adjusting seam allowances and stitch length enough when sewing fabrics together which have dramatically different weaves - when in doubt leave a larger seam allowance and use a smaller stitch. I always prewash everything, but I think even if you aren't normally a pre-washer you would want to test any unorthodox fabrics before including them.
Alison