My thoughts would be, generally, no, but when you have the fabric and want to use it, sometimes that just spurns us on to give it all a try! Besides, many of us have used fleece (which is a knit) for a backing, so why should we say no?
Too, it may depend on what type of quilt you are making, and what results you would consider acceptable to you.
Anything is possible!!!! ........ you might want to do a trial quilt, where if things go wrong, you won't be too upset.
When I used fleece for my backing, I reversed my sandwich making order ..... the quilt top went against the table, then the fleece (no batting), on top, so I could be aware of any stretch I was getting into the fleece. It worked fine and several years later, I am quite pleased with the quilt.
Give it a try ... remember, anything is possible!