While I haven't finished the quilts yet, I have 4 WIPS using both cotton fabric and duponi silk.
Using interfacing is a must as you've been told. The only other thing I can think of is that once the interfacing is in place, there is no longer any "stretch". You know how when you are piecing and sometimes to make something fit you have to stretch just that hint of a 1/16 or 1/8? You can't as the interfacing gives it absolutely no movement.
No biggy, I only notice it as I'm working with the two different types of fabric on my quilts.
BTW, having both types gives it a wonderful look with the changes in texture.
HTH, Viv