I have asked myself this question ... many times. I think its too big when you can not accomplish your quilting goals becasue the process of finding the fabric(s) in your stash is a project all in itself.... and you do not get on to your original goal of making quilts.
A second intidicator is that you purchase new because the process of finding fabric in your stash is just too overwhelming.
A third is that you often make the same purchase because you could not find or did not remember you had that same or very similiar fabric.
Fourth is that un-opened boxes/bags or opened but no place to put them are gathering.
I have taken the "oath" on fabric purchases since I realized that there have been times when a few of the above applied to me. A personal reality check was in order.