If it is causing you stress, it is too much. If it is causing financial burden, it is too much.
Maybe you just need a new organizational scheme. Spend a few days playing with it all.
And then, use it! No use having the most beautiful fabric in the world if it is only going to be tucked away in a closet forever.
If after organizing it you have too much and are overwhelmed, donate it or sell some of it. Get back to a manageable level.