Oh my gosh don't quit. You have found something you enjoy. Life is too short to give up something you enjoy. So what if you don't have a lot of fabric. Ask people you know if they would give you their scraps. Everyone knows I love scraps and people in my guild give me the stuff they don't want to mess with. I cut them up and sort them and have made many quilts with them. I do have a large stash but hate seeing fabric tossed away. Do you belong to a quilt guild ask them to give you their scraps. A lot of quilters just toss stuff and you can get some good out of it. Yard sales sometimes have great deals. Let your friends know you are looking and they may find stuff for you. Stick with it.