I purchase fabrics from many places. My local quilt shop has beautiful fabrics and I do buy certain lines from her. I also buy from Joanns , hobby lobby, and Walmart. When we travel, I will stop at mom and pop fabric shops and some quilt shops...I get "souvenirs" of our trips. I have found fantastic fabrics...wonderful quality...at great prices at both quilt shops and big box stores. And have found the opposite...low quality....high price. Actually stopped at a cute shop on one of our trips. Beautiful fabric but the prices were out of this world....one piece was 15.00 a yard. The woman was nasty when I questioned the price...of course she did not make a sale. Just down the road, the next town down, was a small Ben Franklin store (I thought they closed). There was the same beautiful fabric for 6.00 a yard. I got several yards of that and of others she had.
We all purchase where we can, and most seem to understand "high price" does not equate to "high quality", nor does " low price" mean " poor quality".
I am sure the swap is not requiring large amounts of fabric to be purchased by each participant...maybe your budget can allow a small amount to splurge and enjoy the participation in a swap.