Check out the quilts on Etsy, find some similar to yours and see what their prices are. At least you'll get a very general idea of what others are asking (key word is asking). The price range on Etsy goes from sublime to ridiculous.
http://www.etsy.com/listing-category...elevant&page=0
Some people triple the cost of materials, but that doesn't take the complexity of the design or techniques used into consideration at all. That may be the 'formula' you were thinking of.
As an aside, I hope your SIL is charging a more realistic commission percentage to non-relative consignment sellers. You are getting a VERY good deal at 20%.