You just missed the show, but there's plenty to see and do year round. The quilt museum has already been mentioned... they buy many of the winning quilts from the show every year, so it's great to still get to see them there.
Shop at Hancock's of Paducah for sure, but also at Eleanor Burn's Quilt in a Day store, too. I found lots of good bargains in there. The marked down bolts are all mixed in with the regular bolts, you just have to look for the painter's tape on the bolt ends. Some great prices!
The downtown area has lots of antique shops and antique malls to explore too.
For food, the famous Kirchoff's Bakery is not to be missed. They have a fried green tomato BLT that is to die for! Also a Tuscan Chicken sandwich I really loved. And their breads and treats are all baked right there. Pick up some cranberry walnut bread to take with you! The little coffee shop attached to them (Etc.) makes really good coffee drinks.
Right off the downtown is the Arts District, where you can stroll around and look at all the old houses and gardens. There's another Coffee Etc. shop there, too. And a restaurant called The Pork Peddler serves an amazing buffet for a reasonable price. The food is the best "home cooked" type food I've had in years. Everything on the buffet is great!
There are other museums around that I didn't have time to explore during the quilt show. There is the River Discovery Center, the Railroad Museum, The Market House Museum, and the Lloyd Tillghman House and Civil War Museum. Plenty of museums to check out!