Ex-pat here ;-) I haven't visited the Quilt Museum, but York is a beautiful, historic city that is definitely worth a visit in its own right. I love The Shambles, and the Railway Museum is always a must see for me. York is one of my favorite places and I don't think you will regret taking the time to go there.