It depends on what month you are making your travel plans...I would definitely spend a day in the Old Port, Portland Maine. There are a ton of wonderful restaurants and there is a cute quilt shop Z Fabrics.
If you are traveling in the summer months, you should entertain the idea of a clambake in Boothbay Harbor. A family owned business in it's 29th year but we only operate June 17th to September 11th.
www.cabbageislandclambakes.com
Camden is also a quaint little town as is Belfast...and both have quilt shops. : )