No, you are definitely not alone! I just posted not to long ago about this same thing. It was titled "How do you stay motivated" or something similar. I think it's just like any other "hobby", we don't feel like doing it ALL the time.
My solution: quilt when you feel like it, when you will enjoy it, that is the point of a hobby! If I force myself to do something, it usually doesn't turn out very nice, because I'm in a hurry.
On the other hand, sometimes it just takes me sewing a seam, and the next thing I know, hours have passed!
Do what feels right....