I feel that way too. Some weekends I plan so I have nothing to do but sew and other things just get in the way. I also cross stitch and get up so I can stitch about 1/2 every morning before I leave for work. Just having that needle in my hand is calming.
You can get alot done in segments, too. Don't think of it as needing hours to sew. Think of it in sections. Right now I'm working on potholders to better my machine quilting skills. I pieced them last weekend, one night cut the backing, batting, and insul-bright, and now they are waiting to be stitched. Even though I would have liked to get them done in one afternoon (the plan) if I had waited until I had the time to finish them I wouldn't have even started them!