I think it has something to do with old time religous beleives. Sunday was the day of rest and you were supposed to rest. I never quite understood that because Grandma, Mom and aunts in my family spent the whole day cooking dinner after church for the extended family and the only ones resting were the men as they watched TV. Then the kids got stuck washing all the dishes. I was never so happy to see a tradition fade.
I sew on Sundays if I can find the time. It's a hobby, not a job. I don't feel that quilting is work...now washing the dishes or cleaning the house, that's work and it can wait until Monday.