I wear something very comfortable, just like you. I have many pairs of those pajama looking pants for winter, and shorts (baggy) with sleeveless t-shirts in spring/summer.
Who can sew for hours at a time if they are not totally comfortable? I sew in my bare feet, winter or summer.