I think you just shared too much information with your muggle co-worker. You could have just said, "I'm planning to stay home and do some sewing", or something to that effect, without going into any more detail. Just like I don't want to hear about anyone's "gaming" weekend or their vacation, neither do they really care what I'm doing on a sewing weekend.