I would only think it's a problem if the smell won't come out! You, as a smoker, would not be able to tell. Why don't you whip me up a quilt, mail it to me, and I'll let you know. I'm a former smoker, so I can sniff out any problems!
By the way, you should quit. Your children are toooooo precious and my dad died from lung cancer when I was 12. My mom died when I was 7, and they never said it was directly related to smoking, but I think it was. That was enough for me to quit when I got pregnant. Okay, lecture is done for the day...I'm saying this with lots of love - not trying to be judgmental!! Okay?