None of my coworkers quilt but several of them do other things like scrapbooking, crochet, knitting, etc. So they "get it" when I babble about projects or bring something in to show them. I actually take my cross stitch projects in to work and use my lunch break to make progress. I guess I'm lucky with that.
On the other hand, I did get the "eye-roll" when I talked about my other hobby, which is Dagorhir (competitive medieval reenactments) and including some comments on "quitting now that you're an adult" etc. Incredibly annoying. But if I was hitting the bars every weekend instead, no one would blink an eye.