You say" I'm sorry, it sounds like a fun project, but I'm just not going to be able to do that." Not, " I'm too busy right now" because it I plays that you MIGHT be someday. When pressed ( as lots of folks do!) you can say " No, really, I already have too much on my plate." Hopefully they'll get the message. If not, change the subject, they don't get any further explanation!