I would not opt to do that if the quilt would ever be handled by the baby, just not worth the risk.
My husband when in grade school lost a button on his coat and rather than lose the button he put it in his mouth for safe keeping. Don't you know, someone bumped him and he swallowed the button. Had it not been for the fact it had large holes he would have died. As it was he almost lost his life.