you won't be happy. That glue is NOT made for fabric. It will be gummy and will turn stiff and yellow over time.
Just cut a bit of fusible and save yourself the heart ache of using a product that is not meant for this application! OR even use a washable glue stick dab in the middle.