I've pondered over this for the past day and a half now and I think I've decided what I'll do. I'm going to print out some letters in a font I like, trace them on to fusible and applique those letters on to a new piece of banner. Applique that new banner over the old one, then cut the old one away. A little more work but I'm just worried that I'll spend even more time quilting on this cruddy banner, only to find out it didn't work to even the puckers out. Then I've really got a mess on my hands. It's at a fixable point now so I'm going to fix it.
Thanks everyone for your suggestions and help!