A swap that comes in a manila or tyvek envelope will cost $3.80 while the same swap in a flat rate cardboard envelope costs $5.75. Unless you are mailing something that weighs more than 12 oz, ship by first class mail. It really will get there within a day or two of priority (if not the same). And Many post offices will not allow "fat" flat rate cardboard mailers to go at flat rate. they are supposed to be flat!
Even the free PO Priority tyvek envelopes will be cheaper than the flat rate. I have mailed a quilt top and back in the Priority envelope (not the cardboard one) for under $7 while a flat rate box is $12.65.