a friend is trying to put button holes on a polyester fabric, she says her machine sews fine on the test sample (same fabric, thickness etc), but when she sews the garment, it appears good, but the thread pulls out.

she said it also sometimes jams up underneath, a big ball of thread

she says she has adjusted the top tension and is at a loss what else to do?

she may be using the wrong needle. i would try a ballpoint needle, but she may have to try something else. polys are strange. they all have a different hand. i would also use smaller stitches than normal. it doesn't sound like tension. i hope she's using interfacing, either iron-on or non-iron. i would pre-shrink that. poly doesn't shrink but interfacing does.

I agree on the interfacing. It's a must in this situation.

Is she pulling up the bobbin thread when she starts the buttonhole? If she does that and then holds both threads for the first few stitches it should help. Also, if she uses a stabilizing paper underneath the buttonhole, that should also help. I find that using gimp thread in the buttonhole foot usually helps. There's a little notch at the back of the buttonhole foot that you catch it on, then cross it at the front of the foot and secure it in the little hook that is there. Hope this helps.